A log retrieval method, device and equipment of a forwarding device

Abstract

The application discloses a log retrieval method, device and equipment of a forwarding device, acquires a retrieval keyword and a retrieval data source, divides the retrieval data source into a plurality of data sets first, processes each data set in batches, the data volume of each batch is small, and normal service traffic forwarding of the forwarding device is not affected. Then, according to the processor usage rate, processor load, memory usage rate of the forwarding device and the matching entry percentage of the last data set, the matching accuracy of the current data set is dynamically determined. The log text with the matching degree meeting the matching accuracy is retrieved from the current data set and the retrieval keyword. The log retrieval does not need to set an index, so that the log retrieval is more accurate while the performance of the forwarding device is not affected, and the user can analyze the running state of the forwarding device and events occurring in the network more conveniently.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a log retrieval method, apparatus, and device for a forwarding device. Background Technology

[0002] Firewalls and other online network forwarding devices store logs, which can be used to view and analyze the device's operational status and network events. Since the primary function of online network forwarding devices is traffic forwarding and processing, logs are not typically checked frequently during use.

[0003] Currently, logs are typically indexed during storage to allow users to view relevant logs by entering search keywords. While numerous indexes may improve log retrieval efficiency, they also lead to unnecessary resource waste and negatively impact the performance of forwarding devices for service traffic.

[0004] Therefore, how to retrieve logs when viewing logs by entering search keywords, without affecting the performance of business traffic forwarding, is a technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of this application provide a log retrieval method, apparatus, and device for a forwarding device, so as to achieve relatively accurate log retrieval without affecting the performance of service traffic forwarding.

[0006] To address the above problems, the technical solutions provided in this application are as follows:

[0007] A log retrieval method for a forwarding device, the method comprising:

[0008] Obtain the search keywords and search data source, wherein the search data source includes multiple log texts;

[0009] The retrieval data source is divided into multiple data sets;

[0010] Select one of the datasets as the current dataset;

[0011] The matching accuracy of the current data set is determined based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous data set.

[0012] Retrieve log texts from the current dataset whose matching degree with the search keywords matches the matching accuracy, and obtain the log texts retrieved from the current dataset;

[0013] The percentage of matching entries in the current data set is calculated based on the number of log texts retrieved from the current data set and the total number of log texts in the current data set.

[0014] Repeat the steps of selecting one of the datasets as the current dataset and subsequent steps until all the datasets have been selected, and output the retrieved log text.

[0015] In one possible implementation, determining the matching accuracy of the current data set based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous data set includes:

[0016] Calculate the target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device;

[0017] Calculate 1 minus the target parameter, and add the preset base value to obtain the initial matching accuracy of the current data set;

[0018] The matching accuracy of the current data set is obtained by adding the initial matching accuracy of the current data set to the percentage of matching entries of the previous data set.

[0019] Specifically, when a dataset is selected as the current dataset for the first time, the percentage of matching entries in the previous dataset is 0; when the initial matching accuracy of the current dataset is greater than 1, the initial matching accuracy of the current dataset is determined to be 1; when the matching accuracy of the current dataset is greater than 1, the matching accuracy of the current dataset is determined to be 1.

[0020] In one possible implementation, calculating the target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device includes:

[0021] Calculate the first weight by multiplying the processor utilization rate of the forwarding device, and then multiply by the processor load to obtain the first value;

[0022] Calculate the second weight by multiplying the memory usage rate to obtain the second value;

[0023] Calculate the sum of the first value and the second value to obtain the third value; calculate the sum of the first weight and the second weight to obtain the fourth value.

[0024] The target parameter is obtained by dividing the third value by the fourth value.

[0025] In one possible implementation, retrieving log text from the current dataset whose matching degree with the search keyword meets the matching accuracy requirement, to obtain the log text retrieved from the current dataset, includes:

[0026] Select one log text from the current data set as the current log text;

[0027] Determine whether there exists a text string in the current log text that matches the search keyword with the matching accuracy.

[0028] If so, the current log text is determined as the log text whose matching degree with the search keyword meets the matching accuracy.

[0029] Repeat the process of selecting a log text from the current data set as the current log text and subsequent steps until all log texts in the current data set have been selected, thus obtaining the log text retrieved from the current data set.

[0030] In one possible implementation, determining whether there exists a text string in the current log text that matches the search keyword with the matching accuracy includes:

[0031] Find the starting feature in the current log text, and start from the position corresponding to the starting feature to determine whether there is a text string in the current log text that matches the search keyword with the matching accuracy.

[0032] In one possible implementation, the method further includes:

[0033] When retrieving log texts from the current dataset that match the search keywords with the matching accuracy, for each log text retrieved, it is determined whether the number of retrieved log texts has reached a first threshold. If the first threshold is reached, the retrieval is stopped and the retrieved log texts are output.

[0034] And / or,

[0035] When retrieving log texts from the current dataset whose matching degree with the search keyword meets the matching accuracy, for each log text whose matching degree with the search keyword meets the matching accuracy, it is determined whether the number of retrieved log texts has reached a second threshold. If the second threshold is reached, the retrieval is stopped and the retrieved log texts are output.

[0036] In one possible implementation, the output of the retrieved log text includes:

[0037] Based on the set output parameters, select a preset number of log texts and / or log texts that meet the preset sorting position from the retrieved log texts for output.

[0038] A log retrieval device for a forwarding device, the device comprising:

[0039] The acquisition unit is used to acquire search keywords and search data sources, wherein the search data sources include multiple log texts;

[0040] A partitioning unit is used to divide the retrieval data source into multiple data sets;

[0041] The selection unit is used to select one of the datasets as the current dataset;

[0042] The determining unit is used to determine the matching accuracy of the current data set based on the processor utilization rate, processor load, memory utilization rate of the forwarding device, and the percentage of matching entries in the previous data set.

[0043] The retrieval unit is used to retrieve log texts from the current data set whose matching degree with the retrieval keyword meets the matching accuracy, and obtain the log texts retrieved from the current data set.

[0044] The calculation unit is used to calculate the percentage of matching entries in the current data set based on the number of log texts retrieved from the current data set and the total number of log texts in the current data set.

[0045] A triggering unit is used to return the selection unit to execute the selection of one of the datasets as the current dataset until all the datasets have been selected, and then trigger the output unit.

[0046] The output unit is used to output the retrieved log text.

[0047] A log retrieval device for a forwarding device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the log retrieval method for the forwarding device as described in any of the preceding claims.

[0048] A computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the log retrieval method of the forwarding device as described in any of the preceding claims.

[0049] Therefore, the embodiments of this application have the following beneficial effects:

[0050] This application's embodiments acquire search keywords and search data sources. First, the search data sources are divided into multiple data sets, and each data set is processed in batches. Each batch processes a small amount of data, ensuring no impact on the normal business traffic forwarding of the forwarding device. Then, based on the processor utilization, processor load, memory utilization, and the percentage of matching entries from the previous data set of the forwarding device, the matching accuracy of the current data set is dynamically determined. Log texts matching the search keywords with the required accuracy are retrieved from the current data set, thus ensuring relatively accurate log retrieval while maintaining the performance of the forwarding device. For the forwarding device, no indexing is required when storing logs, and a good user experience is provided when querying logs using keywords. Attached Figure Description

[0051] Figure 1 A schematic diagram illustrating an exemplary application scenario provided in this application embodiment;

[0052] Figure 2 A flowchart illustrating a log retrieval method for a forwarding device provided in this application embodiment;

[0053] Figure 3 A flowchart illustrating another log retrieval method for a forwarding device provided in this application embodiment;

[0054] Figure 4 This is a schematic diagram of a log retrieval device for a forwarding device provided in an embodiment of this application. Detailed Implementation

[0055] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0056] To facilitate understanding and explanation of the technical solutions provided in the embodiments of this application, the background technology of this application will be described first.

[0057] Firewalls and other online network forwarding devices store logs, which can be used to view and analyze the device's operational status and network events. Since the primary function of online network forwarding devices is traffic forwarding and processing, logs are not typically checked frequently during use.

[0058] Currently, logs are typically indexed during storage to allow users to view relevant logs by entering search keywords. While numerous indexes may improve log retrieval efficiency, they also lead to unnecessary resource waste and negatively impact the performance of forwarding devices for service traffic.

[0059] Therefore, how to perform fuzzy searches on logs when viewing logs by entering search keywords, without affecting the performance of business traffic forwarding, is a technical problem that needs to be solved.

[0060] Based on this, embodiments of this application provide a log retrieval method, apparatus, and device for forwarding devices. The method involves acquiring search keywords and search data sources, first dividing the search data sources into multiple data sets, and then processing each data set in batches. Each batch processes a small amount of data, minimizing impact on the normal business traffic forwarding of the forwarding device. Then, based on the processor utilization, processor load, memory utilization, and the percentage of matching entries from the previous data set, the matching accuracy of the current data set is dynamically determined. Log text matching the search keywords that meets the matching accuracy requirement is retrieved from the current data set. This log retrieval method eliminates the need for indexing, enabling accurate log retrieval without affecting the performance of the forwarding device, and facilitating user analysis of the forwarding device's operating status and network events.

[0061] To facilitate understanding of the log retrieval method for the forwarding device provided in the embodiments of this application, the following is combined with... Figure 1 The example scenario is shown below. See also... Figure 1 As shown in the figure, this figure is a schematic diagram of an exemplary application scenario provided in the embodiments of this application.

[0062] The search keywords and search data sources are obtained, and the search data sources are divided into multiple data sets. This embodiment takes dividing the search data sources into 3 data sets as an example. Each data set includes at least one log text.

[0063] Select dataset 1 as the current dataset. Determine the matching accuracy of dataset 1 based on the processor utilization, processor load, and memory utilization of the forwarding devices. Retrieve log text from dataset 1 that matches the search keyword with the required matching accuracy. For example, if the search keyword is "administrator" and the matching accuracy is 100%, then retrieve log text containing "administrator" from dataset 1. Then, calculate the percentage of matching entries for dataset 1 using the number of log texts retrieved and the total number of log texts in dataset 1, thus completing the processing for dataset 1.

[0064] Then, data set 2 is selected as the current data set. Based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in data set 1, the matching accuracy of data set 2 is determined. Log texts from data set 2 that match the search keywords with the required matching accuracy are retrieved, thus obtaining the log texts retrieved from data set 2. Finally, using the number of log texts retrieved from data set 2 and the total number of log texts in data set 2, the percentage of matching entries in data set 2 is calculated, completing the processing for data set 2.

[0065] Finally, dataset 3 is selected as the current dataset. Based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in dataset 2, the matching accuracy of dataset 3 is determined. Log texts matching the search keywords that meet the matching accuracy criteria are retrieved from dataset 3, thus obtaining the retrieved log texts. Then, using the number of retrieved log texts in dataset 3 and the total number of log texts in dataset 3, the percentage of matching entries in dataset 2 is calculated, completing the processing for dataset 3. Finally, the retrieved log texts from datasets 1, 2, and 3 are obtained, thus achieving log retrieval.

[0066] Those skilled in the art will understand that Figure 1 The schematic diagram shown is merely one example in which embodiments of this application can be implemented. The scope of application of the embodiments of this application is not limited by any aspect of this framework.

[0067] To facilitate understanding of this application, the following description, in conjunction with the accompanying drawings, illustrates a log retrieval method for a forwarding device provided by an embodiment of this application.

[0068] See Figure 2 As shown, this figure is a flowchart of a log retrieval method for a forwarding device provided in an embodiment of this application. Figure 2 As shown, the method may include S201-S207:

[0069] S201: Obtain the search keywords and search data source, which includes multiple log texts.

[0070] This application embodiment can be applied to any processor in a forwarding device. When a user needs to search the logs in the forwarding device, the system first obtains the user's input search keywords and the search data source. The search data source is usually logs filtered by information such as time, for example, logs generated between 8 PM and 10 PM on a specific day, month, and year. The search data source includes multiple log texts, which are usually composed of strings. Therefore, the search keywords are generally also strings, such as "administrator" or "user".

[0071] S202: Divide the retrieval data source into multiple data sets.

[0072] The retrieval data source is divided into multiple data sets, each containing at least one log text. The number of log texts in the retrieval data source is z, and the number of log texts in each data set is t. If z is divisible by t, the maximum number of batch processing rounds y is z / t. If z is not divisible by t, the maximum number of batch processing rounds y is z / t rounded down and then incremented by 1. Here, z and t are positive integers, and the value of t is determined based on program limitations or empirical values; this embodiment does not limit its value.

[0073] S203: Select a dataset as the current dataset.

[0074] When performing log retrieval, one dataset is selected at a time as the current dataset for processing. This results in a small batch size, preventing the data from competing for processor resources on the forwarding device and ensuring that the performance of the forwarding device for normal business traffic forwarding is not affected.

[0075] S204: Determine the matching accuracy of the current data set based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous data set.

[0076] Higher accuracy in log retrieval consumes more resources. Therefore, this embodiment dynamically adjusts the matching accuracy based on processor utilization, processor load, and memory utilization. Matching accuracy allows control over the accuracy of log retrieval. Higher processor utilization, processor load, or memory utilization results in lower matching accuracy, thus conserving resources. Furthermore, considering the impact of previous retrieval processes, the percentage of matching entries from the previous dataset is also taken into account when determining matching accuracy. A higher percentage of matching entries from the previous dataset indicates less accurate retrieval in the previous batch, thus requiring increased matching accuracy for the current dataset to improve log retrieval accuracy.

[0077] When a dataset is selected for the first time as the current dataset, the percentage of matching entries in the previous dataset is 0. At this time, the matching accuracy of the current dataset is determined based on the processor utilization, processor load, and memory utilization of the forwarding device. When a dataset is selected for the second time as the current dataset, the matching accuracy of the current dataset is determined based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous dataset.

[0078] The specific implementation process of S204 will be described in detail in subsequent embodiments of this application, and will not be repeated here.

[0079] S205: Retrieve log texts from the current dataset that match the search keywords with a matching accuracy, and obtain the log texts retrieved from the current dataset.

[0080] For any log text in the current dataset, if the match degree between the log text and the search keyword is greater than or equal to the match accuracy, then the log text is considered a log text that meets the match accuracy. For example, if the search keyword is "administrator" and the match accuracy is 100%, then if a log text contains "administrator," then that log text meets the match accuracy. As another example, if the search keyword is "administrator" and the match accuracy is 35%, then if a log text contains "admin," the match degree between "admin" and "administrator" is approximately 38%, which is greater than 35%, so that log text meets the match accuracy. Through these two examples, when the match accuracy is 100%, log text containing "admin" cannot be retrieved; when the match accuracy drops to 35%, log text containing "admin" can be retrieved. Therefore, it is clear that the lower the match accuracy, the lower the accuracy of log retrieval, and the higher the number of retrieved log texts, and vice versa.

[0081] In one possible implementation, S205 retrieves log text from the current dataset that matches the search keywords with a matching accuracy. The specific implementation of obtaining the retrieved log text from the current dataset may include:

[0082] A1: Select one log text from the current dataset as the current log text.

[0083] A2: Determine if there exists a text string in the current log text that matches the search keywords with a matching accuracy.

[0084] A3: If so, identify the current log text as the log text whose matching accuracy matches the search keywords.

[0085] A4: Repeat the process of selecting one log text from the current data set as the current log text and subsequent steps until all log texts in the current data set have been selected, thus obtaining the log text retrieved from the current data set.

[0086] By determining the degree of match between each log text in the current dataset and the search keywords, the log text retrieved from the current dataset can be obtained.

[0087] To improve matching efficiency, step A2, which determines whether a text string exists in the current log text that matches the search keyword with a matching accuracy, may include the following specific implementation:

[0088] Find the starting feature in the current log text, and start from the position corresponding to the starting feature to determine whether there is a text string in the current log text that matches the search keyword with the matching accuracy.

[0089] For the current log text, a starting feature can be matched first. For example, if the starting feature is "|", then the content before "|" in the log text is considered invalid information. The judgment starts directly from the starting feature, which effectively improves efficiency and saves the resources occupied by the forwarding device. The starting feature is related to the product type of the forwarding device and can be set according to the actual observation of different forwarding devices.

[0090] S206: Calculate the percentage of matching entries in the current dataset based on the number of log texts retrieved from the current dataset and the total number of log texts in the current dataset.

[0091] Dividing the number of log texts retrieved in the current dataset by the total number of log texts in the current dataset gives the percentage of matching entries for that dataset. The more log texts retrieved in the current dataset, the higher the percentage of matching entries, indicating that the log retrieval is less accurate and may retrieve many log texts that do not meet the user's expectations. Therefore, the matching accuracy needs to be improved when selecting a dataset next time.

[0092] S207: Repeat S203 to select a dataset as the current dataset and follow up with subsequent steps until all datasets have been selected, and output the retrieved log text.

[0093] Return to S203 to select the next dataset as the current dataset, and continue to execute steps S204-S206 to complete the processing of the next batch of datasets. This process continues until all datasets have been selected, and the log text that has been retrieved from each dataset can be output.

[0094] Since the number of retrieved log texts may be large, users can set relevant output parameters to extract a portion for viewing. Therefore, one possible implementation for outputting the retrieved log texts could include:

[0095] Based on the set output parameters, select a preset number of log texts and / or log texts that meet the preset sorting position from the retrieved log texts for output.

[0096] Based on user-defined output parameters, a preset number of log texts can be selected for output, such as selecting 100 log texts from the retrieved logs. Alternatively, log texts can be selected according to a preset sorting position, such as starting output from the 50th retrieved log text. Furthermore, a preset number of log texts with a preset sorting position can be selected for output, such as selecting 100 log texts starting from the 50th retrieved log text. This allows users to understand the operational status of the forwarding device by randomly viewing the logs.

[0097] Based on the descriptions in S201-S207, this embodiment of the application obtains search keywords and search data sources. First, the search data sources are divided into multiple data sets, and each data set is processed in batches. The amount of data processed in each batch is small, so as not to affect the normal business traffic forwarding of the forwarding device. Then, based on the processor utilization, processor load, memory utilization, and the percentage of matching entries in the previous data set of the forwarding device, the matching accuracy of the current data set is dynamically determined. Log texts matching the search keywords with the matching accuracy are retrieved from the current data set. Log retrieval does not require setting an index, thus enabling relatively accurate log retrieval without affecting the performance of the forwarding device, making it easier for users to analyze the operating status of the forwarding device and events occurring in the network.

[0098] In one possible implementation, the specific implementation of S204 determining the matching accuracy of the current data set based on the processor utilization, processor load, memory utilization, and the percentage of matching entries in the previous data set may include:

[0099] B1: Calculate the target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device.

[0100] B2: Calculate 1 minus the target parameter, and add the preset base value to obtain the initial matching accuracy of the current data set.

[0101] In this embodiment of the application, the formula b = (100% - a) + k can be used. base Calculate the initial matching accuracy b for the current dataset. Where a is the target parameter, k... base Using this as a base value, in practical applications, 1 can also be used as 100%. base The value can be obtained through actual experiments based on the type, model and hardware information of the forwarding device. This application does not limit its value.

[0102] B3: Calculate the initial matching accuracy of the current dataset by adding the percentage of matching entries from the previous dataset to obtain the matching accuracy of the current dataset.

[0103] The matching accuracy d of the current data set can be calculated using the formula d = b + c. Here, b is the initial matching accuracy of the current data set, and c is the percentage of matching entries in the previous data set. In practical applications, the percentage of matching entries in the previous data set can also be corrected, and the matching accuracy d of the current data set can be calculated using the formula d = b + c * k3. Here, k3 is a correction coefficient, and the value of k3 can be obtained through actual experiments based on the type, model, and hardware information of the forwarding device. This application embodiment does not limit its value.

[0104] In one possible implementation, the specific implementation of step B1, which calculates the target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device, may include:

[0105] Calculate the first weight by multiplying the processor utilization rate of the forwarding device, and then multiply by the processor load to obtain the first value;

[0106] Calculate the second weight by multiplying the memory usage rate to obtain the second value;

[0107] Calculate the sum of the first value and the second value to obtain the third value; calculate the sum of the first weight and the second weight to obtain the fourth value.

[0108] Calculate the third value and divide it by the fourth value to obtain the target parameter.

[0109] That is, in the embodiments of this application, the target parameter can be calculated using the following formula:

[0110] a=(k1*x cpu *x cpu_load +k2*x mem ) / (k1+k2)*100%

[0111] Where, x cpu To determine processor utilization, when a forwarding device includes multiple processors and has a clearly defined forwarding processor, a weight can be assigned to each processor. The weight of the forwarding processor is higher than that of other processors. The utilization of each processor is then weighted and averaged to obtain x. cpu When the forwarding device includes multiple processors, but the forwarding processor is not explicitly defined, the average utilization rate of each processor is taken to obtain x. cpu .

[0112] x mem The memory utilization rate is calculated by subtracting the reserved memory from both the memory usage and total memory, and then dividing the result by the reserved memory to obtain x. mem If the forwarding device system does not reserve memory for other programs, then the calculated content usage divided by the total memory usage gives x.mem .

[0113] x cpu_load This refers to the processor load, for example, the average load of each processor in the forwarding device over 15 minutes.

[0114] k1 is the first weight, k2 is the second weight, and a is the target parameter. In practical applications, a can be converted to a percentage.

[0115] The values ​​of k1 and k2 can be obtained through actual experiments based on the type, model and hardware information of the forwarding device. The embodiments of this application do not limit their values.

[0116] In the above calculation process, when the initial matching accuracy of the current data set is greater than 1 (or 100%), the initial matching accuracy of the current data set is determined to be 1 (or 100%).

[0117] In this embodiment, the matching accuracy of the current data set is dynamically determined based on the processor utilization, processor load, memory utilization, and the percentage of matching entries in the previous data set of the forwarding device, thereby enabling relatively accurate log retrieval without affecting the performance of business traffic forwarding.

[0118] To further reduce the resource consumption of log retrieval based on the above embodiments, in one possible implementation, the log retrieval method for the forwarding device provided in this application embodiment may further include:

[0119] When retrieving log texts from the current dataset that match the search keywords with a matching accuracy, for each log text retrieved, it is determined whether the number of retrieved log texts has reached the first threshold. If the first threshold is reached, the retrieval is stopped and the retrieved log texts are output.

[0120] And / or,

[0121] When retrieving log texts from the current dataset that match the search keywords with a matching accuracy, for each log text that matches the search keywords with a matching accuracy, determine whether the number of retrieved log texts has reached the second threshold. If the second threshold is reached, stop the search and output the retrieved log texts.

[0122] In other words, users can set a first threshold corresponding to the number of log texts that have been retrieved, and / or a second threshold corresponding to the number of log texts that have been retrieved. When the number of log texts that have been retrieved reaches the first threshold, and / or the number of log texts that have been retrieved reaches the second threshold, the retrieval will stop directly and the currently retrieved log texts will be output.

[0123] When the number of logs required by the user is small, the above process can be used to interrupt log retrieval in a timely manner, minimizing the impact of log retrieval on the forwarding performance of the forwarding device.

[0124] See Figure 3 The diagram shows a flowchart of a log retrieval method for a forwarding device in a practical application. This method may include:

[0125] S301: Obtain relevant parameters.

[0126] When users need to retrieve logs, they can input relevant parameters. In practical applications, these parameters can be divided into required and optional fields. If a required field is empty, the system will prompt the user to enter the required parameter information and return the result. If the required field is valid, default values ​​will be assigned to optional fields that are not filled in.

[0127] The main relevant parameters include: whether the number of retrieved log texts has reached the first threshold f1, whether the number of retrieved log texts has reached the second threshold f2, the search keyword msg (required), the search data source S (required), the preset number of output log texts limit, and the preset sorting position offset of the output log texts offset.

[0128] S302: Based on the retrieved data source S, obtain the maximum batch processing number y. If y <= 0, output empty and end directly; otherwise, divide the retrieved data source into multiple data sets, with batch i = 0 for each data set.

[0129] S303: Determine the matching accuracy of the current data set based on the processor utilization, processor load, and memory utilization of the forwarding device.

[0130] S304: i = i + 1, select a dataset based on batch i of the dataset as the current dataset.

[0131] S305: The sequence number of a single log text in the current dataset is index = 1.

[0132] S306: If a log text can be obtained based on the index, execute S307; otherwise, execute S313.

[0133] S307: If the current log text matches the starting feature, execute S308; otherwise, index = index + 1, and then proceed to S306.

[0134] S308: Use the search keyword msg to match the current log text, and dynamically check the accuracy of the match during the matching process.

[0135] S309: If the matching accuracy is achieved, record this log text in the output data D; the number of log texts retrieved is c1 = c1 + 1; the number of log texts retrieved is c2 = c2 + 1. If the matching accuracy is not achieved, the number of log texts retrieved is c2 = c2 + 1. The initial values ​​of D, c1, and c2 are empty, 0, and 0, respectively.

[0136] S310: If “number of log texts retrieved c1” >= “second threshold f2” or “number of log texts retrieved c2” >= “first threshold f1”, proceed to S313; otherwise, execute S311.

[0137] S311: If “log text sequence number index” >= “number of log texts t included in each data set”, execute S312; otherwise, index = index + 1, and then go to S306.

[0138] S312: If "batch i of data set" >= "maximum number of rounds y for data batch processing", proceed to S313; otherwise, calculate the percentage of matching entries in the current data set. Use the percentage of matching entries in the current data set as the percentage of matching entries in the previous data set, and re-determine the matching accuracy of the current data set based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous data set, then proceed to S304.

[0139] S313: Output the retrieved log text based on the output data D, offset, and limit.

[0140] Through the above process, this embodiment of the application achieves relatively accurate log retrieval while ensuring the performance of the forwarding device. For the forwarding device, no index needs to be set when storing logs, and a good user experience can be obtained when searching logs by entering keywords.

[0141] Based on the log retrieval method for a forwarding device provided in the above-described method embodiments, this application also provides a log retrieval device for a forwarding device, which will be described below with reference to the accompanying drawings.

[0142] See Figure 4As shown in the figure, this is a schematic diagram of the structure of a log retrieval device for a forwarding device provided in an embodiment of this application. Figure 4 As shown, the log retrieval device of the forwarding device includes:

[0143] The acquisition unit 401 is used to acquire search keywords and search data sources, wherein the search data sources include multiple log texts;

[0144] The partitioning unit 402 is used to divide the retrieval data source into multiple data sets;

[0145] Selection unit 403 is used to select one of the datasets as the current dataset;

[0146] The determining unit 404 is used to determine the matching accuracy of the current data set based on the processor utilization rate, processor load, memory utilization rate of the forwarding device and the percentage of matching entries in the previous data set;

[0147] The retrieval unit 405 is used to retrieve log texts from the current data set whose matching degree with the retrieval keyword meets the matching accuracy, and obtain the log texts retrieved from the current data set;

[0148] The calculation unit 406 is used to calculate the percentage of matching entries in the current data set based on the number of log texts retrieved from the current data set and the number of log texts in the current data set.

[0149] Trigger unit 407 is used to return to the selection unit to execute the selection of one of the datasets as the current data set until all the data sets have been selected, and then trigger the output unit;

[0150] The output unit is used to output the retrieved log text.

[0151] In one possible implementation, the determining unit includes:

[0152] The first calculation subunit is used to calculate the target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device;

[0153] The second calculation subunit is used to calculate 1 minus the target parameter and add the preset base value to obtain the initial matching accuracy of the current data set;

[0154] The third calculation subunit is used to calculate the initial matching accuracy of the current data set plus the percentage of matching entries of the previous data set to obtain the matching accuracy of the current data set.

[0155] Specifically, when a dataset is selected as the current dataset for the first time, the percentage of matching entries in the previous dataset is 0; when the initial matching accuracy of the current dataset is greater than 1, the initial matching accuracy of the current dataset is determined to be 1; when the matching accuracy of the current dataset is greater than 1, the matching accuracy of the current dataset is determined to be 1.

[0156] In one possible implementation, the first computational subunit includes:

[0157] The fourth calculation subunit is used to calculate the first weight multiplied by the processor utilization rate of the forwarding device, and then multiplied by the processor load to obtain the first value;

[0158] The fifth calculation subunit is used to calculate the second weight multiplied by the memory usage rate to obtain the second value;

[0159] The sixth calculation subunit is used to calculate the sum of the first value and the second value to obtain the third value, and to calculate the sum of the first weight and the second weight to obtain the fourth value;

[0160] The seventh calculation subunit is used to calculate the third value divided by the fourth value to obtain the target parameter.

[0161] In one possible implementation, the retrieval unit includes:

[0162] Select a sub-unit, used to select a log text from the current data set as the current log text;

[0163] The judgment subunit is used to determine whether there is a text string in the current log text that matches the search keyword with the matching accuracy.

[0164] A determining subunit is used to determine the current log text as a log text whose matching degree with the search keyword matches the matching accuracy if the judgment result of the determining subunit is yes;

[0165] The trigger subunit is used to return the selection subunit to perform the selection of a log text from the current data set as the current log text, until all log texts in the current data set have been selected, thus obtaining the log text retrieved from the current data set.

[0166] In one possible implementation, the judgment subunit is specifically used for:

[0167] Find the starting feature in the current log text, and start from the position corresponding to the starting feature to determine whether there is a text string in the current log text that matches the search keyword with the matching accuracy.

[0168] In one possible implementation, the device further includes:

[0169] The first judgment unit is used to retrieve log texts from the current data set whose matching degree with the search keyword meets the matching accuracy. For each log text retrieved, it determines whether the number of log texts already retrieved has reached a first threshold. If the first threshold is reached, the retrieval is stopped and the output unit is triggered.

[0170] And / or,

[0171] The second judgment unit is used to retrieve log texts from the current data set whose matching degree with the search keyword meets the matching accuracy. For each log text whose matching degree with the search keyword meets the matching accuracy, the unit determines whether the number of retrieved log texts has reached a second threshold. If the second threshold is reached, the retrieval is stopped and the output unit is triggered.

[0172] In one possible implementation, the output unit is specifically used for:

[0173] Based on the set output parameters, select a preset number of log texts and / or log texts that meet the preset sorting position from the retrieved log texts for output.

[0174] In addition, this application embodiment also provides a log retrieval device for a forwarding device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the log retrieval method for the forwarding device as described in any of the above claims.

[0175] In addition, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the log retrieval method of the forwarding device as described in any of the preceding claims.

[0176] This application's embodiments acquire search keywords and search data sources. First, the search data sources are divided into multiple data sets, and each data set is processed in batches. Each batch processes a small amount of data, ensuring no impact on the normal business traffic forwarding of the forwarding device. Then, based on the processor utilization, processor load, memory utilization, and the percentage of matching entries from the previous data set of the forwarding device, the matching accuracy of the current data set is dynamically determined. Log texts matching the search keywords with the required accuracy are retrieved from the current data set, thus ensuring relatively accurate log retrieval while maintaining the performance of the forwarding device. For the forwarding device, no indexing is required when storing logs, and a good user experience is provided when querying logs using keywords.

[0177] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0178] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0179] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0180] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0181] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A log retrieval method for a forwarding device, characterized in that, The method includes: Obtain the search keywords and search data source, wherein the search data source includes multiple log texts; The retrieval data source is divided into multiple data sets; Select one of the datasets as the current dataset; The matching accuracy of the current data set is determined based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous data set. Retrieve log texts from the current dataset whose matching degree with the search keywords matches the matching accuracy, and obtain the log texts retrieved from the current dataset; The percentage of matching entries in the current data set is calculated based on the number of log texts retrieved from the current data set and the total number of log texts in the current data set. Repeat the steps of selecting one of the datasets as the current dataset and subsequent steps until all the datasets have been selected, and output the retrieved log text.

2. The method according to claim 1, characterized in that, The step of determining the matching accuracy of the current data set based on the processor utilization, processor load, memory utilization of the forwarding device, and the percentage of matching entries in the previous data set includes: Calculate the target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device; Calculate 1 minus the target parameter, and add the preset base value to obtain the initial matching accuracy of the current data set; The matching accuracy of the current data set is obtained by adding the initial matching accuracy of the current data set to the percentage of matching entries of the previous data set. Specifically, when a dataset is selected as the current dataset for the first time, the percentage of matching entries in the previous dataset is 0; when the initial matching accuracy of the current dataset is greater than 1, the initial matching accuracy of the current dataset is determined to be 1; when the matching accuracy of the current dataset is greater than 1, the matching accuracy of the current dataset is determined to be 1.

3. The method according to claim 2, characterized in that, The calculation of target parameters based on the processor utilization, processor load, and memory utilization of the forwarding device includes: Calculate the first weight by multiplying the processor utilization rate of the forwarding device, and then multiply by the processor load to obtain the first value; Calculate the second weight by multiplying the memory usage rate to obtain the second value; Calculate the sum of the first value and the second value to obtain the third value; calculate the sum of the first weight and the second weight to obtain the fourth value. The target parameter is obtained by dividing the third value by the fourth value.

4. The method according to claim 1, characterized in that, The step of retrieving log text from the current dataset that matches the search keyword with the specified matching accuracy, to obtain the log text retrieved from the current dataset, includes: Select one log text from the current data set as the current log text; Determine whether there exists a text string in the current log text that matches the search keyword with the matching accuracy. If so, the current log text is determined as the log text whose matching degree with the search keyword meets the matching accuracy. Repeat the process of selecting a log text from the current data set as the current log text and subsequent steps until all log texts in the current data set have been selected, thus obtaining the log text retrieved from the current data set.

5. The method according to claim 4, characterized in that, The step of determining whether there exists a text string in the current log text that matches the search keyword with the required matching accuracy includes: Find the starting feature in the current log text, and start from the position corresponding to the starting feature to determine whether there is a text string in the current log text that matches the search keyword with the matching accuracy.

6. The method according to claim 1 or 4, characterized in that, The method further includes: When retrieving log texts from the current dataset that match the search keywords with the matching accuracy, for each log text retrieved, it is determined whether the number of retrieved log texts has reached a first threshold. If the first threshold is reached, the retrieval is stopped and the retrieved log texts are output. And / or, When retrieving log texts from the current dataset whose matching degree with the search keyword meets the matching accuracy, for each log text whose matching degree with the search keyword meets the matching accuracy, it is determined whether the number of retrieved log texts has reached a second threshold. If the second threshold is reached, the retrieval is stopped and the retrieved log texts are output.

7. The method according to claim 1, characterized in that, The output includes the retrieved log text, including: Based on the set output parameters, select a preset number of log texts and / or log texts that meet the preset sorting position from the retrieved log texts for output.

8. The method according to claim 6, characterized in that, The output includes the retrieved log text, including: Based on the set output parameters, select a preset number of log texts and / or log texts that meet the preset sorting position from the retrieved log texts for output.

9. A log retrieval device for a forwarding device, characterized in that, The device includes: The acquisition unit is used to acquire search keywords and search data sources, wherein the search data sources include multiple log texts; A partitioning unit is used to divide the retrieval data source into multiple data sets; The selection unit is used to select one of the datasets as the current dataset; The determining unit is used to determine the matching accuracy of the current data set based on the processor utilization rate, processor load, memory utilization rate of the forwarding device, and the percentage of matching entries in the previous data set. The retrieval unit is used to retrieve log texts from the current data set whose matching degree with the retrieval keyword meets the matching accuracy, and obtain the log texts retrieved from the current data set. The calculation unit is used to calculate the percentage of matching entries in the current data set based on the number of log texts retrieved from the current data set and the total number of log texts in the current data set. A triggering unit is used to return the selection unit to execute the selection of one of the datasets as the current dataset until all the datasets have been selected, and then trigger the output unit. The output unit is used to output the retrieved log text.

10. A log retrieval device for a forwarding device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the log retrieval method of the forwarding device as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the log retrieval method for a forwarding device as described in any one of claims 1-8.

Images