Log analysis method and device

Abstract

The embodiment of the present invention discloses a log analysis method and device, which are applied to the field of information processing technology. In the method of this embodiment, the log analysis device divides the acquired development log into multiple log segments according to the time interval, and then classifies the problems of the application corresponding to each log segment according to the preset log analysis strategy, and finally determines the processing information based on the application in the corresponding time period of the application development according to the obtained multiple problem classification results. In this way, by analyzing the multiple log segments included in the development log, and each log segment corresponds to a time interval, it is possible to accurately locate which development process will have problems during the development of the application, and then perform corresponding processing, thereby improving the development efficiency of the application.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a log analysis method and device. Background Art

[0002] The existing log analysis method mainly analyzes the operation logs of the terminal device during the application program running, so as to obtain the failures of the terminal device running the application program, or the loopholes of the application program in the actual application process.

[0003] Specifically, an existing software defect fault identification method based on Internet log data mainly targets Internet source system log data and user system source log data, uses Internet source system log data as a training set and extracts features from it, generates a software defect fault log identification prediction model through machine learning or similarity matching; for user system source log data, analyzes and identifies the log fragments that characterize software defect faults, thereby obtaining the software defect fault type for the user system log. However, if there is a large amount of user system source log data, it takes a long time to locate the fault using the existing software defect fault identification method based on Internet log data. Summary of the invention

[0004] The embodiment of the present invention provides a log analysis method and device, which realizes determining the processing information based on the application program in the corresponding time period of the application program through the log fragments of the development log.

[0005] A first aspect of an embodiment of the present invention provides a log analysis method, including:

[0006] Get the application's development log;

[0007] Dividing the development log into multiple log segments according to time intervals;

[0008] According to the preset log analysis strategy, the problems of the application corresponding to each log segment are classified respectively to obtain multiple problem classification results;

[0009] According to the plurality of problem classification results, processing information based on the application program in a corresponding time period of the application program development is determined.

[0010] In a specific implementation of a system analysis method provided by the first aspect of an embodiment of the present invention, dividing the development log into multiple log segments according to time intervals specifically includes:

[0011] According to the time interval, the development log is divided into multiple groups of log segments, each group of log segments includes multiple log fragments.

[0012] A second aspect of an embodiment of the present invention provides a log analysis method, including:

[0013] A log acquisition unit, used to acquire the development log of the application;

[0014] A division unit, used for dividing the development log into a plurality of log segments according to time intervals;

[0015] A classification unit is used to classify the problems of the application corresponding to each log segment according to a preset log analysis strategy, and obtain multiple problem classification results;

[0016] The processing determination unit is used to determine processing information based on the application program within a corresponding time period of application program development according to the multiple problem classification results.

[0017] A third aspect of an embodiment of the present invention provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the log analysis method as described in the first aspect of the embodiment of the present invention.

[0018] A fourth aspect of an embodiment of the present invention provides a server, comprising: a processor and a storage medium, wherein the processor is used to implement various instructions; the storage medium is used to store multiple instructions, and the instructions are used to be loaded by the processor and execute the log analysis method described in the first aspect of the embodiment of the present invention.

[0019] It can be seen that in the method of this embodiment, the log analysis device will divide the acquired development log into multiple log segments according to the time interval, and then classify the problems of the application corresponding to each log segment according to the preset log analysis strategy, and finally determine the processing information based on the application in the corresponding time period of application development according to the obtained multiple problem classification results. In this way, by analyzing the multiple log segments included in the development log, and each log segment corresponds to a time interval, it is possible to accurately locate which development process will have problems during the development of the application, and then perform corresponding processing, thereby improving the development efficiency of the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 is a structural schematic diagram of a system to which a log analysis method provided by an embodiment of the present invention is applied;

[0022] Figure 2 is a flow chart of a log analysis method provided by an embodiment of the present invention;

[0023] Figure 3a is a schematic diagram of dividing a development log in one embodiment of the present invention;

[0024] Figure 3b is a schematic diagram of another method of dividing a development log in one embodiment of the present invention;

[0025] Figure 4 is a flow chart of a method for training a log analysis model in one embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of a log analysis method provided by an application embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of preprocessing a development log in an application embodiment of the present invention;

[0028] Figure 7 It is a structural schematic diagram of a log analysis device provided by an embodiment of the present invention;

[0029] Figure 8 It is a structural diagram of a server provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The embodiment of the present invention provides a log analysis method, which can be mainly applied to Figure 1 The system shown includes: a log analysis device, which is mainly used to obtain the development log of the application program and analyze the obtained development log to determine the processing information based on the application program.

[0033] Specifically, the system may also include a log terminal, wherein the log terminal refers to a terminal that generates a development log of an application program, and is mainly a terminal used by application developers in the process of developing the application program, such as a terminal for developers to edit application code, a terminal for debugging code, and a terminal for printing information, etc. During the development of the application program, these log terminals will record the operations performed by the log terminal at various time points and the log information at various time points.

[0034] Furthermore, the system may also include: a processing information receiving terminal, which is mainly used to receive application-based processing information distributed by the log analysis device. Specifically, it can be any terminal that receives information, such as an email application terminal, or a terminal of an application development user (such as the above-mentioned log terminal, etc.).

[0035] In this embodiment, the log analysis device can perform log analysis through the following steps:

[0036] Obtain a development log of an application; divide the development log into multiple log segments according to time intervals (n log segments are taken as an example in the figure); classify the problems of the application corresponding to each log segment according to a preset log analysis strategy to obtain multiple problem classification results; determine processing information based on the application within a corresponding time period of application development according to the multiple problem classification results.

[0037] In this way, by analyzing the multiple log segments included in the development log, and each log segment corresponds to a time interval, it is possible to accurately locate which development processes in the application development process will have problems, and then perform corresponding processing, thereby improving the development efficiency of the application.

[0038] An embodiment of the present invention provides a log analysis method, which is mainly a method performed by the above-mentioned log analysis device, and the flow chart is as follows: Figure 2 As shown, including:

[0039] Step 101, obtaining the development log of the application.

[0040] It can be understood that the log analysis device can directly initiate the process of this embodiment according to a certain period. In this way, in one case, when the log analysis device initiates the process of this embodiment, it can obtain the development log of the application between the previous process initiation cycle and the current cycle from the log terminal, specifically including: log information at each time point within this time period.

[0041] In another case, after generating the development log, the log terminal will actively report it to the log analysis device for storage. When the log analysis device initiates the process of this embodiment, it can directly extract the development log between the previous process initiation cycle and the current cycle from the local storage.

[0042] The development log in this embodiment refers to the log information generated by the log terminal during the process of the application development user developing the application through the log terminal, such as the log terminal editing the application code log, debugging the code log, and printing information log.

[0043] Step 102: divide the development log into multiple log segments according to time intervals.

[0044] Specifically, the log analysis device can divide the development log in the following two ways, but not limited to:

[0045] (1) The preset maximum time interval is used as the time window of the development log; when the time window moves along the development log, the log information in the time window is used as a log segment, wherein the step length of the time window moving along the development log is consistent with the preset minimum time interval. Usually, a problem based on an application often occurs at a certain moment, and the related log information is printed a short time after that moment. Therefore, the time interval for dividing the development log can be the minimum time of the problem log, or the maximum time of the problem log, etc.

[0046] For example Figure 3a As shown, the preset maximum time interval Tmax is used as the time window of the development log. The time window will move along the development log, and the step length of the movement is the preset minimum time interval Tmin. The arrow indicates the direction in which the time window moves along the development log. When the time window is at a certain position in the development log, the log information in the time window is log fragment 1; when the time window moves along the development log after the minimum time interval Tmin, the log information in the time window is log fragment 2, and so on, until the time window moves to the last fragment of the development log, and the last fragment is used as another log fragment.

[0047] (2) The log analysis device can divide the development log into multiple rounds according to time intervals, and each round of division can obtain a group of log segments. In this way, the log analysis device will divide the development log into multiple groups of log segments according to time intervals, and each group of log segments includes multiple log fragments.

[0048] In a specific embodiment, the log analysis device may first divide the development log into a group of log segments according to a preset maximum time interval; then delete the log information within the preset minimum time interval in the development log, and divide the development log after deletion into another group of log segments according to the preset maximum time interval. Furthermore, the log analysis device may also delete the log information within the preset minimum time interval in the development log after deletion, and divide the development log after deletion into another group of log segments according to the preset maximum time interval; then cyclically execute the steps of deleting and dividing for the development log after deletion. The preset maximum time interval may be the maximum time of the problem log, and the preset minimum time interval may be the minimum time of the problem log. The time interval of each log segment in each group of log segments is equal to or less than the preset maximum time interval.

[0049] For example Figure 3b The development log shown includes log information corresponding to T moments, the preset minimum time interval may be Tmin, and the maximum time interval may be Tmax.

[0050] In the first division, the log information of T moments is divided into three log segments with a time interval of Tmax and a log segment with a time interval of T1, wherein T1 is less than Tmax; in the second division, the log information with a time interval of Tmin is deleted from the beginning of the development log, and the log information of T-Tmin moments is divided into three log segments with a time interval of Tmax and a log segment with a time interval of T2, wherein T2 is less than Tmax; in the third division, the log information with a time interval of Tmin is deleted from the beginning of the log information of T-Tmin moments, and the log information of T-2Tmin moments is divided into two log segments with a time interval of Tmax and a log segment with a time interval of T3, wherein T3 is less than Tmax. In this way, three groups of log segments can be formed, wherein log segment group 1 and log segment group 2 each include four log segments, and log segment group 3 includes three log segments.

[0051] Step 103 , classifying the problems of the application corresponding to each log segment according to a preset log analysis strategy, and obtaining a plurality of problem classification results.

[0052] Specifically, the log analysis device can first obtain the key tags of each log fragment respectively; then, according to the key tags of each log fragment and the preset log analysis strategy, classify the problems of the application corresponding to each log fragment respectively, and obtain multiple problem classification results, each problem classification result corresponds to a log fragment.

[0053] Among them, since the log fragment of the development log is a data in a mixed data format, including structured data and unstructured data, if a log fragment includes structured data, the key tags of the log fragment include the tags contained in the structured data, such as system tags, log levels (errors, warnings, debug, etc.), or definition tags, etc.; if a log fragment includes unstructured data, the key tags of the log fragment are mainly obtained through keyword extraction. Specifically, the unstructured data can be segmented first to obtain multiple segmentations, and then some of the multiple segmentations can be determined as key tags.

[0054] The problem classification result corresponding to any of the above log segments may include: the problem type of the application corresponding to the log segment, which may be a fault type or a vulnerability (BUG) type, etc., or may be other types without faults or vulnerabilities.

[0055] The above-mentioned preset log analysis strategy can be the operating logic of the log analysis model, so that the log analysis model can directly output the problem classification results through the key tags of the log fragment; the above-mentioned preset log analysis strategy can also be a similarity calculation strategy, so that the log analysis device can calculate the similarity between the log fragment and the preset log information of various problem types. When the similarity between the log fragment and the log information of a certain problem type is greater than a certain threshold, it is determined that the problem classification result of the log fragment includes the problem type.

[0056] Step 104, determining processing information based on the application program within a corresponding time period when the application program is opened according to the multiple problem classification results.

[0057] Since a problem classification result corresponds to a log segment, and a log segment corresponds to a time interval, a problem classification result can represent a failure or vulnerability that occurs in an application during a certain period of development. When executing step 104, the log analysis device can provide a solution within the corresponding period of application development for each problem classification result, that is, based on the processing information of the application.

[0058] Specifically, the correspondence between the problem type and the processing information based on the application program is preset in the log analysis device, so that when determining the processing information, the problem classification result obtained in the above step 103 is matched with the preset correspondence, so that the final processing information based on the application program can be obtained. Among them, for the problem classification result of the fault type or vulnerability type, the corresponding processing information can be a fault solution or a vulnerability solution; for the problem classification result of non-fault or vulnerability type, the corresponding processing information can be: sending a notification to the terminal of the user who developed the application program, etc.

[0059] It should be noted that if multiple groups of log segments are obtained in the above step 102, each group of log segments includes multiple log fragments, so that the log analysis device will obtain the problem classification results corresponding to each log fragment in each group of log segments when executing step 103. Furthermore, when the log analysis device executes this step 104, it can combine the problem classification results corresponding to the log fragments in the multiple groups of log segments to determine the processing information in the corresponding time period of application development.

[0060] For example, for the above Figure 3b Each log fragment in the three groups of log segments shown can respectively obtain a problem classification result. For example, the problem classification results corresponding to the first log fragment Tmax1 and the second log fragment Tmax2 in log segment group 1 are: no fault and problem type 1, respectively. However, the problem classification result corresponding to the first log fragment Tmax1 in log segment group 2 is: problem type 1. The log analysis device can determine that during the time period corresponding to the first log fragment Tmax1 in log segment group 2, problem type 1 occurs in the development of the application, and a corresponding solution, i.e., processing information, needs to be given.

[0061] Furthermore, the log analysis device may also send processing information based on the application program to a corresponding processing information receiving terminal.

[0062] It can be seen that in the method of this embodiment, the log analysis device will divide the acquired development log into multiple log segments according to the time interval, and then classify the problems of the application corresponding to each log segment according to the preset log analysis strategy, and finally determine the processing information based on the application in the corresponding time period of application development according to the obtained multiple problem classification results. In this way, by analyzing the multiple log segments included in the development log, and each log segment corresponds to a time interval, it is possible to accurately locate which development process will have problems during the development of the application, and then perform corresponding processing, thereby improving the development efficiency of the application.

[0063] It should be noted that if the above-mentioned log analysis strategy includes the operation logic of the log analysis model. In a specific embodiment, the log analysis device can train the log analysis model according to the following steps, as shown in the flowchart Figure 4 As shown, including:

[0064] Step 201, determine the initial model for log analysis.

[0065] It can be understood that when the log analysis device determines the initial model of log analysis, it will determine the initial values ​​of the fixed parameters in the multi-layer structure and each layer of the initial model, including the above-mentioned feature extraction module and log classification module, wherein the feature extraction module is used to extract the feature information of any log sample, specifically, the key label of the log sample can be obtained first, and then the feature vector of the log sample can be obtained according to the key label of the log sample; the log classification module is used to obtain the problem type of the application corresponding to the log sample according to the feature information extracted by the feature extraction module. Specifically, the multi-layer structure in the initial model of log analysis can be any of the following algorithm structures: Convolutional Neural Network (CNN), Support Vector Machines (SVM), etc.

[0066] Among them, fixed parameters refer to the fixed parameters used in the calculation process of each layer structure in the initial model of log analysis, which do not need to be assigned at any time, such as weights, angles and other parameters.

[0067] Step 202: determine a training sample, where the training sample includes a plurality of log samples and a problem type corresponding to each log sample.

[0068] Step 203 , using the log analysis initial model, the problems of the application corresponding to each log sample are determined and classified to obtain an initial result of problem classification.

[0069] Specifically, the key tags of the log samples are obtained through the feature extraction module in the initial model of log analysis, and then the feature vectors of the log samples are obtained based on the key tags of the log samples; the log classification module is used to obtain the problem type of the application corresponding to the log sample based on the feature information extracted by the feature extraction module.

[0070] Step 204, according to the initial problem classification results determined by the initial log analysis model in step 203 and the problem types in the training samples, the fixed parameter values ​​in the initial log analysis model are adjusted to obtain the final log analysis model.

[0071] Specifically, the log analysis unit will first calculate the loss function related to the initial log analysis model based on the initial problem classification results determined by the initial log analysis model in the above step 203 and the problem types in the training samples. The loss function is used to indicate the error of the initial log analysis model in classifying the problems of the application corresponding to each log sample.

[0072] Here, the loss function includes: a function used to represent the difference between the initial problem classification results corresponding to each log sample determined according to the log analysis initial model and the actual problem type of each log sample in the training sample. The mathematical representation of these errors usually uses a cross entropy loss function to establish a loss function, and the training process of the log analysis initial model is to minimize the value of the above error. The training process is to continuously optimize the parameter values ​​of the fixed parameters in the log analysis initial model determined in the above step 201 through a series of mathematical optimization methods such as back propagation derivation and gradient descent, and minimize the calculated value of the above loss function.

[0073] Therefore, after calculating the loss function, the log analysis device needs to adjust the fixed parameter values ​​in the initial log analysis model according to the calculated loss function to obtain the final log analysis model. Specifically, when the function value of the calculated loss function is large, such as greater than a preset value, it is necessary to change the fixed parameter value, such as reducing the weight value of a certain weight, so that the function value of the loss function calculated according to the adjusted fixed parameter value is reduced.

[0074] It should be noted that the above steps 203 to 204 are to obtain the initial problem classification results through the log analysis initial model calculation, and to adjust the fixed parameter values ​​in the log analysis initial model according to the initial problem classification results. In actual applications, it is necessary to continuously loop through the above steps 203 to 204 until the adjustment of the fixed parameter values ​​meets certain stop conditions.

[0075] Therefore, after executing steps 201 to 204 of the above embodiment, the log analysis device still needs to determine whether the current adjustment of the fixed parameter value meets the preset stop condition. If it does, the process ends; if it does not, the initial log analysis model after adjusting the fixed parameter value is returned to execute steps 203 to 204.

[0076] Among them, the preset stop conditions include but are not limited to any one of the following conditions: the difference between the currently adjusted fixed parameter value and the last adjusted fixed parameter value is less than a threshold, that is, the adjusted fixed parameter value reaches convergence; and the number of adjustments to the fixed parameter value is equal to the preset number, etc.

[0077] The following is a specific application example to illustrate the log classification method in the present invention. The method of this embodiment can be applied to the above Figure 1 In the system shown, the preset log analysis strategy in this embodiment is specifically the operation logic of the log analysis model, and the log analysis model can be trained by the method in the above embodiment.

[0078] like Figure 5 As shown, the method of this embodiment includes the following steps:

[0079] Step 301, when an application developer uses a log terminal to develop an application, such as editing application code, debugging code and printing information, the log terminal will record the operation information of the log terminal at each moment, that is, the development log.

[0080] In step 302, the log analysis device initiates a log analysis process according to a certain period, so that the log analysis device obtains log information between the last initiated period and the current period, ie, the development log, from the log terminal.

[0081] In step 303, the log analysis device pre-processes the acquired development log. Specifically, the log analysis device removes abnormal data in the development log, such as development logs with incorrect formats.

[0082] Specifically, Figure 6 As shown, in the process of preprocessing the development log, the log analysis device can use the source data module and the preprocessing model. Each module will create a data flow block respectively, and implement preprocessing through the input (Put) module and the output (Take) module, where:

[0083] The input module in the source data module is used to continuously add data in the development log to the created data flow block. If the data in the data flow block is full, it will wait; the output module in the source data module is used to output data of a specified size or length. When the data flow block is empty, it will wait.

[0084] The input module in the preprocessing module is used to continuously add the data output by the source data module to the created data flow block. If the data in the data flow block is full, it waits. The output module in the preprocessing module is used to preprocess the data in the data flow block and output the preprocessed development log. If the data flow block is empty, it waits.

[0085] Step 304, the log analysis device divides the preprocessed development log. Specifically, the preprocessed development log exists in the form of a character string, and has the characteristics of continuity and time series. The preprocessed development log can be divided into multiple groups of log segments according to time intervals, and each group of log segments includes multiple log fragments.

[0086] Step 305 , the log analysis device classifies the problems of the application corresponding to each log fragment in each group of log segments according to a preset log analysis model to obtain multiple groups of problem classification results, each group of problem classification results including the problem type corresponding to each log fragment.

[0087] Specifically, the key tags of each log segment can be first obtained through the log analysis model, and then the problem type of each log segment can be directly output according to the key tags. The acquisition of the key tags of the log segments is described in the above embodiment, and will not be repeated here.

[0088] Step 306 , the log analysis device determines the processing information based on the application program in the corresponding time period of the application program development according to the problem classification result corresponding to each log segment in each group of log segments.

[0089] Step 307: the log analysis device notifies the corresponding development user's terminal, ie, the processing information receiving terminal, of the processing information in each time period.

[0090] It can be seen that in the embodiment of the present invention, the development log of the application is used as the analysis object, and the log analysis model is used to achieve rapid problem classification and location, and then provide a solution, that is, the above-mentioned processing information, to improve the development efficiency of the application. The development log used has richer information and stronger usability, thereby improving the classification and location of problems. In addition, in this embodiment, the determined processing information is notified to the terminal of the development user, reducing manual intervention in the flow of problems and further improving the development efficiency of the application.

[0091] The embodiment of the present invention also provides a log analysis device, the structural diagram of which is shown in FIG. Figure 7 As shown, it may specifically include:

[0092] The log acquisition unit 10 is used to acquire the development log of the application program;

[0093] The dividing unit 11 is used to divide the development log obtained by the log obtaining unit 10 into a plurality of log segments according to time intervals.

[0094] Specifically, in one case, the dividing unit 11 is specifically used to use a preset maximum time interval as a time window of the development log; when the time window moves along the development log, the log information within the time window is used as a log segment, wherein the step size of the time window moving along the development log is consistent with the preset minimum time interval.

[0095] In another case, the division unit 11 is specifically used to divide the development log into multiple groups of log segments according to time intervals, and each group of log segments includes multiple log fragments. Wherein, when the division unit 11 divides the development log into multiple groups of log segments, it divides the development log into one group of log segments according to a preset maximum time interval; deletes the log information within a preset minimum time interval in the development log, and divides the development log after deletion into another group of log segments according to the preset maximum time interval, and the time interval of each log fragment in each group of log segments is equal to or less than the preset maximum time interval.

[0096] The classification unit 12 is used to classify the problems of the application corresponding to each log segment obtained by the division unit 11 according to a preset log analysis strategy, and obtain multiple problem classification results.

[0097] Specifically, the classification unit 12 is used to obtain key tags of each log fragment respectively; according to the key tags of each log fragment and the preset log analysis strategy, the problems of the application corresponding to each log fragment are classified respectively to obtain multiple problem classification results.

[0098] Among them, when obtaining the key tag of a certain log fragment, the classification unit 12 is specifically used to determine the tag contained in the structured data as the key tag of the certain log fragment when the certain log fragment includes structured data; when the certain log fragment includes unstructured data, determine at least one word in the unstructured data as the key tag of the certain log fragment.

[0099] The processing determination unit 13 is used to determine the processing information based on the application program in a corresponding time period of the application program development according to the multiple problem classification results obtained by the classification unit 12.

[0100] Furthermore, if the log analysis strategy includes the operation logic of the log analysis model, the log analysis device of this embodiment may also include: a training unit 14, which is used to determine the initial log analysis model; determine the training samples, wherein the training samples include multiple log samples and the problem types corresponding to each log sample; classify the problems of the application corresponding to each log sample respectively through the initial log analysis model to obtain the initial problem classification results; according to the initial problem classification results determined by the initial log analysis model and the problem types in the training samples, adjust the fixed parameter values ​​in the initial log analysis model to obtain the final log analysis model. In this way, the above-mentioned classification unit 12 will classify the problems of the application corresponding to each log segment according to the log analysis model trained by the training unit 14.

[0101] The training unit 14 is further configured to stop adjusting the fixed parameter value when the number of adjustments to the fixed parameter value is equal to a preset number, or when the difference between the currently adjusted fixed parameter value and the last adjusted fixed parameter value is less than a threshold.

[0102] It can be seen that in the log analysis device of this embodiment, the division unit 11 will divide the acquired development log into multiple log segments according to the time interval, and then the classification unit 12 will classify the problems of the application corresponding to each log segment according to the preset log analysis strategy, and finally the processing determination unit 13 will determine the processing information based on the application in the corresponding time period of the application development according to the obtained multiple problem classification results. In this way, by analyzing the multiple log segments included in the development log, and each log segment corresponds to a time interval, it is possible to accurately locate which development process will have problems during the development of the application, and then perform corresponding processing, thereby improving the development efficiency of the application.

[0103] The embodiment of the present invention further provides a server, the structural diagram of which is shown in Figure 8 As shown, the server may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 20 (for example, one or more processors) and memory 21, and one or more storage media 22 (for example, one or more mass storage devices) storing application programs 221 or data 222. Among them, the memory 21 and the storage medium 22 can be short-term storage or permanent storage. The program stored in the storage medium 22 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 20 can be configured to communicate with the storage medium 22 and execute a series of instruction operations in the storage medium 22 on the server.

[0104] Specifically, the application 221 stored in the storage medium 22 includes an application for log analysis, and the program may include the log acquisition unit 10, the division unit 11, the classification unit 12, the processing determination unit 13 and the training unit 14 in the above-mentioned log analysis device, which will not be described in detail here. Furthermore, the central processor 20 can be configured to communicate with the storage medium 22 and execute a series of operations corresponding to the application for log analysis stored in the storage medium 22 on the server.

[0105] The server may also include one or more power supplies 23, one or more wired or wireless network interfaces 24, and / or one or more operating systems 223, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0106] The steps performed by the log analysis device in the above method embodiment can be based on the Figure 8 The structure of the server is shown.

[0107] An embodiment of the present invention further provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the log analysis method executed by the above-mentioned log analysis device.

[0108] An embodiment of the present invention also provides another server, including a processor and a storage medium, wherein the processor is used to implement various instructions; the storage medium is used to store multiple instructions, and the instructions are used to be loaded by the processor and execute the log analysis method performed by the above-mentioned log analysis device.

[0109] A person skilled in the art may understand that all or part of the steps in the various methods of the above-mentioned embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0110] The log analysis method and device provided in the embodiments of the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A log analysis method, characterized in that: include: Get the application's development log; According to the time interval, the development log is divided into multiple log segments; wherein, the development logs at T moments are subjected to the following operations for multiple rounds: in the first round, the development logs at the T moments are divided according to the preset maximum time interval to obtain a first group of log segments, the first group of log segments including multiple log segments; in the second round, after the development logs at the T moments start to delete the log information of the preset minimum time interval, the remaining log information is divided according to the maximum time interval to obtain a second group of log segments, the second group of log segments including multiple log segments; in the third round, after the development logs at the T moments start to delete the log information of twice the minimum time interval, the remaining log information is divided according to the maximum time interval to obtain a third group of log segments, the third group of log segments including multiple log segments; the steps of deleting and dividing are repeatedly performed for the development logs after deletion to obtain multiple groups of log segments, each log segment including multiple log segments; According to the preset log analysis strategy, the problems of the application corresponding to each log segment in each group of log segments are classified respectively, and multiple problem classification results corresponding to each group of log segments are obtained; wherein, when any log segment includes structured data, the tag contained in the structured data is determined as the key tag of any log segment; when any log segment includes unstructured data, at least one word in the unstructured data is determined as the key tag of a certain log segment; according to the key tags of each log segment and the preset log analysis strategy, the problems of the application corresponding to each log segment are classified respectively; Determine a problem classification result within a corresponding time period of application development according to a combination of problem classification results corresponding to the log segments in the plurality of groups of log segments, and then determine processing information based on the application; When determining a problem classification result within the corresponding time period, the problem classification result of the corresponding log fragment in one group of log segments and the problem classification results of the corresponding log fragment in other groups of log segments are combined to obtain the problem classification result.

2. The method according to claim 1, characterized in that The development log is divided into a plurality of log segments according to the time interval, specifically including: Use the preset maximum time interval as the time window for development logs; When the time window moves along the development log, the log information in the time window is taken as a log segment, wherein the step length of the time window moving along the development log is consistent with the preset minimum time interval.

3. The method according to claim 1, characterized in that The development log is divided into a plurality of log segments according to the time interval, specifically including: Dividing the development log into a set of log segments according to a preset maximum time interval; Deleting log information within a preset minimum time interval in the development log, and dividing the development log after deletion into another group of log segments according to a preset maximum time interval; Each group of log segments includes multiple log fragments, and the time interval between each log fragment is equal to or less than a preset maximum time interval.

4. The method according to any one of claims 1 to 3, characterized in that The preset log analysis strategy includes the operation logic of the log analysis model, and the method further includes: Determine the initial model for log analysis; Determine a training sample, wherein the training sample includes a plurality of log samples and a problem type corresponding to each log sample; Classify the problems of the application corresponding to each log sample respectively through the initial log analysis model to obtain an initial problem classification result; According to the initial result of problem classification determined by the initial log analysis model and the problem types in the training samples, the fixed parameter values ​​in the initial log analysis model are adjusted to obtain the final log analysis model.

5. The method according to claim 4, characterized in that When the number of adjustments to the fixed parameter value is equal to a preset number, or when the difference between the currently adjusted fixed parameter value and the last adjusted fixed parameter value is less than a threshold, the adjustment of the fixed parameter value is stopped.

6. A log analysis device, characterized in that: include: A log acquisition unit, used to acquire the development log of the application; A division unit is used to divide the development log into multiple log segments according to time intervals; wherein, the following operations are performed for the development logs at T moments: in the first round, the development logs at the T moments are divided according to a preset maximum time interval to obtain a first group of log segments, the first group of log segments including multiple log segments; in the second round, after the development logs at the T moments start to delete log information with a preset minimum time interval, the remaining log information is divided according to the maximum time interval to obtain a second group of log segments, the second group of log segments including multiple log segments; in the third round, after the development logs at the T moments start to delete log information with a time interval twice the minimum time interval, the remaining log information is divided according to the maximum time interval to obtain a third group of log segments, the third group of log segments including multiple log segments; the steps of deleting and dividing are repeatedly performed for the development logs after deletion to obtain multiple groups of log segments, each log segment including multiple log segments; A classification unit, used to classify the problems of the application corresponding to each log segment in each group of log segments according to a preset log analysis strategy, and obtain multiple problem classification results corresponding to each group of log segments; wherein, when any log segment includes structured data, a tag contained in the structured data is determined as a key tag of any log segment; when any log segment includes unstructured data, at least one word in the unstructured data is determined as a key tag of a certain log segment; according to the key tags of each log segment and the preset log analysis strategy, the problems of the application corresponding to each log segment are classified respectively; A processing determination unit is used to determine a problem classification result within a corresponding time period of application development based on a combination of problem classification results corresponding to log fragments in the multiple groups of log segments, and then determine processing information based on the application; wherein, when determining a problem classification result within the corresponding time period, the problem classification result of the corresponding log fragments in one group of log segments and the problem classification results of the corresponding log fragments in other groups of log segments are combined to obtain the problem classification result.

7. A storage medium, characterized in that: The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the log analysis method according to any one of claims 1 to 5.

8. A server, characterized in that: It includes a processor and a storage medium, wherein the processor is used to implement various instructions; The storage medium is used to store a plurality of instructions, and the instructions are used to be loaded by a processor and execute the log analysis method according to any one of claims 1 to 5.

Images