Log diagnosis method and device, storage medium and terminal

Abstract

The embodiment of the invention discloses a log diagnosis method and device, a storage medium and a terminal. A target log corresponding to the user operation is obtained, the target log is split into at least one log fragment according to the data operation type, and all the log fragments correspond to different data operation types; respectively inputting the log fragments of the data operation types into fragment structure diagnosis models corresponding to the data operation types to obtain fragment structure diagnosis results of the log fragments; according to the row structure of each single-row log in each log fragment, inputting each single-row log into a row structure diagnosis model corresponding to the row structure of each single-row log to obtain a row structure diagnosis result of each single-row log; and determining abnormal log fragments and / or abnormal log fields in the target log based on all fragment structure diagnosis results and all row structure diagnosis results of the target log. According to the embodiment of the invention, the log is automatically diagnosed through the model, the log checking cost is reduced, and the diagnosis efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a log diagnosis method, device, storage medium, and terminal. Background Art

[0002] During the operation of the program, a large number of operation logs will be generated. These logs record in detail the system's operating status, user behavior, data processing flow, and any possible abnormal or error information. Therefore, developers often analyze log data to quickly locate and fix bugs. However, in addition to abnormal data, the logs also contain a large amount of redundant non-abnormal data. If we simply rely on increasing manpower to analyze them one by one, it will not only be costly and inefficient, but also easily affect the accuracy of information interpretation. Therefore, finding and implementing efficient and intelligent log analysis solutions has become a top priority. Summary of the invention

[0003] The embodiments of the present specification provide a log diagnosis method, device, storage medium and terminal, which can solve the technical problem of low log diagnosis efficiency in related technologies.

[0004] In a first aspect, an embodiment of the present specification provides a log diagnosis method, the method comprising:

[0005] Obtain a target log corresponding to the user operation, and split the target log into at least one log segment according to the data operation type, each log segment corresponding to a different data operation type;

[0006] Input the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type, and obtain the fragment structure diagnosis result of each log fragment;

[0007] According to the row structure of each single-line log in each log fragment, each single-line log is input into the row structure diagnosis model corresponding to its own row structure to obtain the row structure diagnosis result of each single-line log;

[0008] Based on all the segment structure diagnosis results and all the row structure diagnosis results of the target log, abnormal log segments and / or abnormal log fields in the target log are determined.

[0009] In a second aspect, an embodiment of the present specification provides a log diagnosis device, the device comprising:

[0010] A log acquisition module is used to acquire a target log corresponding to a user operation, and split the target log into at least one log segment according to a data operation type, each log segment corresponding to a different data operation type;

[0011] A fragment diagnosis module is used to input the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type, and obtain the fragment structure diagnosis result of each log fragment;

[0012] The field diagnosis module is used to input each single-line log into a row structure diagnosis model corresponding to its own row structure according to the row structure of each single-line log in each log fragment, so as to obtain the row structure diagnosis result of each single-line log;

[0013] The result acquisition module is used to determine abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

[0014] In a third aspect, an embodiment of the present specification provides a computer program product comprising instructions, which, when executed on a computer or a processor, enables the computer or the processor to execute the steps of the above method.

[0015] In a fourth aspect, an embodiment of the present specification provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of the above method.

[0016] In a fifth aspect, an embodiment of the present specification provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0017] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0018] The embodiment of this specification provides a log diagnosis method, which obtains the target log corresponding to the user operation, splits the target log into at least one log segment according to the data operation type, and each log segment corresponds to a different data operation type; the log segments of each data operation type are respectively input into the segment structure diagnosis model corresponding to the data operation type to obtain the segment structure diagnosis results of each log segment; according to the row structure of each single-line log in each log segment, each single-line log is input into the row structure diagnosis model corresponding to its own row structure to obtain the row structure diagnosis results of each single-line log; based on all the segment structure diagnosis results and all the row structure diagnosis results of the target log, the abnormal log segments and / or abnormal log fields in the target log are determined. In the embodiment of this specification, the model is used to automatically complete the diagnosis of the log, thereby reducing the cost of log troubleshooting. The diagnosis process includes two parts, one part is to diagnose the segment structure of the log segment, and the other part is to diagnose the field structure of the single-line log. Among them, since the logs generated by different types of data operations have different log structures, the types of log fragments are divided according to the data operation types, so that various log fragments can be adaptively input into the corresponding model for diagnosis. Finally, the diagnosis results of the fragment structure and row structure are integrated to accurately determine the abnormal fragments and fields in the original log, thereby improving the diagnostic efficiency of the log. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 An exemplary system architecture diagram of a log diagnosis method provided in an embodiment of this specification;

[0021] Figure 2 A flow chart of a log diagnosis method provided in an embodiment of this specification;

[0022] Figure 3 A logical flow chart of a log diagnosis method provided in an embodiment of this specification;

[0023] Figure 4 A flow chart of a log diagnosis method provided in an embodiment of this specification;

[0024] Figure 5 A flow chart of training and diagnosis of a segment structure diagnosis model in a log diagnosis method provided in an embodiment of this specification;

[0025] Figure 6A flow chart of a log diagnosis method provided in an embodiment of this specification;

[0026] Figure 7 A training and diagnosis flow chart of a field structure diagnosis model in a log diagnosis method provided in an embodiment of this specification;

[0027] Figure 8 A structural block diagram of a log diagnostic device provided in an embodiment of this specification;

[0028] Fig. 9 A schematic diagram of the structure of a terminal provided in an embodiment of this specification. DETAILED DESCRIPTION

[0029] In order to make the features and advantages of the embodiments of this specification more obvious and easy to understand, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the embodiments of this specification.

[0030] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of this specification as detailed in the appended claims. And in the description of the embodiments of this specification, unless otherwise indicated, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this specification, "multiple" refers to two or more than two.

[0031] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0032] For large-scale Internet service platforms with large scale and complex functions, the breadth and depth of various service logics often mean that a large number of program errors (i.e. bugs) will inevitably be encountered and generated during the development, integration and operation of the system. These bugs may be hidden in every corner of the code, from basic data processing to complex user interaction logic, all of which become potential hidden dangers. Although in the entire life cycle of software development, various means such as unit testing, integration testing, system testing and stress testing will be used in advance to try to discover and fix these bugs as comprehensively as possible before the software goes online, it is still difficult to ensure that all bugs can be intercepted and fixed in advance before going online due to the complexity of the system and the limited testing resources.

[0033] After the software is launched, with the use of a large number of users, some bugs that were missed during testing are eventually triggered by users in the online environment. In order to promptly discover and solve these bugs triggered in the online environment, various service platforms often adopt a logging strategy. During the operation of the program, a large number of operation logs will be generated. These logs record in detail the system's operating status, user behavior, data processing flow, and any possible abnormal or error information. This means that the rich operation data in these logs can help developers quickly locate and fix bugs.

[0034] However, the challenge in practice is that due to the huge scale of the platform and the complexity of various service logics, the log information generated has also shown explosive growth. The large number and complex content make it difficult to effectively process them through manual analysis. These logs contain a large number of operation records, and the log formats are diverse and the timestamps are staggered, making it extremely difficult to accurately filter out valuable exceptions, warnings, and error information from the massive logs.

[0035] Therefore, the embodiments of this specification provide a log diagnosis method to solve the above technical problem of low efficiency of log diagnosis.

[0036] See also Figure 1 , Figure 1 An exemplary system architecture diagram of a log diagnosis method provided in an embodiment of this specification.

[0037] like Figure 1 As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired communication links or wireless communication links, for example, the wired communication link includes an optical fiber, a twisted pair, or a coaxial cable, and the wireless communication link includes a Bluetooth communication link, a Wireless-Fidelity (Wi-Fi) communication link, or a microwave communication link.

[0038] The terminal 101 can interact with the server 103 through the network 102 to receive messages from the server 103 or send messages to the server 103, or the terminal 101 can interact with the server 103 through the network 102 to receive messages or data sent by other users to the server 103. The terminal 101 can be hardware or software. When the terminal 101 is hardware, it can be various electronic devices, including but not limited to smart watches, smart phones, tablet computers, laptop portable computers and desktop computers. When the terminal 101 is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules (for example: used to provide distributed services), or it can be implemented as a single software or software module, which is not specifically limited here.

[0039] In an embodiment of the present specification, terminal 101 first obtains a target log corresponding to a user operation, and splits the target log into at least one log segment according to the data operation type, each log segment corresponding to a different data operation type; then, terminal 101 can input the log segments of each data operation type into a segment structure diagnosis model corresponding to the data operation type, and obtain a segment structure diagnosis result of each log segment; terminal 101 can also input each single-line log into a row structure diagnosis model corresponding to its own row structure according to the row structure of each single-line log in each log segment, and obtain a row structure diagnosis result of each single-line log; finally, terminal 101 determines abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

[0040] The server 103 may be a business server that provides various services. It should be noted that the server 103 may be hardware or software. When the server 103 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or it may be implemented as a single server. When the server 103 is software, it may be implemented as multiple software or software modules (for example, for providing distributed services), or it may be implemented as a single software or software module, which is not specifically limited here.

[0041] Alternatively, the system architecture may not include the server 103. In other words, the server 103 may be an optional device in the embodiments of this specification, that is, the method provided in the embodiments of this specification may be applied to a system structure that only includes the terminal 101, and the embodiments of this specification do not limit this.

[0042] It should be understood that Figure 1 The number of terminals, networks and servers in the figure is only for illustration and any number of terminals, networks and servers may be used according to implementation requirements.

[0043] See also Figure 2 , Figure 2 A flow chart of a log diagnosis method provided in an embodiment of this specification. The execution subject of the embodiment of this specification can be a terminal that executes log diagnosis, or a processor in the terminal that executes the log diagnosis method, or a log diagnosis service in the terminal that executes the log diagnosis method. For the convenience of description, the specific execution process of the log diagnosis method is introduced below by taking the execution subject as the processor in the terminal as an example.

[0044] like Figure 2 As shown, the log diagnosis method may at least include:

[0045] S202: Obtain a target log corresponding to the user operation, and split the target log into at least one log segment according to the data operation type, where each log segment corresponds to a different data operation type.

[0046] Optionally, when diagnosing logs, obtaining the target log corresponding to the user operation is a basic and crucial link. Usually, in order to comprehensively monitor program errors (i.e. bugs) in the system, a complete log collection will be performed for each operation of the user in the system or application, whether it is click, input, submission or any other interactive behavior. Generally, a log framework or integrated third-party log service is used to capture and store the log information generated by each user operation. These log information often contains key elements such as the timestamp of the operation, user identity, operation type, and related data details, providing basic data support for subsequent analysis and processing.

[0047] Optionally, when a user uses a service in the system, different types of data operations may be generated due to different user operations. For example, in one user operation, the user successively accesses the application container, the network, and the service. Then the log stream corresponding to this operation will include the log fragment corresponding to the application container, the log fragment corresponding to the network, and the log fragment corresponding to the service. The logs generated by these different types of data operations have different log structures, so it is necessary to adopt appropriate diagnostic strategies for logs of different operation types so that various types of log fragments can be accurately analyzed.

[0048] In the embodiment of this specification, before diagnosing the target log, the target log is first split into at least one log segment according to the data operation type. In order to achieve effective splitting, the system needs to pre-define a set of clear operation type classification standards, and judge the target log based on these standards, and divide it into multiple log segments according to the corresponding operation type. Figure 3 , Figure 3 The following is a logical flow chart of a log diagnosis method provided in an embodiment of this specification. Figure 3As shown in the figure, in the target log, the first 4 lines of log fragments belong to data operation type a; the next 3 lines belong to data operation type b, and the last 2 lines belong to data operation type c. Such splitting not only facilitates the subsequent analysis and audit of log data, but also provides strong support for system performance monitoring, troubleshooting, and security auditing.

[0049] In addition, it is worth noting that in actual applications, the splitting and classification of logs often need to be flexibly adjusted according to specific operating scenarios and requirements. For example, in some complex applications, it may be necessary to further segment and classify logs based on factors such as the complexity of the operation, the amount of data involved, and the scope of the operation, so as to more accurately grasp the operating status of the system and the user's operating behavior.

[0050] S204 , inputting the log segments of each data operation type into the segment structure diagnosis model corresponding to the data operation type respectively, and obtaining the segment structure diagnosis result of each log segment.

[0051] Optionally, for log fragments of different data operation types, it is necessary to analyze and diagnose them according to the log characteristics of each data operation type. The logs of each type of operation have different information content and structural characteristics. Therefore, a dedicated fragment structure diagnosis model can be trained in advance for each data operation type. These fragment structure diagnosis models are built based on deep learning or machine learning technology. They learn a large amount of historical log data during the training process and master the standard structure and characteristics of the logs of each data operation type. Please continue to refer to Figure 3 When a new log that needs to be diagnosed appears, each classified log fragment will be matched to the diagnostic model of the data operation type to which it belongs. The model will conduct a comprehensive analysis and diagnosis to check whether its structure conforms to the expected standard format, including key aspects such as field integrity, order correctness, and data type matching. The diagnostic model corresponding to each data operation type will detect a fragment structure diagnosis result for the input log fragment. It should be noted that the matching between log fragments and models of corresponding types is automated. Through preset algorithms and programs, log fragments are efficiently matched and input with corresponding models.

[0052] S206 . According to the row structure of each single-line log in each log segment, each single-line log is input into a row structure diagnosis model corresponding to its own row structure to obtain a row structure diagnosis result of each single-line log.

[0053] Optionally, diagnosing the fragment structure of the log fragment can clarify whether there are abnormalities in the relationship and order between the lines of log, but there is also a risk of abnormalities in the multiple fields included in each line of log. Therefore, the line structure of each single line of log in the log can also be diagnosed to determine whether the relationship between the fields within the single line is accurate.

[0054] Specifically, the log contains multiple single-line logs, each of which records the operation information of the system or application in its specific line structure. In order to accurately parse this information, a diagnostic model can be pre-trained for each line structure, and each model can recognize and parse single-line logs in a specific format. The design of the model takes into account various fields that may appear in the log, such as timestamp, log level, message content, source information, and the relationship between various fields, and can accurately locate and extract these elements. The line structure of each single-line log needs to be carefully analyzed.

[0055] Optionally, see Figure 3 After the model is trained, for each single-line log, it is necessary to input it into the corresponding row structure diagnosis model according to its row structure characteristics. After the single-line log is input into the corresponding row structure diagnosis model, the model will perform in-depth analysis and processing on the log. In this process, the model will use machine learning algorithms and natural language processing technology to identify and classify the various fields in the log and the relationship between the fields, and generate corresponding diagnostic results. These diagnostic results may include log integrity checks (such as whether key elements are missing), format correctness verification (such as whether the timestamp format meets expectations), and identification of potential problems (such as the appearance of abnormal log fields). After the model is targeted, the row structure diagnosis results of each single-line log will be obtained. These results can reflect the detailed content and structural characteristics of the log, and provide strong support for subsequent log analysis, monitoring, and alarm tasks.

[0056] S208: Determine abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

[0057] Optionally, when determining the abnormal log fragments and / or abnormal log fields in the target log, the detailed information of all fragment structure diagnosis results and all row structure diagnosis results can be comprehensively considered. First, based on the fragment structure diagnosis results, it can be reflected whether each log fragment follows the established structural specifications, including whether the start mark, end mark, delimiter and field order of the log fragment meet expectations. If a fragment is missing a necessary component, or its structural organization method deviates significantly from the standard mode, then this fragment may be determined by the model as an abnormal log fragment. Further, based on the row structure diagnosis results within each single-line log, the row structure diagnosis results can explain whether there is a logical inconsistency between the fields in each line of the log, or whether there is a logical contradiction between two related fields (such as the wrong order of the timestamp field, etc.). If the row structure does not meet expectations, then these lines may contain abnormal log fields. Finally, combining the fragment structure diagnosis results and the row structure diagnosis results will be able to determine which fragments and / or fields in the target log are abnormal. These abnormal information can play an important role in subsequent problem location, system optimization and troubleshooting.

[0058] Optionally, in order to facilitate users to check and handle anomalies in the target log, you can mark the anomalies when abnormal log segments and abnormal log fields are found. Figure 3 If there are abnormal log fragments, a specific "fragment abnormality identifier" can be added to these abnormal log fragments. This identifier can not only be a simple label, but may also further contain detailed information about the abnormality type, occurrence time, and possible causes, so as to facilitate subsequent analysis and processing; if there are abnormal log fields, a special "field abnormality identifier" is also added. This identifier is directly associated with the specific field, and may intuitively indicate which fields have problems on the page through color changes, icon annotations, or highlighting.

[0059] Furthermore, in the embodiments of this specification, the "segment exception mark" and "field exception mark" have different display effects on the page to distinguish different types of exceptions and enhance the user's reading experience. The segment exception mark may use more eye-catching colors (such as red or orange) and borders, as well as concise and clear text descriptions, so that users can recognize the abnormal status of the entire log segment at a glance. The field exception mark may use more sophisticated visual designs, such as using small dots, underlines, or pop-up prompt boxes in different colors, to accurately indicate which specific field has a problem, and provide more detailed exception information or recommended operation steps. This helps to quickly locate and solve problems, and also improves the efficiency and accuracy of log analysis, ensuring the stability and reliability of system operation.

[0060] In an embodiment of the present specification, a log diagnosis method is provided, which obtains a target log corresponding to a user operation, splits the target log into at least one log segment according to the data operation type, and each log segment corresponds to a different data operation type; the log segments of each data operation type are respectively input into the segment structure diagnosis model corresponding to the data operation type to obtain the segment structure diagnosis results of each log segment; according to the row structure of each single-line log in each log segment, each single-line log is input into the row structure diagnosis model corresponding to its own row structure to obtain the row structure diagnosis results of each single-line log; based on all the segment structure diagnosis results and all the row structure diagnosis results of the target log, the abnormal log segments and / or abnormal log fields in the target log are determined. In the embodiment of the present specification, the model is used to automatically complete the diagnosis of the log, thereby reducing the cost of log troubleshooting. The diagnosis process includes two parts, one part is to diagnose the segment structure of the log segment, and the other part is to diagnose the field structure of the single-line log. Among them, since the logs generated by different types of data operations have different log structures, the types of log fragments are divided according to the data operation types, so that various log fragments can be adaptively input into the corresponding model for diagnosis. Finally, the diagnosis results of the fragment structure and row structure are integrated to accurately determine the abnormal fragments and fields in the original log, thereby improving the diagnostic efficiency of the log.

[0061] See also Figure 4 , Figure 4 A flow chart of a log diagnosis method provided in an embodiment of this specification.

[0062] like Figure 4 As shown, the log diagnosis method may at least include:

[0063] S402: Obtain a target log corresponding to the user operation, and split the target log into at least one log segment according to the data operation type, where each log segment corresponds to a different data operation type.

[0064] Regarding step S402, please refer to the detailed description in step S202, which will not be repeated here.

[0065] S404: Remove redundant values ​​in each log segment to obtain a key field of each log segment.

[0066] Optionally, in addition to some specific fields, the log will also carry specific parameter values ​​according to the user's specific operation information. For example, if the user performs an operation on October 25, the parameter value after the timestamp field is "October 25". These values ​​change with the actual operation situation and are not unique or fixed. Therefore, when diagnosing the log structure, these values ​​should not be used for diagnosis. Only the key fields should be retained for structural diagnosis. For details, please refer to Figure 5 , Figure 5 The following is a flow chart of the training and diagnosis of a segment structure diagnosis model in a log diagnosis method provided in an embodiment of this specification. Figure 5 As shown, in the process of diagnosis using the fragment structure diagnosis model, the redundant values ​​in the log fragments are first removed to obtain the key fields of each log fragment to avoid the influence of the values ​​on the diagnosis results.

[0067] S406: Generate a row code for each log line according to all key fields in each log line in each log segment.

[0068] Optionally, in order to diagnose the relationship between the log lines in the log segment through the segment structure diagnosis model, it is also necessary to convert each log line in each log segment into an input form that the model can accept. Figure 5 , a line code of each log line can be generated according to all key fields in each log line in each log fragment, and the line code has a format and length that conforms to the input specification of the fragment structure diagnosis model.

[0069] S408. Mask part of the row codes in each log segment to obtain a fuzzy code sequence of each log segment, input each fuzzy code sequence into a corresponding segment structure diagnosis model, and the segment structure diagnosis model is used to predict unknown row codes based on known row codes in the fuzzy code sequence.

[0070] Optionally, the log lines in the log segment have a specific order and relationship structure. In order to diagnose whether the relationship between the lines in the log segment is as expected, some of the line codes in each log segment can be masked to obtain the fuzzy code sequence of each log segment. For example, the log segment A to be diagnosed includes 4 lines of logs, which are converted into 4 line codes, namely ID1, ID2, ID3, and ID4. The 3rd line is masked. In the fuzzy code sequence obtained after masking, the 1st, 2nd, and 4th lines are known, while the 3rd line is unknown. Then please continue to refer to Figure 5 , input each fuzzy code sequence into the corresponding fragment structure diagnosis model, and the fragment structure diagnosis model can predict the unknown row code based on the known row code in the fuzzy code sequence. Use the model that has learned all the sequence patterns of the data operation type log to predict the masked part. If the predicted result is consistent with the true result, it means that the true result is in line with expectations and normal. On the contrary, if the predicted result is inconsistent with the true result, it means that the true result is not in line with expectations and abnormal.

[0071] It should be noted that the masking methods of the sample logs and the target logs to be diagnosed in actual applications are consistent, and the masked positions and number of rows can be fixed or random.

[0072] Optionally, in order to achieve the above-mentioned diagnosis effect, it is necessary to pre-train the segment structure diagnosis model corresponding to each data operation type. For the convenience of description, in the embodiments of this specification, a single segment structure diagnosis model is taken as an example to explain the model training process in detail. Figure 5 The fragment structure diagnosis model is trained based on the fuzzy coding sequence of at least one sample log. These fuzzy coding sequences are consistent with the processing methods in actual application scenarios. They are obtained by first removing redundant values ​​from each line of the sample log and retaining only the key fields and converting them into line codes, and then masking some line codes. During the training process of the fragment structure diagnosis model, the predicted line code is output for the fuzzy coding sequence of each sample log, and the loss value of the loss function is calculated based on the predicted line code and the standard line code of each sample log. The model parameters are adjusted based on the loss value until the fragment structure diagnosis model can predict the expected normal line code, and it can be determined that the fragment structure diagnosis model has met the preset convergence conditions. The sample logs used by the fragment structure diagnosis models of various data operation types are different, and the training process and principles can be the same, which will not be repeated here.

[0073] In a possible implementation, the fragment structure diagnosis model can be implemented based on LogBERT. LogBERT can use the characteristics of the BERT model to capture the pattern of normal log sequences. The BERT structure is expected to capture the context of each log, so it is suitable for correctly predicting randomly masked log lines in normal log sequences.

[0074] S410: Determine the predicted row code output by each segment structure diagnosis model for each log segment.

[0075] S412: Compare the predicted row code of each log segment with the original true row code corresponding to the predicted row code. If they are inconsistent, determine that the log segment corresponding to the original true row code is an abnormal log segment.

[0076] Optionally, the fragment structure diagnosis model outputs predicted row codes for the input fuzzy code sequence, and then compares the predicted row codes of each log fragment with the original true row codes corresponding to the predicted row codes. If they are inconsistent, it means that the original true log of the masked row code does not meet expectations, and the log fragment corresponding to the original true row code can be determined as an abnormal log fragment.

[0077] Furthermore, when using the fragment structure diagnosis model for diagnosis, since each masking only covers a part of the log fragment, multiple diagnoses can be performed on a log fragment. In each diagnosis, the position of the log fragment covered is different, so that the log fragment can be diagnosed more comprehensively.

[0078] S414. According to the row structure of each single-line log in each log segment, each single-line log is input into a row structure diagnosis model corresponding to its own row structure to obtain a row structure diagnosis result of each single-line log.

[0079] S416: Determine abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

[0080] Regarding steps S414-S416, please refer to the detailed description in steps S206-S208, which will not be repeated here.

[0081] In an embodiment of the present specification, a log diagnosis method is provided. When diagnosing the fragment structure between lines in a log, redundant values ​​in each log fragment are removed to obtain the key fields of each log fragment; part of the line codes in each log fragment are masked to obtain the fuzzy code sequence of each log fragment, and each fuzzy code sequence is input into a corresponding fragment structure diagnosis model, and the fragment structure diagnosis model is used to predict the unknown line code according to the known line code in the fuzzy code sequence; the predicted line code output by each fragment structure diagnosis model for each log fragment is determined; the predicted line code of each log fragment is compared with the original true line code corresponding to the predicted line code, and if they are inconsistent, the log fragment corresponding to the original true line code is determined to be an abnormal log fragment. Using a model that has learned all log sequence patterns of the data operation type to predict the masked part, it is possible to accurately determine the abnormal fragments in the original log and improve the diagnostic efficiency of the log.

[0082] See also Figure 6 , Figure 6 A flow chart of a log diagnosis method provided in an embodiment of this specification.

[0083] like Figure 6 As shown, the log diagnosis method may at least include:

[0084] S602: Obtain a target log corresponding to the user operation, and split the target log into at least one log segment according to the data operation type, where each log segment corresponds to a different data operation type.

[0085] S604: Input the log segments of each data operation type into the segment structure diagnosis model corresponding to the data operation type to obtain the segment structure diagnosis result of each log segment.

[0086] Regarding steps S602-S604, please refer to the detailed description in steps S202-S204, which will not be repeated here.

[0087] S606. Cover the first half of the fields of each single-line log in each log fragment and input the first-line structure diagnosis model corresponding to the line structure of the single-line log to obtain the predicted second half of the fields output by each first-line structure diagnosis model. The first-line structure diagnosis model is used to predict unknown fields based on known fields.

[0088] Optionally, the fields in a single log line also have a specific order and relationship structure, so please refer to Figure 7 A training and diagnosis flow chart of a field structure diagnosis model in a log diagnosis method provided in an embodiment of this specification, such as Figure 7 As shown, in order to diagnose whether the relationship between the fields is as expected, the first half of the fields of each single-line log in each log segment can also be covered. For example, the single-line log B to be diagnosed includes 8 fields, and the first 4 of them are covered. Figure 7 Unmasked fields are represented by circles containing the number 1, and masked fields are represented by circles containing the number 0. After masking, the first half of the log line is unknown, and the second half is known. Then please continue to refer to Figure 7 , input each single-line log after the masked field into the first-line structure diagnosis model corresponding to the row structure of the single-line log. The first-line structure diagnosis model (represented by c in the figure) can predict the timing information of the unknown field (i.e., h5, h6, h7) according to the timing information of the known field (i.e., h1, h2, h3), thereby obtaining the predicted second half of the field output by each first-line structure diagnosis model.

[0089] S608: Compare the predicted second half field of each single-line log with the original real field corresponding to the predicted second half field. If they are inconsistent, determine that the original real field is an abnormal log field.

[0090] Optionally, a model that has learned all field sequence patterns of the row structure is used to predict the masked part, and the predicted second half fields of each single-line log are compared with the original real fields corresponding to the predicted second half fields. If the predicted result is consistent with the real result, it means that the real result is in line with expectations and normal. Otherwise, if the predicted result is inconsistent with the real result, it means that the real result is inconsistent with expectations and abnormal.

[0091] S610, mask the second half of the fields of each single-line log in each log fragment and input the second line structure diagnosis model corresponding to the line structure of the single-line log to obtain the predicted first half of the fields output by each second line structure diagnosis model, and the second line structure diagnosis model is used to predict unknown fields based on known fields.

[0092] Optionally, when masking the first half of the field in a row, it is possible to detect whether there are abnormal fields in the first half. Accordingly, in order to fully diagnose the row of logs, the second half of the field also needs to be masked, so as to diagnose the second half of the field. For details, please refer to Figure 7 , cover the second half of the fields of each single-line log in each log fragment and input the second line structure diagnosis model corresponding to the line structure of the single-line log to obtain the predicted first half of the fields output by each second line structure diagnosis model. The second line structure diagnosis model is also used to predict unknown fields based on known fields. By comparing the predicted results with the actual results, it can be determined whether the currently covered field has an abnormality.

[0093] Optionally, in order to achieve the above-mentioned diagnostic effect, it is necessary to pre-train the first field structure diagnostic model and the second field structure diagnostic model corresponding to each row structure, wherein the training method of each field structure diagnostic model is consistent. For the convenience of description, in the embodiments of this specification, a single field structure diagnostic model is taken as an example to explain the model training process in detail. Figure 7 The row structure diagnosis model is trained based on at least one sample single-line log, where the sample single-line log consists of masked fields and uncovered fields. During the training of the row structure diagnosis model, the predicted field corresponding to the masked field is output for each sample single-line log, and the loss value of the loss function is calculated based on the predicted field and the standard field of each sample single-line log, and the model parameters are adjusted based on the loss value until the row structure diagnosis model meets the preset convergence conditions.

[0094] S612: Compare the predicted first half field of each single-line log with the original real field corresponding to the predicted first half field. If they are inconsistent, determine that the original real field is an abnormal log field.

[0095] Optionally, the predicted first half field of each single-line log is compared with the original real field corresponding to the predicted first half field. If the predicted result is consistent with the real result, it means that the real result is in line with expectations and normal. Otherwise, if the predicted result is inconsistent with the real result, it means that the real result is inconsistent with expectations and abnormal.

[0096] S614: Determine abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

[0097] Regarding step S614, please refer to the detailed description in step S208, which will not be repeated here.

[0098] In an embodiment of the present specification, a log diagnosis method is provided. When diagnosing abnormal fields in logs, the first half of the fields of each single-line log in each log segment are covered and the first row structure diagnosis model corresponding to the row structure of the single-line log is input to obtain the predicted second half of the fields output by each first row structure diagnosis model, and the first row structure diagnosis model is used to predict unknown fields based on known fields; the predicted second half of the fields of each single-line log are compared with the original real fields corresponding to the predicted second half of the fields, and if they are inconsistent, the original real fields are determined to be abnormal log fields; the second half of the fields of each single-line log in each log segment are covered and the second row structure diagnosis model corresponding to the row structure of the single-line log is input to obtain the predicted first half of the fields output by each second row structure diagnosis model, and the second row structure diagnosis model is used to predict unknown fields based on known fields; the predicted first half of the fields of each single-line log are compared with the original real fields corresponding to the predicted first half of the fields, and if they are inconsistent, the original real fields are determined to be abnormal log fields. By using a model that has learned all the field patterns of its own row structure to predict the masked parts, it is possible to accurately determine the abnormal fields in the original log and improve the diagnostic efficiency of the log.

[0099] See also Figure 8 , Figure 8 This is a structural block diagram of a log diagnostic device provided in an embodiment of this specification. Figure 8 As shown, the log diagnosis device 800 includes:

[0100] The log acquisition module 810 is used to acquire a target log corresponding to a user operation, and split the target log into at least one log segment according to the data operation type, each log segment corresponding to a different data operation type;

[0101] The fragment diagnosis module 820 is used to input the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type, and obtain the fragment structure diagnosis result of each log fragment;

[0102] The field diagnosis module 830 is used to input each single-line log into a row structure diagnosis model corresponding to its own row structure according to the row structure of each single-line log in each log segment, so as to obtain a row structure diagnosis result of each single-line log;

[0103] The result acquisition module 840 is used to determine abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

[0104] Optionally, the fragment diagnosis module 820 is also used to remove redundant numerical values ​​in each log fragment to obtain the key fields of each log fragment; generate the row code of each log line according to all the key fields in each log line in each log fragment; mask part of the row codes in each log fragment to obtain the fuzzy code sequence of each log fragment, input each fuzzy code sequence into the corresponding fragment structure diagnosis model, the fragment structure diagnosis model is used to predict the unknown row code according to the known row code in the fuzzy code sequence; determine the predicted row code output by each fragment structure diagnosis model for each log fragment; compare the predicted row code of each log fragment with the original true row code corresponding to the predicted row code, if they are inconsistent, determine that the log fragment corresponding to the original true row code is an abnormal log fragment.

[0105] Optionally, the fragment structure diagnosis model is trained based on a fuzzy coding sequence of at least one sample log, where the fuzzy coding sequence is obtained by converting each line of log in the sample log into a line code and then masking part of the line code; during the training process of the fragment structure diagnosis model, the fuzzy coding sequence of each sample log is output with a predicted line code, and the loss value of the loss function is calculated based on the predicted line code and the standard line code of each sample log, and the model parameters are adjusted based on the loss value until the fragment structure diagnosis model meets the preset convergence conditions.

[0106] Optionally, the field diagnosis module 830 is also used to cover the first half of the fields of each single-line log in each log fragment and input the first line structure diagnosis model corresponding to the row structure of the single-line log to obtain the predicted second half of the fields output by each first line structure diagnosis model, and the first line structure diagnosis model is used to predict unknown fields based on known fields; compare the predicted second half of the fields of each single-line log with the original real field corresponding to the predicted second half of the fields, and if there is inconsistency, determine that the original real field is an abnormal log field; cover the second half of the fields of each single-line log in each log fragment and input the second line structure diagnosis model corresponding to the row structure of the single-line log to obtain the predicted first half of the fields output by each second line structure diagnosis model, and the second line structure diagnosis model is used to predict unknown fields based on known fields; compare the predicted first half of the fields of each single-line log with the original real field corresponding to the predicted first half of the fields, and if there is inconsistency, determine that the original real field is an abnormal log field.

[0107] Optionally, the row structure diagnosis model is trained based on at least one sample single-line log, and the sample single-line log consists of masked fields and uncovered fields; during the training process of the row structure diagnosis model, the predicted field corresponding to the masked field is output for each sample single-line log, and the loss value of the loss function is calculated based on the predicted field and the standard field of each sample single-line log, and the model parameters are adjusted based on the loss value until the row structure diagnosis model meets the preset convergence conditions.

[0108] Optionally, the log diagnosis device 800 also includes: an exception identification module, which is used to add a segment exception identification to the abnormal log segment if there is an abnormal log segment; if there is an abnormal log field, add a field exception identification to the abnormal log field; wherein the segment exception identification and the field exception identification have different page display effects.

[0109] In an embodiment of the present specification, a log diagnosis device is provided, wherein a log acquisition module is used to acquire a target log corresponding to a user operation, and the target log is split into at least one log segment according to the data operation type, and each log segment corresponds to a different data operation type; a segment diagnosis module is used to input the log segments of each data operation type into the segment structure diagnosis model corresponding to the data operation type, and obtain the segment structure diagnosis result of each log segment; a field diagnosis module is used to input each single-line log into the row structure diagnosis model corresponding to its own row structure according to the row structure of each single-line log in each log segment, and obtain the row structure diagnosis result of each single-line log; a result acquisition module is used to determine the abnormal log segment and / or abnormal log field in the target log based on all the segment structure diagnosis results and all the row structure diagnosis results of the target log. In the embodiment of the present specification, the model is used to automatically complete the diagnosis of the log, thereby reducing the cost of log troubleshooting. The diagnosis process includes two parts, one is to diagnose the segment structure of the log segment, and the other is to diagnose the field structure of the single-line log. Among them, since the logs generated by different types of data operations have different log structures, the types of log fragments are divided according to the data operation types, so that various log fragments can be adaptively input into the corresponding model for diagnosis. Finally, the diagnosis results of the fragment structure and row structure are integrated to accurately determine the abnormal fragments and fields in the original log, thereby improving the diagnostic efficiency of the log.

[0110] The embodiments of this specification provide a computer program product including instructions. When the computer program product is executed on a computer or a processor, the computer or the processor executes the steps of any one of the methods in the above embodiments.

[0111] The embodiments of this specification also provide a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of any method in the above embodiments.

[0112] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of a terminal provided in an embodiment of this specification. Fig. 9 As shown, the terminal 900 may include: at least one terminal processor 901 , at least one network interface 904 , a user interface 903 , a memory 905 , and at least one communication bus 902 .

[0113] The communication bus 902 is used to realize the connection and communication between these components.

[0114] The user interface 903 may include a display screen (Display) and a camera (Camera), and the optional user interface 903 may also include a standard wired interface and a wireless interface.

[0115] The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0116] Among them, the terminal processor 901 may include one or more processing cores. The terminal processor 901 uses various interfaces and lines to connect various parts within the entire terminal 900, and executes various functions and processes data of the terminal 900 by running or executing instructions, programs, code sets or instruction sets stored in the memory 905, and calling data stored in the memory 905. Optionally, the terminal processor 901 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The terminal processor 901 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the terminal processor 901, and it can be implemented separately through a chip.

[0117] Among them, the memory 905 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory, ROM). Optionally, the memory 905 includes a non-transitory computer-readable storage medium. The memory 905 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 905 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 905 may also be optionally at least one storage device located away from the aforementioned terminal processor 901. As Fig. 9 As shown, the memory 905 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a log diagnosis program.

[0118] exist Fig. 9 In the terminal 900 shown, the user interface 903 is mainly used to provide an input interface for the user and obtain the data input by the user; and the terminal processor 901 can be used to call the log diagnosis program stored in the memory 905 and specifically perform the following operations:

[0119] Obtain a target log corresponding to the user operation, and split the target log into at least one log segment according to the data operation type, where each log segment corresponds to a different data operation type;

[0120] Input the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type, and obtain the fragment structure diagnosis result of each log fragment;

[0121] According to the row structure of each single-line log in each log fragment, each single-line log is input into the row structure diagnosis model corresponding to its own row structure to obtain the row structure diagnosis result of each single-line log;

[0122] Based on all segment structure diagnosis results and all row structure diagnosis results of the target log, abnormal log segments and / or abnormal log fields in the target log are determined.

[0123] In some embodiments, when the terminal processor 901 inputs the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type to obtain the fragment structure diagnosis results of each log fragment, it specifically performs the following steps: remove redundant numerical values ​​in each log fragment to obtain the key fields of each log fragment; generate the row code of each line of log according to all the key fields in each line of log in each log fragment; mask part of the row code in each log fragment to obtain the fuzzy code sequence of each log fragment, and input each fuzzy code sequence into the corresponding fragment structure diagnosis model, the fragment structure diagnosis model is used to predict the unknown row code according to the known row code in the fuzzy code sequence; determine the predicted row code output by each fragment structure diagnosis model for each log fragment; compare the predicted row code of each log fragment with the original true row code corresponding to the predicted row code, and if they are inconsistent, determine that the log fragment corresponding to the original true row code is an abnormal log fragment.

[0124] In some embodiments, the fragment structure diagnosis model is trained based on a fuzzy coding sequence of at least one sample log, where the fuzzy coding sequence is obtained by converting each line of log in the sample log into a line code and then masking part of the line code; during the training process of the fragment structure diagnosis model, the fuzzy coding sequence of each sample log is output with a predicted line code, and the loss value of the loss function is calculated based on the predicted line code and the standard line code of each sample log, and the model parameters are adjusted based on the loss value until the fragment structure diagnosis model meets the preset convergence conditions.

[0125] In some embodiments, when the terminal processor 901 executes the following steps when inputting each single-line log into a row structure diagnosis model corresponding to its own row structure according to the row structure of each single-line log in each log fragment to obtain the row structure diagnosis result of each single-line log: covering the first half of the fields of each single-line log in each log fragment and inputting the first row structure diagnosis model corresponding to the row structure of the single-line log to obtain the predicted second half of the fields output by each first row structure diagnosis model, the first row structure diagnosis model is used to predict the unknown field based on the known field; comparing the predicted second half of the fields of each single-line log with the original real field corresponding to the predicted second half of the fields, if there is inconsistency, determining the original real field to be an abnormal log field; covering the second half of the fields of each single-line log in each log fragment and inputting the second row structure diagnosis model corresponding to the row structure of the single-line log to obtain the predicted first half of the fields output by each second row structure diagnosis model, the second row structure diagnosis model is used to predict the unknown field based on the known field; comparing the predicted first half of the fields of each single-line log with the original real field corresponding to the predicted first half of the fields, if there is inconsistency, determining the original real field to be an abnormal log field.

[0126] In some embodiments, the row structure diagnosis model is trained based on at least one sample single-line log, and the sample single-line log consists of masked fields and uncovered fields; during the training process of the row structure diagnosis model, the predicted field corresponding to the masked field is output for each sample single-line log, and the loss value of the loss function is calculated based on the predicted field and the standard field of each sample single-line log, and the model parameters are adjusted based on the loss value until the row structure diagnosis model meets the preset convergence conditions.

[0127] In some embodiments, after determining the abnormal log fragment and / or abnormal log field in the target log, the terminal processor 901 further specifically performs the following steps: if there is an abnormal log fragment, a fragment abnormality identifier is added to the abnormal log fragment; if there is an abnormal log field, a field abnormality identifier is added to the abnormal log field; wherein the fragment abnormality identifier and the field abnormality identifier have different page display effects.

[0128] In the several embodiments provided in this specification, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0129] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above computer program product includes one or more computer instructions. When the above computer program instructions are loaded and executed on a computer, the above process or function according to the embodiment of this specification is generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium or transmitted by the above computer-readable storage medium. The above computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVD)), or semiconductor media (for example, solid state drives (SSD)), etc.

[0131] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0132] In addition, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the logs and user operations involved in this specification are all obtained with full authorization.

[0133] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0134] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] The above is a description of a log diagnosis method, device, storage medium, and terminal provided in the embodiments of this specification. For technicians in this field, according to the ideas of the embodiments of this specification, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of this specification.

Claims

1. A log diagnosis method, the method comprising: Obtain a target log corresponding to the user operation, and split the target log into at least one log segment according to the data operation type, each log segment corresponding to a different data operation type; Input the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type, and obtain the fragment structure diagnosis result of each log fragment; According to the row structure of each single-line log in each log fragment, each single-line log is input into the row structure diagnosis model corresponding to its own row structure to obtain the row structure diagnosis result of each single-line log; Based on all the segment structure diagnosis results and all the row structure diagnosis results of the target log, abnormal log segments and / or abnormal log fields in the target log are determined.

2. The method according to claim 1, wherein the log segments of each data operation type are input into the segment structure diagnosis model corresponding to the data operation type to obtain the segment structure diagnosis result of each log segment, including: Remove redundant values ​​from each log segment to obtain the key fields of each log segment; Generate a line code for each line of log according to all key fields in each line of log in each log fragment; Masking part of the row codes in each log segment to obtain a fuzzy code sequence of each log segment, and inputting each fuzzy code sequence into a corresponding segment structure diagnosis model, wherein the segment structure diagnosis model is used to predict unknown row codes according to known row codes in the fuzzy code sequence; Determine the predicted row code output by each segment structure diagnosis model for each log segment; The predicted row code of each log segment is compared with the original true row code corresponding to the predicted row code. If they are inconsistent, the log segment corresponding to the original true row code is determined to be an abnormal log segment.

3. According to the method of claim 2, the segment structure diagnosis model is trained based on a fuzzy code sequence of at least one sample log, and the fuzzy code sequence is obtained by converting each line of the sample log into a line code and then masking part of the line code; During the training process of the fragment structure diagnosis model, the predicted row code is output for the fuzzy code sequence of each sample log, and the loss value of the loss function is calculated based on the predicted row code and the standard row code of each sample log, and the model parameters are adjusted based on the loss value until the fragment structure diagnosis model meets the preset convergence conditions.

4. The method according to claim 1, wherein, according to the row structure of each single-line log in each log segment, each single-line log is input into a row structure diagnosis model corresponding to its own row structure to obtain a row structure diagnosis result of each single-line log, comprising: Mask the first half of the fields of each single-line log in each log segment and input the first line structure diagnosis model corresponding to the line structure of the single-line log to obtain the predicted second half of the fields output by each first line structure diagnosis model, where the first line structure diagnosis model is used to predict unknown fields based on known fields; Compare the predicted second half field of each single-line log with the original real field corresponding to the predicted second half field, and if they are inconsistent, determine that the original real field is an abnormal log field; Mask the second half of the fields of each single-line log in each log fragment and input the second line structure diagnosis model corresponding to the line structure of the single-line log to obtain the predicted first half of the fields output by each second line structure diagnosis model, where the second line structure diagnosis model is used to predict unknown fields based on known fields; The predicted first half field of each single-line log is compared with the original real field corresponding to the predicted first half field. If they are inconsistent, it is determined that the original real field is an abnormal log field.

5. The method according to claim 4, wherein the row structure diagnosis model is trained based on at least one sample single-row log, wherein the sample single-row log consists of a covered field and an uncovered field; During the training process of the row structure diagnosis model, the predicted field corresponding to the masked field is output for each sample single line log, and the loss value of the loss function is calculated based on the predicted field and the standard field of each sample single line log, and the model parameters are adjusted based on the loss value until the row structure diagnosis model meets the preset convergence conditions.

6. The method according to claim 1, after determining the abnormal log segment and / or abnormal log field in the target log, further comprising: If there is an abnormal log segment, adding a segment abnormality identifier to the abnormal log segment; If there is an abnormal log field, adding a field abnormality flag to the abnormal log field; Among them, the segment abnormality flag and the field abnormality flag have different page display effects.

7. A log diagnosis device, comprising: A log acquisition module is used to acquire a target log corresponding to a user operation, and split the target log into at least one log segment according to a data operation type, each log segment corresponding to a different data operation type; A fragment diagnosis module is used to input the log fragments of each data operation type into the fragment structure diagnosis model corresponding to the data operation type, and obtain the fragment structure diagnosis result of each log fragment; The field diagnosis module is used to input each single-line log into a row structure diagnosis model corresponding to its own row structure according to the row structure of each single-line log in each log fragment, so as to obtain the row structure diagnosis result of each single-line log; The result acquisition module is used to determine abnormal log segments and / or abnormal log fields in the target log based on all segment structure diagnosis results and all row structure diagnosis results of the target log.

8. A computer program product comprising instructions, which, when executed on a computer or a processor, causes the computer or the processor to execute the steps of the method according to any one of claims 1 to 6.

9. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 6.

10. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

Images