Log error information processing method and device based on conditional random fields

Abstract

A method and device for processing log error information based on conditional random fields relates to the field of intelligent operation and maintenance and can be used in the financial field or other fields. The method includes: using a preset conditional random field feature model to process the acquired log text to obtain log error information; determining the log error time point based on the log error information, and using the log error time point to divide the log text to obtain multiple text segments; determining the corresponding stage information based on the text segments, linking the stage information of the text segments to obtain an error processing process chain set; determining the similarity corresponding to the acquired error information to be tested based on the similarity algorithm and the error processing process chain set, and processing the error information to be tested based on the similarity. The present invention obtains more effective information from the log text by extracting and mining the log text information, and is effectively applied to the auxiliary processing of log error information analysis, saving programmers' time and energy.

Description

Technical Field

[0001] The present invention relates to the technical field of log processing, and in particular to a method and device for processing log error information based on conditional random fields. Background Art

[0002] When various internet service providers provide network services to a wide range of users, their service platforms must perform millions of computing operations, generating massive amounts of logs, including a large number of error logs. Some errors may be caused by user issues, while others may be caused by issues on the service platform itself. In many cases, multiple errors resulting from the same cause may contain different content.

[0003] Currently, operations and maintenance personnel are often overwhelmed by massive error logs and have to perform a large amount of manual troubleshooting, resulting in low efficiency in troubleshooting system problems. It is difficult to accurately locate the cause of the error and how to solve the abnormal problem in a timely manner. Summary of the Invention

[0004] In response to the problems existing in the prior art, the main purpose of the embodiments of the present invention is to provide a method and device for processing log error information based on conditional random fields, so as to effectively analyze log error information, improve efficiency, and save labor costs.

[0005] To achieve the above objectives, an embodiment of the present invention provides a method for processing log error information based on conditional random fields, the method comprising:

[0006] Using the preset conditional random field feature model, the acquired log text is processed to obtain log error information;

[0007] According to the log error information, determine the log error time point, and use the log error time point to divide the log text into multiple text segments;

[0008] According to the text fragment, the corresponding stage information is determined, and the stage information of the text fragment is linked to obtain an error processing process chain set;

[0009] According to the similarity algorithm and the error processing process chain set, the similarity corresponding to the acquired error information to be tested is determined, and the error information to be tested is processed according to the similarity.

[0010] Optionally, in an embodiment of the present invention, the method further includes: obtaining multiple log texts from an error log database.

[0011] Optionally, in one embodiment of the present invention, the obtained log text is processed using a preset conditional random field feature model to obtain log error information including:

[0012] Preprocess the obtained log text;

[0013] The pre-processed log text is analyzed using the preset conditional random field feature model to obtain log error information.

[0014] Optionally, in an embodiment of the present invention, preprocessing the obtained log text includes: removing the error thread number and descriptive information in the log text.

[0015] Optionally, in one embodiment of the present invention, the time point of the log error is determined according to the log error information, and the log text is divided using the log error time point to obtain multiple text segments including:

[0016] Determine the log error reporting time point and phase transition time point based on the log error information;

[0017] According to the log error reporting time point and the stage transition time point, the log text is divided into multiple text segments.

[0018] Optionally, in one embodiment of the present invention, linking the stage information of the text segments to obtain an error processing process chain set includes:

[0019] Scoring the stage information using a preset scoring model to obtain a stage scoring result, and classifying the text segments based on the stage scoring result;

[0020] Link the stage information of the classified text segments to obtain the error handling process chain corresponding to the text segments;

[0021] According to the error processing process chain corresponding to each text segment, an error processing process chain set is obtained.

[0022] Optionally, in an embodiment of the present invention, determining the similarity corresponding to the acquired error information to be tested according to the similarity algorithm and the error processing process chain set includes:

[0023] Obtain the error information to be tested in the log to be tested, and obtain the feature vector to be tested based on the error information to be tested;

[0024] According to the stage information belonging to the same stage in the error processing process chain set, the stage feature vector is obtained;

[0025] According to the similarity algorithm, the similarity between the feature vector to be tested and the stage feature vector is determined.

[0026] An embodiment of the present invention further provides a log error information processing device based on a conditional random field, the device comprising:

[0027] The error information module is used to process the acquired log text using a preset conditional random field feature model to obtain log error information;

[0028] The text segmentation module is used to determine the time point of the log error according to the log error information, and use the log error time point to divide the log text into multiple text segments;

[0029] The processing process chain module is used to determine the corresponding stage information according to the text fragment, and link the stage information of the text fragment to obtain an error processing process chain set;

[0030] The information processing module is used to determine the similarity corresponding to the acquired error information to be tested according to the similarity algorithm and the error processing process chain set, and process the error information to be tested according to the similarity.

[0031] Optionally, in one embodiment of the present invention, the device further includes a log acquisition module, configured to acquire a plurality of log texts from an error log database.

[0032] Optionally, in one embodiment of the present invention, the error information module includes:

[0033] A preprocessing unit, used for preprocessing the acquired log text;

[0034] The error information unit uses a preset conditional random field feature model to perform text structure analysis on the preprocessed log text to obtain log error information.

[0035] Optionally, in one embodiment of the present invention, the pre-processing unit is further configured to remove the error thread number and descriptive information in the log text.

[0036] Optionally, in one embodiment of the present invention, the text segment module includes:

[0037] A time point unit, configured to determine a log error reporting time point and a phase transition time point according to the log error reporting information;

[0038] The text segmentation unit is used to segment the log text according to the log error reporting time point and the stage transition time point to obtain multiple text segments.

[0039] Optionally, in one embodiment of the present invention, the processing process chain module includes:

[0040] A classification processing unit, configured to score the stage information using a preset scoring model to obtain a stage scoring result, and classify the text fragments according to the stage scoring result;

[0041] A processing process chain unit is used to link the stage information of the text fragment after classification processing to obtain the error processing process chain corresponding to the text fragment;

[0042] The process chain collection unit is used to obtain an error processing process chain collection based on the error processing process chain corresponding to each text segment.

[0043] Optionally, in one embodiment of the present invention, the information processing module includes:

[0044] The information unit to be tested is used to obtain the error information to be tested in the log to be tested, and obtain the feature vector to be tested according to the error information to be tested;

[0045] The stage information unit is used to obtain a stage feature vector based on the stage information belonging to the same stage in the error processing process chain set;

[0046] The similarity unit is used to determine the similarity between the feature vector to be tested and the feature vector of the stage according to a similarity algorithm.

[0047] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0048] The present invention also provides a computer-readable storage medium storing a computer program for executing the above method.

[0049] The present invention extracts and mines useful information from log texts, making it possible to intuitively display previously dormant valuable data, thereby achieving in-depth mining of log text information. More effective information can be obtained from log texts, and effectively applied to auxiliary processing decisions for log error information analysis, saving programmers the time and energy they would otherwise have spent on searching and reading logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is a flowchart of a method for processing log error information based on conditional random fields according to an embodiment of the present invention;

[0052] Figure 2 This is a flow chart of obtaining log error information in an embodiment of the present invention;

[0053] Figure 3 A flowchart of obtaining multiple text segments in an embodiment of the present invention;

[0054] Figure 4 A flowchart of obtaining an error handling process chain set in an embodiment of the present invention;

[0055] Figure 5 This is a flow chart for determining the similarity of error information to be tested in an embodiment of the present invention;

[0056] Figure 6 This is a flowchart of log error information processing based on conditional random fields in a specific embodiment of the present invention;

[0057] Figure 7 This is a schematic diagram of a conditional random field linear model in a specific embodiment of the present invention;

[0058] Figure 8 This is a structural diagram of a stage information storage unit in a specific embodiment of the present invention;

[0059] Figure 9 This is a structural diagram of a log error information processing device based on conditional random fields according to an embodiment of the present invention;

[0060] Figure 10 Schematic diagram of the structure of a log error information processing device based on conditional random fields in another embodiment of the present invention;

[0061] Figure 11 This is a schematic diagram of the structure of the error information module in an embodiment of the present invention;

[0062] Figure 12 This is a structural diagram of a text segment module in an embodiment of the present invention;

[0063] Figure 13 This is a schematic diagram of the structure of the processing process chain module in an embodiment of the present invention;

[0064] Figure 14 This is a schematic diagram of the structure of the information processing module in an embodiment of the present invention;

[0065] Figure 15 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The embodiments of the present invention provide a method and device for processing log error information based on conditional random fields, which can be used in the financial field and other fields. It should be noted that the method and device for processing log error information based on conditional random fields of the present invention can be used in the financial field, and can also be used in any field other than the financial field. The application field of the method and device for processing log error information based on conditional random fields of the present invention is not limited.

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] like Figure 1 The flowchart of a method for processing log error information based on conditional random fields according to an embodiment of the present invention is shown. The execution subject of the method for processing log error information based on conditional random fields provided by the embodiment of the present invention includes but is not limited to a computer. Figure 1 The methods shown include:

[0069] Step S1: Using a preset conditional random field feature model, the acquired log text is processed to obtain log error information.

[0070] Among them, the log text can be obtained from internationally renowned authoritative log databases such as CSDN. Using the preset conditional random field feature model, such as Figure 7 The conditional random field feature model shown in the figure is used to analyze the log error information in the log text. The log error information includes user IP, HTTP request method, request background interface information, interface parameters, service status, number of bytes transmitted, parent page, error name, and error description.

[0071] Step S2: determining the log error reporting time point according to the log error reporting information, and dividing the log text using the log error reporting time point to obtain multiple text segments.

[0072] The time node at which the error occurs in the log, that is, the log error time point, is identified from the log error information, and the log text is segmented according to the error processing stage to obtain multiple text segments.

[0073] Furthermore, the time nodes of error occurrence in the log are also identified from the log error information, which includes the time nodes marked with the processing stage transition, that is, the stage transition time points. The log text is segmented according to the error processing stage using the log error time points and the stage transition time points.

[0074] Step S3: determining the corresponding stage information according to the text fragment, and linking the stage information of the text fragment to obtain an error processing process chain set.

[0075] The corresponding stage information is determined from the text fragments. Specifically, the error manifestation, handling means, method used and solution effect of each stage are extracted from each text fragment. Furthermore, the extracted stage information is stored as a stage information unit for later processing.

[0076] Step S4: Determine the similarity corresponding to the acquired error information to be tested according to the similarity algorithm and the error processing process chain set, and process the error information to be tested according to the similarity.

[0077] The stage information of each error processing stage is scored using a preset conditional random field scoring feature model and stored in the corresponding stage information unit. The stage information units are linked in sequence using the scoring structure to form an error processing process chain.

[0078] Furthermore, the processing chains extracted from the large amount of error log text are formed into a processing chain set. A similarity algorithm, such as a conventional recommendation algorithm, is used to scan the processing chain set. A similarity is calculated between the error information to be tested in the obtained error log and the error information in the set to obtain a similarity score. Based on the similarity score, the error information to be tested is processed. Specifically, an accurate and targeted processing solution is provided for the error information to be tested based on the similarity score.

[0079] As an embodiment of the present invention, the method further includes: acquiring a plurality of log texts from an error log database.

[0080] Among them, log text can be obtained from internationally renowned authoritative log databases such as CSDN. Specifically, crawler technology can be used to crawl log text information from the authoritative program log library on the Internet, and then text mining technology can be used to extract and mine useful information from the log text.

[0081] Furthermore, the relevant log error information and processing data are evaluated, the more authoritative log error information processing data are screened, and a crawler is written to crawl the target text.

[0082] As an embodiment of the present invention, Figure 2 As shown, the obtained log text is processed using the preset conditional random field feature model to obtain the log error information including:

[0083] Step S21, preprocessing the obtained log text;

[0084] Step S22: Using a preset conditional random field feature model, perform text structure analysis on the pre-processed log text to obtain log error information.

[0085] Among them, the preset conditional random field feature model can be as follows Figure 7The linear model of the linear conditional random field shown in Figure 7 It is a set of nodes and edges connecting the nodes.

[0086] In this embodiment, preprocessing the obtained log text includes removing the error thread number and descriptive information in the log text.

[0087] Among them, the process of obtaining log error information includes: text preprocessing: analyzing the text structure characteristics; using the conditional random field feature model to mine the user IP, http request method, request background interface information, interface parameters, service status, number of transmitted bytes, parent page, error name, and error description information in the log.

[0088] Furthermore, text preprocessing specifically includes: removing the error thread number in the basic log information, and the number has no practical effect on the mined error information; based on the analysis of the log text structure, deleting the descriptive text paragraphs at the beginning and middle of the log text used to record information such as the operating system, browser version, browser kernel, and program language version, to improve the efficiency and accuracy of the mining method.

[0089] As an embodiment of the present invention, Figure 3 As shown, according to the log error information, the log error time point is determined, and the log text is divided using the log error time point to obtain multiple text segments including:

[0090] Step S31, determining the log error reporting time point and the phase transition time point according to the log error reporting information;

[0091] Step S32: dividing the log text according to the log error reporting time point and the stage transition time point to obtain multiple text segments.

[0092] The time point of the log error and the phase transition time point can be determined from the log error information. By detecting time descriptors, it can be determined whether the time point is the time point at which the error occurred, and the text can be divided into segments according to the time points. Furthermore, the text segments are marked with serial numbers in order.

[0093] As an embodiment of the present invention, Figure 4 As shown, the stage information of the text fragment is linked to obtain the error processing process chain set including:

[0094] Step S41, using a preset scoring model to score the stage information to obtain a stage scoring result, and classifying the text fragments according to the stage scoring result;

[0095] Step S42: linking the stage information of the classified text segments to obtain an error processing process chain corresponding to the text segments;

[0096] Step S43: obtaining an error processing process chain set according to the error processing process chain corresponding to each text segment.

[0097] Among them, the preset scoring model can be an existing conditional random field scoring feature model, and the specific scoring process includes: detecting the solution effect descriptive words; using the solution effect scoring model (which can be an existing conditional random field module) to classify the words and find the category of the descriptive words; storing the score corresponding to the category in the stage information unit.

[0098] Specifically, the scoring scores are divided into the following categories: very good 3 points, good 2 points, average 1 point, poor -1 point, poor -2 points, and very poor -3 points.

[0099] As an embodiment of the present invention, Figure 5 As shown, according to the similarity algorithm and the error processing process chain set, determining the similarity corresponding to the error information to be tested includes:

[0100] Step S51, obtaining the error information to be tested in the log to be tested, and obtaining the feature vector to be tested according to the error information to be tested;

[0101] Step S52: obtaining a stage feature vector based on stage information belonging to the same stage in the error processing process chain set;

[0102] Step S53: Determine the similarity between the feature vector to be tested and the stage feature vector according to a similarity algorithm.

[0103] The process of obtaining the error processing process chain includes: obtaining the error information to be tested from the log to be tested, forming the obtained error information to be tested into a feature vector to be tested; and performing similarity calculation between the feature vector to be tested and a feature vector library composed of error information of different process chains in the processing process chain set and error information of different processing stages of the same process chain. The formula is as follows:

[0104]

[0105] Where d1 and d2 are two feature vectors, ai and bi are the i-th eigenvalue word components in each feature vector. Sim(d1, d2) is the similarity between d1 and d2. Furthermore, the similarity is sorted to form a recommendation sequence.

[0106] In a specific embodiment of the present invention, Figure 6 The flowchart of log error information processing based on conditional random fields shown in the figure specifically includes the following steps:

[0107] Step (1) uses a preset conditional random field feature model to mine the basic error information in the text (user IP, http request method, request background interface information, interface parameters, service status, number of bytes transmitted, parent page, error name, error description information);

[0108] Step (2) identifying the time nodes of error occurrence and the time nodes of processing stage transition marks in the log, and dividing the log text into segments according to the error processing stage;

[0109] Step (3) extracting the error manifestation, handling means / method used and solution effect of each stage in each text segment and storing them as a stage information unit;

[0110] Step (4) scores the effect of each processing stage according to the conditional random field scoring feature model and stores it in the corresponding stage information unit, and finally links the stage information units in sequence to form an error processing process chain;

[0111] Step (5) forms a processing process chain set by extracting the processing process chains from a large number of error log texts, uses a recommendation algorithm to scan the processing process chain set, calculates the similarity between the error information in the log and the error information in the set, and provides accurate and targeted processing solutions.

[0112] Before the above step (1), data needs to be acquired to achieve the rating of major log database websites. In this example, the error log database in CSDN is selected.

[0113] In the above step (1), the conditional random field feature model is used to mine the basic error information in the text (user IP, http request method, request background interface information, interface parameters, service status, number of bytes transmitted, parent page, error name, error description information). The specific principle is as follows:

[0114] like Figure 7 It is a linear model of linear conditional random fields. A graph is a set of nodes and edges connecting the nodes. The nodes and edges are denoted as v and e respectively, the sets of nodes and edges are denoted as V and E respectively, and the graph is denoted as G = (V, E). An undirected graph is a graph with no direction on the edges. A probabilistic graphical model is a probability distribution represented by a graph. Suppose there is a joint probability distribution P(Y), where Y∈y is a set of random variables. The probability distribution is represented by an undirected graph G, that is, in the graph G, the node v∈V represents a random variable Yv, and Y=Y v | v∈V ; The edge e∈E represents the probabilistic dependency between random variables.

[0115] By observing the sequence x(x1,x2,…,x n ) and the state sequence y(y1,y2,…,y n), use t m (y n -1,y n ,x,n) represents the transfer characteristic function between the observation sequence markers n-1 and n, s k (y n ,x,n) represents the state characteristic function of the observation sequence marker position n, using a function f m (y n -1,y n ,x,n) to unify the characteristic functions in the future, according to the basic theory of random fields:

[0116]

[0117] Where M(x:△) is the normalized probability of all state sequences after input normalization. The left side of the equal sign in formula (2) represents the conditional probability of predicting the output sequence y given the input sequence x. Formula (2) is the preset conditional random field feature model, also known as the conditional probability distribution.

[0118] Further, t m It represents the transfer characteristic function defined at two adjacent marker positions in the observation sequence, which is used to characterize the correlation between adjacent marker variables and the influence of the observation sequence on them. k Represents the state characteristic function at the mark position n of the observation sequence, which is used to characterize the influence of the observation sequence on the mark variable.

[0119] For example, in part-of-speech tagging, how can we determine the reliability of a given tag sequence? The transfer feature function primarily determines whether two adjacent tags are reasonable, for example, a verb + verb is grammatically incoherent. The state feature function determines whether the observed value and the corresponding tag are reasonable, for example, a word ending in "ly" -> adverb is more reasonable. Therefore, it is possible to define a set of feature functions and use this set of feature functions to score a standard sequence, thereby selecting reliable tag sequences. Each feature function can be used to score a standard sequence, and the combined scores of all feature functions in the set for the same tag sequence are the final score for the tag sequence. Feature functions are empirical properties.

[0120] The information mining process using a pre-set conditional random field feature model specifically includes: labeling each item in the error message; obtaining interface parameter and error name recognition models based on the conditional random field model learning process; using the labeled interface parameters and error name as features and incorporating them into the basic features for error information extraction; and finally, obtaining the error information extraction model based on the conditional random field model learning process. When presented with a new text, the interface parameters and error name recognition models are first used to identify the interface parameters and error names, respectively. The recognition results are then directly incorporated into the target text as features. The error information is then identified using the error information extraction model combined with the conditional random field decoding algorithm. Finally, the error information is extracted based on the labeling.

[0121] Furthermore, the specific steps of using the preset conditional random field feature model for information mining are: text preprocessing; analyzing the text structure characteristics; using the conditional random field feature model to mine the user IP, http request method, request background interface information, interface parameters, service status, number of transmitted bytes, parent page, error name, and error description information in the log text.

[0122] Furthermore, the specific steps of text preprocessing are: remove the number of the error thread in the basic information of the log, and the number has no practical effect on the mined error information; based on the analysis of the log text structure, delete the descriptive text paragraphs at the beginning and middle of the log text used to record information such as the operating system, browser version, browser kernel, and program language version, so as to improve the efficiency and accuracy of the mining method.

[0123] In this embodiment, the above steps identify the time nodes when errors occur in the log and the time nodes marked with transitions in the processing stages, and divide the log text into segments according to the error processing stages. The sequential marking sequence number examples are shown in Table 1, which shows the segment division table for the processing stages.

[0124] Table 1

[0125]

[0126]

[0127] The specific process of text segmentation includes: detecting time description words; judging whether the time node is the node where the error occurs; dividing the text into segments according to the time nodes; and marking the segments with serial numbers in order.

[0128] Wherein, the stage information storage unit mentioned in the above step (3) is as follows: Figure 8 Table 2 is a specific example. In addition, a specific scoring example of step (4) is shown in Table 3 (description word score table).

[0129] Table 2

[0130] <![CDATA[ ModuleNotFoundError :No module named′ Cryptodome′ ]]> <![CDATA[pip install pycryptodomex ]]> 34 3 <![CDATA[ ModuleNotFoundError :No module named′ gnupg′ ]]> <![CDATA[pip install gnupg ]]> 22 3 <![CDATA[ rosbag .bag. ROSBagException :unsupportedcompression type:lz4]]> pip install lz4 45 -1 <![CDATA[ rosbag .bag. ROSBagException :unsupportedcompression type:lz4]]> <![CDATA[python2 extract_bag_images. py ]]> 56 2

[0131] Table 3

[0132] sentence Descriptive words Synonyms 1 Score Successfully resolved the error success smoothly 3 More serious errors More serious More -2 The error cannot be resolved Unable Still not -1

[0133] Among them, the specific scoring categories are: very good 3 points, good 2 points, average 1 point, poor -1 point, poor -2 points, very poor -3 points.

[0134] The above step (5) forms a processing process chain set by extracting the processing process chains from a large amount of log text, uses the recommendation algorithm to scan the processing process chain set, calculates the similarity between the error information in the log and the error information in the set, and provides accurate and targeted solutions. The specific steps are: obtaining the error information in the log (user IP, http request method, request background interface information, interface parameters, service status, number of bytes transmitted, parent page, error name, error description information); forming the new error information obtained into a feature vector to be tested; calculating the similarity between the feature vector to be tested and the feature vector library composed of error information of different process chains in the processing process chain set and different processing stages of the same process chain, the formula is shown in formula (1); sorting in order of similarity to form a recommendation sequence.

[0135] Specifically, let's take an example to illustrate how to use the above theory to calculate the similarity of text. For the sake of simplicity, we start with sentences.

[0136] Sentence A: This leather boot is too big. That size is right.

[0137] Sentence B: This leather boot is not too small, that one is more suitable

[0138] The basic idea is that if the words used in two sentences are more similar, their contents should be more similar. Therefore, we can start with the word frequency to calculate their similarity.

[0139] The first step is word segmentation.

[0140] Sentence A: This boot is too big. That one is the right size.

[0141] Sentence B: This leather boot is not too small, that one is more suitable.

[0142] The second step is to list all the words.

[0143] This leather boot is too big. That one is suitable. No, it is too small.

[0144] The third step is to calculate the word frequency.

[0145] Sentence A: This 1, leather boots 1, size 2, 1 too big. That 1, fits 1, not 0, 0 too small, 0 too big.

[0146] Sentence B: This 1, leather boots 1, size 1, is 0 too big. That 1, fits 1, not 1, is 1 smaller, is 1 more

[0147] The fourth step is to write the word frequency vector.

[0148] Sentence A: (1, 1, 2, 1, 1, 1, 0, 0, 0)

[0149] Sentence B: (1, 1, 1, 0, 1, 1, 1, 1)

[0150] At this point, the question becomes how to calculate the similarity between two vectors. We can use the angle between them to determine the similarity between the two vectors. The smaller the angle, the more similar they are.

[0151] Use the above formula to calculate the two sentence vectors:

[0152] The vector cosine value of sentence A: (1, 1, 2, 1, 1, 1, 0, 0, 0) and sentence B: (1, 1, 1, 0, 1, 1, 1, 1) is used to determine the similarity between the two sentences.

[0153] The cosine value of the angle in the calculation result is 0.81, which is very close to 1. Therefore, the above sentences A and B are basically similar.

[0154] By extracting and mining useful information from log text, this method allows previously dormant valuable data to be intuitively displayed. This enables deep mining of log text information, allowing for the acquisition of more effective information from log texts. This information can be effectively applied to assist in decision-making during log error analysis, saving programmers time and effort spent searching and reading logs. Furthermore, by evaluating the effectiveness of error resolution, recommended solutions and methods are made more objective.

[0155] like Figure 9 FIG. 1 is a schematic diagram of a log error information processing device based on a conditional random field according to an embodiment of the present invention. The device shown in the figure includes:

[0156] The error information module 10 is used to process the acquired log text using a preset conditional random field feature model to obtain log error information;

[0157] The text segmentation module 20 is used to determine the log error reporting time point according to the log error reporting information, and divide the log text into multiple text segments using the log error reporting time point;

[0158] The processing process chain module 30 is used to determine the corresponding stage information according to the text segment, and link the stage information of the text segment to obtain an error processing process chain set;

[0159] The information processing module 40 is configured to determine the similarity corresponding to the acquired error information to be tested according to a similarity algorithm and the error processing process chain set, and process the error information to be tested according to the similarity.

[0160] As an embodiment of the present invention, Figure 10 As shown, the device further includes a log acquisition module 50 for acquiring a plurality of log texts from an error log database.

[0161] As an embodiment of the present invention, Figure 11 As shown, the error information module 10 includes:

[0162] A preprocessing unit 11 is used to preprocess the acquired log text;

[0163] The error information unit 12 uses a preset conditional random field feature model to perform text structure analysis on the pre-processed log text to obtain log error information.

[0164] In this embodiment, the pre-processing unit 11 is further configured to remove the error thread number and descriptive information in the log text.

[0165] As an embodiment of the present invention, Figure 12 As shown, the text segment module 20 includes:

[0166] A time point unit 21 is used to determine a log error reporting time point and a phase transition time point according to the log error reporting information;

[0167] The text segmentation unit 22 is used to segment the log text according to the log error reporting time point and the stage transition time point to obtain multiple text segments.

[0168] As an embodiment of the present invention, Figure 13 As shown, the processing chain module 30 includes:

[0169] The classification processing unit 31 is used to score the stage information using a preset scoring model to obtain a stage scoring result, and classify the text segment according to the stage scoring result;

[0170] The processing process chain unit 32 is used to link the stage information of the text segment after classification processing to obtain the error processing process chain corresponding to the text segment;

[0171] The process chain collection unit 33 is used to obtain an error processing process chain collection according to the error processing process chain corresponding to each text segment.

[0172] As an embodiment of the present invention, Figure 14 As shown, the information processing module 40 includes:

[0173] The test information unit 41 is used to obtain the test error information in the test log and obtain the test feature vector according to the test error information;

[0174] The stage information unit 42 is used to obtain a stage feature vector based on the stage information belonging to the same stage in the error processing process chain set;

[0175] The similarity unit 43 is used to determine the similarity between the feature vector to be tested and the feature vector of the stage according to a similarity algorithm.

[0176] Based on the same application concept as the aforementioned method for processing log error information based on a conditional random field, the present invention also provides the aforementioned device for processing log error information based on a conditional random field. Since the principle of solving the problem of the device for processing log error information based on a conditional random field is similar to that of the method for processing log error information based on a conditional random field, the implementation of the device for processing log error information based on a conditional random field can be referred to the implementation of the method for processing log error information based on a conditional random field, and the repeated parts will not be repeated here.

[0177] By extracting and mining useful information from log text, this method allows previously dormant valuable data to be intuitively displayed. This enables deep mining of log text information, allowing for the acquisition of more effective information from log texts. This information can be effectively applied to assist in decision-making during log error analysis, saving programmers time and effort spent searching and reading logs. Furthermore, by evaluating the effectiveness of error resolution, recommended solutions and methods are made more objective.

[0178] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0179] The present invention also provides a computer-readable storage medium storing a computer program for executing the above method.

[0180] like Figure 15 As shown, the electronic device 600 may further include: a communication module 110, an input unit 120, an audio processor 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily have to include Figure 15In addition, the electronic device 600 may also include all components shown in Figure 15 For components not shown, reference may be made to the prior art.

[0181] like Figure 15 As shown, the central processing unit 100 is sometimes also referred to as a controller or an operation control unit, and may include a microprocessor or other processor device and / or logic device. The central processing unit 100 receives inputs and controls the operations of various components of the electronic device 600 .

[0182] Memory 140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information and may also store programs that execute the relevant information. The CPU 100 may execute the programs stored in memory 140 to implement information storage or processing.

[0183] The input unit 120 provides input to the CPU 100. The input unit 120 may be, for example, a keypad or touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0184] The memory 140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), or a SIM card. Alternatively, it may be a memory that retains information even when power is off, can be selectively erased, and is provided with more data. Examples of such memory are sometimes referred to as EPROMs. The memory 140 may also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142 for storing application programs and function programs or processes for executing the operations of the electronic device 600 via the central processing unit 100.

[0185] The memory 140 may also include a data storage unit 143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 may include various driver programs for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0186] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processor 100 to provide input signals and receive output signals, which may be the same as in a conventional mobile communication terminal.

[0187] Based on different communication technologies, multiple communication modules 110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132, thereby implementing common telecommunication functions. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 130 is also coupled to the central processing unit 100, enabling local recording via the microphone 132 and playback of stored audio via the speaker 131.

[0188] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0192] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A method for processing log error information based on conditional random fields, characterized in that: The method comprises: Using the preset conditional random field feature model, the acquired log text is processed to obtain log error information; Determining a log error reporting time point based on the log error reporting information, and dividing the log text using the log error reporting time point to obtain a plurality of text segments; Determine the corresponding stage information according to the text fragment, and link the stage information of the text fragment to obtain an error processing process chain set; Determining the similarity corresponding to the acquired error information to be tested according to the similarity algorithm and the error processing process chain set, and processing the error information to be tested according to the similarity; The linking of the stage information of the text segments to obtain an error processing process chain set includes: Scoring the stage information using a preset scoring model to obtain a stage scoring result, and classifying the text fragments based on the stage scoring result; Linking the stage information of the classified text segments to obtain an error processing process chain corresponding to the text segments; According to the error processing process chain corresponding to each text segment, an error processing process chain set is obtained.

2. The method according to claim 1, characterized in that The method further includes: acquiring a plurality of log texts from an error log database.

3. The method according to claim 1, characterized in that The method of processing the acquired log text using the preset conditional random field feature model to obtain the log error information includes: Preprocess the obtained log text; The pre-processed log text is analyzed using the preset conditional random field feature model to obtain log error information.

4. The method according to claim 3, characterized in that The preprocessing of the obtained log text includes: removing the error thread number and descriptive information in the log text.

5. The method according to claim 1, wherein The step of determining a log error reporting time point based on the log error reporting information and dividing the log text using the log error reporting time point to obtain multiple text segments includes: Determine the log error reporting time point and the phase transition time point based on the log error reporting information; The log text is divided according to the log error reporting time point and the stage transition time point to obtain multiple text segments.

6. The method according to claim 1, characterized in that Determining the similarity corresponding to the acquired error information to be tested according to the similarity algorithm and the error processing process chain set includes: Obtaining the error information to be tested in the log to be tested, and obtaining the feature vector to be tested based on the error information to be tested; Obtaining a stage feature vector according to stage information belonging to the same stage in the error processing process chain set; The similarity between the feature vector to be tested and the stage feature vector is determined according to a similarity algorithm.

7. A log error information processing device based on conditional random fields, characterized in that: The device comprises: The error information module is used to process the acquired log text using a preset conditional random field feature model to obtain log error information; A text segmentation module is used to determine the log error reporting time point according to the log error reporting information, and divide the log text by using the log error reporting time point to obtain multiple text segments; A processing process chain module is used to determine the corresponding stage information according to the text segment, and link the stage information of the text segment to obtain an error processing process chain set; An information processing module, configured to determine the similarity corresponding to the acquired error information to be tested based on a similarity algorithm and the error processing process chain set, and process the error information to be tested based on the similarity; The processing chain module includes: A classification processing unit, configured to score the stage information using a preset scoring model to obtain a stage scoring result, and classify the text fragments according to the stage scoring result; A processing process chain unit is used to link the stage information of the text fragment after classification processing to obtain the error processing process chain corresponding to the text fragment; The process chain collection unit is used to obtain an error processing process chain collection based on the error processing process chain corresponding to each text segment.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for executing the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, implements the log error information processing method based on conditional random fields according to any one of claims 1 to 6.

Images