Log abnormality detection method, device, and equipment, storage medium, and program product

Abstract

The application discloses an abnormality detection method, device and equipment of a log, a storage medium and a program product. The method comprises the following steps: dividing a log sequence to be detected to obtain a plurality of subsequences, each subsequence comprising a plurality of logs; performing log event statistics on the plurality of logs contained in each subsequence to generate a count vector corresponding to each subsequence; performing abnormality detection on each subsequence according to the count vector corresponding to each subsequence to obtain a subsequence with an abnormality; and selecting a log with an abnormality from the subsequence with an abnormality. Compared with performing abnormality detection on all logs in the log sequence one by one, the abnormality detection method of the log disclosed by the application can reduce the number of log detections and improve the log detection efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for detecting log anomalies. Background Technology

[0002] Currently, problems often arise during system development or after system launch. The conventional method for discovering these problems mainly comes from user feedback, and then professional developers manually eliminate abnormal logs one by one in the system. This reduces the efficiency of abnormal log detection.

[0003] Therefore, improving the efficiency of log anomaly detection has become an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for detecting log anomalies.

[0005] According to one aspect of the embodiments of this application, an anomaly detection method for logs is provided. The anomaly detection method includes: dividing a log sequence to be detected into multiple subsequences, each subsequence including multiple logs; performing log event statistics on the multiple logs contained in each subsequence to generate a counting vector corresponding to each subsequence; performing anomaly detection on each subsequence based on the counting vector corresponding to each subsequence to obtain subsequences with anomalies; and selecting logs with anomalies from the subsequences with anomalies.

[0006] According to one aspect of the embodiments of this application, an anomaly detection device for logs is provided. The device includes: a partitioning module configured to partition a log sequence to be detected into multiple subsequences, each subsequence including multiple logs; a counting vector generation module configured to perform log event statistics on the multiple logs contained in each subsequence to generate a counting vector corresponding to each subsequence; an anomaly detection module configured to perform anomaly detection on each subsequence based on the counting vector corresponding to each subsequence to obtain subsequences with anomalies; and a selection module configured to select logs with anomalies from the subsequences with anomalies.

[0007] According to one aspect of the embodiments of this application, an electronic device is provided, including: a memory storing computer-readable instructions; and a processor reading the computer-readable instructions stored in the memory to execute the log anomaly detection method of any of the above-mentioned methods.

[0008] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the log anomaly detection method as described above.

[0009] According to one aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in the log anomaly detection method as described above.

[0010] In the technical solution provided by the embodiments of this application, the application performs log event statistics on the logs contained in each subsequence to generate a count vector corresponding to each subsequence, and performs anomaly detection on each subsequence based on the count vector to determine the subsequences with anomalies. Then, it detects abnormal logs for the subsequences with anomalies. Therefore, compared with performing anomaly detection on all logs in the log sequence one by one, it can reduce the number of logs to be detected and improve the log detection efficiency. Moreover, compared with manually detecting logs in the log sequence, it reduces labor costs and avoids the problem of false detection caused by human error.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0013] Figure 1 This is a schematic diagram illustrating the implementation environment of an anomaly detection method for logs, as shown in an exemplary embodiment.

[0014] Figure 2 This is a flowchart illustrating an anomaly detection method for logs, as shown in an exemplary embodiment of this application;

[0015] Figure 3 This is a schematic diagram illustrating a real-world application scenario of an anomaly detection method for logs, as shown in an exemplary embodiment.

[0016] Figure 4 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0017] Figure 5This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0018] Figure 6 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0019] Figure 7 This is a schematic diagram illustrating the effect of determining the projection vector in an anomaly detection method for logs according to this application.

[0020] Figure 8 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0021] Figure 9 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0022] Figure 10 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0023] Figure 11 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0024] Figure 12 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application;

[0025] Figure 13 This is a flowchart illustrating an anomaly detection method for logs in an exemplary application scenario.

[0026] Figure 14 This is a schematic diagram illustrating the process of log event extraction in an anomaly detection method for logs, as shown in an exemplary application scenario.

[0027] Figure 15 This is a simplified flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application.

[0028] Figure 16 This is a block diagram illustrating an anomaly detection device for logs, as shown in an exemplary embodiment of this application;

[0029] Figure 17 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0033] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0034] First, it's important to note that as systems grow on a large scale, expanding distributed systems by building thousands of machines and using computers with thousands of processors to scale high-performance computing, the complexity of the system necessitates the recording of the software's real-time running status in a file called a log. This log records detailed runtime information, such as an event during software operation. The log content includes constant and variable sections. The constant section typically contains fixed text content, which can be understood as an event template. The variable section displays runtime information, including ID information, code information, and parameter information. The variable section may output different values ​​at different times in the log.

[0035] Currently, system issues are primarily identified through user feedback, followed by professional developers manually sifting through system logs to identify anomalies. This reduces the efficiency of anomaly detection. Furthermore, manual detection not only increases labor costs but also carries the risk of missing anomalies due to human error.

[0036] Based on this, to improve the detection efficiency of abnormal logs, this application proposes an anomaly detection method, an anomaly detection device, an electronic device, a computer-readable storage medium, and a computer program product for logs. Specifically: the log sequence to be detected is divided into multiple sub-sequences, each sub-sequence including multiple logs; log event statistics are performed on the multiple logs contained in each sub-sequence to generate a counting vector corresponding to each sub-sequence; anomaly detection is performed on each sub-sequence based on the counting vector corresponding to each sub-sequence to obtain the sub-sequences containing anomalies; and the logs containing anomalies are selected from the sub-sequences containing anomalies. Compared to performing anomaly detection on all logs in the log sequence one by one, this reduces the number of logs to be detected and improves the log detection efficiency; and compared to manually detecting logs in the log sequence, it reduces labor costs and avoids false detection problems caused by human error.

[0037] Figure 1 This is a schematic diagram illustrating an implementation environment for an anomaly detection method for logs, as shown in an exemplary embodiment. The implementation environment includes a smart terminal 110 and a server 120. The smart terminal 110 and the server 120 are connected via a network communication connection. The network can include various connection types, such as wired or wireless communication links or fiber optic cables.

[0038] like Figure 1 As shown, during the log anomaly detection process, the smart terminal 110 can record the real-time status of the software to obtain a log sequence containing logs, and send the log sequence to the server 120. The server 120 receives the log sequence to be detected, divides the log sequence to be detected into multiple sub-sequences, and performs log event statistics on the multiple logs contained in each sub-sequence to generate a counting vector corresponding to each sub-sequence. Then, based on the counting vector corresponding to each sub-sequence, anomaly detection is performed on each sub-sequence to obtain the sub-sequences with anomalies. Finally, the logs with anomalies are determined from the sub-sequences with anomalies.

[0039] in, Figure 1 The smart terminal 110 shown can be any terminal device that supports logging, such as a smartphone, in-vehicle computer, tablet computer, laptop computer, or wearable device, but is not limited to these. Figure 1The server 120 shown is a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. No restrictions are placed on this. The smart terminal 110 can communicate with the server 120 via wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), and 5G (fifth-generation mobile information technology). No restrictions are placed on this as well.

[0040] Please see Figure 2 , Figure 2 This is a flowchart illustrating an anomaly detection method for logs, as shown in an exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown is specifically executed by server 120 within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.

[0041] The following section provides a detailed description of the log anomaly detection method proposed in this application embodiment, using the server as the specific execution subject.

[0042] like Figure 2 As shown, in an exemplary embodiment, the log anomaly detection method includes at least steps S210 to S240, which are described in detail below:

[0043] Step S210: Divide the log sequence to be detected into multiple subsequences.

[0044] The log sequence includes multiple log entries. To obtain the log sequence, you can sort the log entries according to their chronological order. It should be noted that the logs can be generated during application live streaming. See details below. Figure 3 , Figure 3 The left side shows the live streaming interface, and the right side shows the logs generated by the live stream.

[0045] A subsequence may contain multiple logs, and these subsequences can be obtained by partitioning the log sequence. For example, the server can divide the log sequence into an equal number of logs according to their chronological order, thus obtaining multiple subsequences. Another example is that the server can divide the logs in the log sequence into multiple subsequences according to their type.

[0046] Step S220: Perform log event statistics on the multiple logs contained in each subsequence to generate a count vector corresponding to each subsequence.

[0047] Log events can be understood as log templates, consisting of the constant parts of the log content.

[0048] The counting vector corresponding to a subsequence is determined based on the occurrence count of the log events corresponding to the logs contained in the subsequence. For example, if a subsequence contains 10 logs, 2 logs correspond to log events of type 1, 5 logs correspond to log events of type 2, and 3 logs correspond to log events of type 3, then the counting vector corresponding to the subsequence can be generated based on the 2 logs, 5 logs, and 3 logs corresponding to the first, second, and third types of log events, respectively.

[0049] The server parses the logs contained in each subsequence to obtain information such as time, thread, session ID, process ID, log level, category label, and log content for each log. Based on the log content obtained from the parsing, the server counts the occurrence of each log event in each subsequence and generates a count vector corresponding to the subsequence based on the occurrence count of each log event.

[0050] Step S230: Perform anomaly detection on each subsequence based on the counting vector corresponding to each subsequence to obtain the subsequences with anomalies.

[0051] An abnormal subsequence is a log sequence that contains at least one abnormal log.

[0052] In this embodiment, there are at least two cases for anomaly detection of each subsequence based on the counting vector corresponding to each subsequence, where:

[0053] Scenario 1: If the server detects an abnormal subsequence based on the counting vector corresponding to each subsequence, then step S240 is executed.

[0054] Scenario 2: If the server does not detect any abnormal subsequences based on the counting vector corresponding to each subsequence, it indicates that there are no abnormalities in the logs in the log sequence.

[0055] Step S240: Select logs containing anomalies from the subsequences of anomalies.

[0056] The server checks the logs of each identified abnormal subsequence, obtaining the abnormal logs within that subsequence. For example, the server can match the logs in the abnormal subsequence with preset logs to determine the matching degree between the logs in the abnormal subsequence and the corresponding preset logs. It then checks if the matching degree is less than a preset matching degree threshold; if so, the logs corresponding to the abnormal subsequence are determined to be abnormal. It should be noted that the preset logs can be normal logs, and these preset logs can be obtained from a historical log repository.

[0057] As can be seen, the log anomaly detection method in this embodiment performs log event statistics on the logs contained in each subsequence, generates a counting vector corresponding to each subsequence, and performs anomaly detection on each subsequence based on the counting vector to determine the subsequences with anomalies. Then, it detects abnormal logs in the subsequences with anomalies. Therefore, compared with performing anomaly detection on all logs in the log sequence one by one, it can reduce the number of logs to be detected and improve the log detection efficiency. Moreover, compared with manually detecting logs in the log sequence, it reduces labor costs and avoids false detection problems caused by human error.

[0058] Figure 4 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application. Figure 4 In the illustrated embodiment, the subsequences with anomalies are determined by the relationship between the counting vector corresponding to each subsequence and the projection vector corresponding to the counting vector. For other steps of log anomaly detection, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0059] like Figure 4 As shown, in an exemplary embodiment, the process of performing anomaly detection on each subsequence according to the counting vector corresponding to each subsequence in step S230 to obtain the subsequences with anomalies further includes at least steps S410 to S420, which are described in detail below:

[0060] Step S410: Project the counting vector corresponding to each subsequence to obtain the projection vector corresponding to each subsequence.

[0061] The server obtains the projection vector of each subsequence by projecting the counting vector corresponding to each subsequence. Specifically, the server can project the counting vectors corresponding to the subsequences according to preset projection rules to obtain the projection vectors corresponding to the counting vectors. For example, the server calculates the distance between the counting vector corresponding to the subsequence and a preset projection line, and determines the projection vector corresponding to the counting vector based on the calculated distance at the end of the preset projection line furthest from the counting vector.

[0062] Step S420: Based on the relationship between the counting vector and the projection vector corresponding to each subsequence, perform anomaly detection on each subsequence to obtain the subsequences with anomalies.

[0063] The relationship between the counting vector and the projection vector corresponding to the subsequence can be the spatial distance between the counting vector and the corresponding projection vector, or the vector size relationship between the counting vector and the corresponding projection vector, etc.

[0064] The server detects each subsequence based on the relationship between the counting vector and the projection vector corresponding to each subsequence, and identifies the subsequences that contain anomalies.

[0065] As an example, when the relationship between the counting vector and the projection vector corresponding to the subsequence is the spatial distance between them, this embodiment can adopt... Figure 5 Steps S510 to S520 shown implement step S420, as detailed below:

[0066] Step S510: Calculate the spatial distance between the counting vector and the projection vector corresponding to each subsequence to obtain the spatial distance value corresponding to each subsequence.

[0067] The server calculates the spatial distance between the counting vector and the projection vector corresponding to each subsequence, thus obtaining the spatial distance value for each subsequence. Specifically, the server can calculate the spatial distance between the counting vector and the projection vector based on the position coordinates of the counting vector in the normal space and the position coordinates of the projection vector in the abnormal space corresponding to the counting vector.

[0068] Step S520: If the spatial distance value corresponding to the subsequence is greater than the preset spatial distance threshold, a detection result is obtained to characterize the subsequence as having an anomaly.

[0069] The preset spatial distance threshold is a pre-set distance value.

[0070] The test results include at least two scenarios, among which:

[0071] In scenario one, if the server determines that the spatial distance between the counting vector and the projection vector corresponding to the subsequence is greater than a preset spatial distance threshold, the detection result is that the subsequence is abnormal.

[0072] In the second scenario, if the server determines that the spatial distance between the counting vector and the projection vector corresponding to the subsequence is less than or equal to a preset spatial distance threshold, the detection result is that the subsequence does not have any abnormalities.

[0073] Based on the spatial distance value corresponding to each subsequence determined in step S510, the server determines whether the spatial distance value corresponding to each subsequence is greater than a preset spatial distance threshold. If so, a detection result is obtained to characterize the anomaly of the subsequence.

[0074] As can be seen, the log anomaly detection method in this embodiment detects abnormal subsequences by using the relationship between the counting vector corresponding to each subsequence and the projection vector corresponding to the counting vector, which can improve the accuracy of abnormal subsequence detection.

[0075] Figure 6 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application. Figure 6 In the illustrated embodiment, step S410, which projects the counting vector corresponding to each subsequence to obtain the projection vector corresponding to each subsequence, further includes at least steps S610 to S630, which are detailed below:

[0076] Step S610: Group the logs contained in the log sequence according to the category label to obtain multiple log groups.

[0077] Category tags are labels used to categorize log entries. Specifically, category tags are obtained by the server through parsing and processing the logs.

[0078] Logs in the same log group have the same category label.

[0079] The server groups the logs in the log sequence into multiple log groups based on the category labels obtained from parsing the logs.

[0080] Step S620: For each log group, project the counting vector corresponding to the logs contained in the log group to obtain the projection vector corresponding to the logs contained in the log group.

[0081] In this embodiment, the method by which the server projects the count vectors corresponding to the logs in each log group can be found in [reference needed]. Figure 7 , Figure 7 The diagram shows the projection of the count vector corresponding to the logs within the log group, such as... Figure 7 As shown, the counting vectors corresponding to the logs contained in the log group exist in the normal space. The server projects the counting vectors in the normal space into the abnormal space to obtain the projection vector of the abnormal space.

[0082] It should be noted that the above method is also known as PCA statistical method, which is widely used in dimensionality reduction. Its core is to use orthogonal transformation to linearly transform a series of potentially related counting vectors, thereby projecting them into a series of linearly unrelated projection vectors, so as to project the high-dimensional counting vectors into a new coordinate system composed of a series of principal components.

[0083] It should also be noted that before projecting the counting vectors corresponding to the logs contained in the log group, this embodiment may select a preset number of counting vectors for projection. The preset number of counting vectors may be selected based on the importance of the counting vectors corresponding to the log group, that is, whether the corresponding counting vector is a principal component in the log group.

[0084] Step S630: Determine the projection vector corresponding to each subsequence based on the projection vectors corresponding to the logs contained in the multiple log groups.

[0085] Since each subsequence corresponds to a counting vector, this embodiment projects the counting vectors corresponding to the logs in the log groups obtained by grouping to obtain the projection vectors corresponding to the logs in each log group. Then, based on the projection vectors corresponding to the logs in each log group, the projection vector corresponding to each subsequence is determined.

[0086] As can be seen, the log anomaly detection method in this embodiment groups the logs in the log sequence according to the category label to obtain multiple log groups, and projects the counting vector corresponding to the logs in each log group to obtain the corresponding projection vector. Then, based on the projection vectors corresponding to the logs contained in multiple log groups, the projection vector corresponding to each subsequence is determined. Thus, by grouping, the logs in each log group are in the same dimension, reducing the interference of logs in other dimensions on the projection processing.

[0087] Figure 8 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application. Figure 8 In the embodiment shown, step S630, which determines the projection vector corresponding to each subsequence based on the projection vectors corresponding to the logs contained in multiple log groups, further includes at least steps S810 to S820, which are described in detail below:

[0088] Step S810: Obtain the projection vectors corresponding to logs belonging to the same subsequence.

[0089] After projecting the count vectors of the logs in the grouped logs, the corresponding projection vectors are obtained. Further, in order to determine the projection vector corresponding to each subsequence, this embodiment selects logs belonging to the same subsequence in each log group and determines the projection vectors corresponding to the logs belonging to the same subsequence.

[0090] Step S820: Calculate the average of the projection vectors corresponding to logs belonging to the same subsequence to obtain the projection vector corresponding to each subsequence.

[0091] Considering that logs in the same subsequence may be divided into different log groups, different logs belonging to the same subsequence have corresponding projection vectors in the corresponding log groups. In order to improve the accuracy of the projection vector corresponding to each subsequence, this embodiment calculates the average of the projection vectors corresponding to logs belonging to the same subsequence in each log group to obtain the projection vector corresponding to each subsequence.

[0092] As can be seen, the log anomaly detection method in this embodiment obtains the projection vector corresponding to each subsequence by averaging the projection vectors of logs belonging to the same subsequence in each log group. This improves the accuracy of obtaining the projection vector corresponding to each subsequence based on the averaging of the projection vectors of different logs belonging to the same subsequence.

[0093] Figure 9 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application. Figure 9 In the illustrated embodiment, in Figure 4 Based on the illustrated embodiment, step S420, which performs anomaly detection on each subsequence according to the relationship between the counting vector and the projection vector corresponding to each subsequence, and obtains the subsequences with anomalies, further includes at least steps S910 to S940, which are detailed below:

[0094] Step S910: Calculate the spatial distance between the counting vector and the projection vector corresponding to the logs in each log group to obtain the spatial distance value corresponding to the logs in each log group.

[0095] After grouping the log sequence into log groups, this embodiment obtains the spatial distance value of the logs contained in each log group based on the spatial distance between the counting vector and the projection vector corresponding to the logs in each log group.

[0096] Step S920: Obtain the spatial distance value corresponding to logs belonging to the same subsequence.

[0097] Considering that logs belonging to the same subsequence may be divided into different log groups, after determining the spatial distance value between the counting vector and the projection vector of the logs in each log group, in order to determine the spatial distance value of each subsequence, this embodiment can obtain the spatial distance value of the logs belonging to the same subsequence in each log group.

[0098] Step S930: Calculate the average spatial distance values ​​of logs belonging to the same subsequence to obtain the average spatial distance between the counting vector and the projection vector corresponding to each subsequence.

[0099] The spatial distance mean refers to the average spatial distance between the counting vector and the projection vector corresponding to logs contained in the same subsequence.

[0100] Since the same subsequence contains multiple logs, and each log corresponds to a spatial distance value, in order to accurately determine the spatial distance value corresponding to each subsequence, this embodiment calculates the average spatial distance value corresponding to the logs belonging to the same subsequence to obtain the average spatial distance between the counting vector and the projection vector corresponding to each subsequence.

[0101] Step S940: Based on the mean spatial distance between the counting vector and the projection vector corresponding to each subsequence, perform anomaly detection on each subsequence to obtain the subsequences with anomalies.

[0102] Based on the average spatial distance between the counting vector and the projection vector corresponding to each subsequence determined in step S930, the server determines whether a subsequence is abnormal by judging whether the average spatial distance between the counting vector and the projection vector corresponding to the subsequence is greater than a preset spatial distance threshold. If the average spatial distance between the counting vector and the projection vector corresponding to the subsequence is greater than the preset spatial distance threshold, the corresponding subsequence is determined to be abnormal; if the average spatial distance between the counting vector and the projection vector corresponding to the subsequence is less than or equal to the preset spatial distance threshold, the corresponding subsequence is determined not to be abnormal.

[0103] As can be seen, the log anomaly detection method in this embodiment calculates the average spatial distance between the counting vector and the projection vector of each subsequence by averaging the spatial distance values ​​of logs belonging to the same subsequence in each log group. Then, it performs anomaly subsequence detection based on the average spatial distance between the counting vector and the projection vector of each subsequence to identify the subsequences with anomalies. This improves the accuracy of obtaining the spatial distance value of each subsequence by processing based on the average spatial distance between the counting vector and the projection vector of each subsequence.

[0104] Figure 10 This is a flowchart illustrating an anomaly detection method for logs, as shown in yet another exemplary embodiment of this application. Figure 10 In the illustrated embodiment, step S220, which involves performing log event statistics on multiple logs contained in each subsequence and generating a count vector corresponding to each subsequence, further includes at least steps S1010 to S1030, which are detailed below:

[0105] Step S1010: For each subsequence, parse the category of the log events to which the logs contained in the subsequence correspond, and count the number of log events belonging to the same category to obtain the statistical count corresponding to the subsequence.

[0106] To obtain the counting vector corresponding to each subsequence, this embodiment parses the log events in each subsequence according to their categories, obtaining the log events of each subsequence. It then counts the number of log events belonging to the same category in each subsequence, obtaining the statistical count for each subsequence. For example, if a subsequence contains 10 logs, and parsing reveals that 2 logs in the subsequence correspond to log events belonging to the same category, 5 logs correspond to log events belonging to the same category, and 3 logs correspond to log events belonging to the same category, then the number of logs in each category in the subsequence is determined to be 2, 3, and 5, respectively.

[0107] Step S1020: Generate a matrix corresponding to each subsequence based on the statistical quantity corresponding to each subsequence.

[0108] This embodiment generates a matrix corresponding to each subsequence based on the statistical counts obtained in step 1010. Taking the example in step S1010 as an illustration, the statistical counts corresponding to the subsequences are 2, 3, and 5, respectively, and a matrix [2,3,5] can be formed based on the statistical counts of 2, 3, and 5.

[0109] Step S1030: Generate a counting vector corresponding to each subsequence based on the matrix corresponding to each subsequence.

[0110] The server generates a counting vector for each subsequence based on the matrix corresponding to each subsequence.

[0111] As can be seen, the log anomaly detection method in this embodiment counts the number of log events belonging to the same category in the subsequence, obtains the statistical quantity corresponding to the subsequence, generates a matrix corresponding to each subsequence based on the statistical quantity corresponding to the subsequence, and then generates a counting vector corresponding to each subsequence based on the matrix corresponding to each subsequence, thereby improving the accuracy of obtaining the counting vector corresponding to each subsequence.

[0112] Figure 11 This is a flowchart illustrating an anomaly detection method for logs, as shown in another exemplary embodiment of this application. Based on the above embodiment, before step S220, which performs log event statistics on the multiple logs contained in each subsequence and generates a count vector corresponding to each subsequence, the anomaly detection method for logs of this application further includes at least steps S1110 to S1130, which are detailed below:

[0113] Step S1110: Parse the log entries contained in the log sequence using word pairs to obtain the word pair information corresponding to the log entries.

[0114] Considering that each log entry contains both constant and variable portions, in order to accurately determine the constant and variable portions in each log entry, the server in this embodiment can perform word pair parsing on the log entries in the log sequence to obtain the corresponding word pair information. It should be noted that each log entry can correspond to multiple word pairs, where each word pair may represent the log content of the constant portion or the log content of the variable portion.

[0115] Step S1120: Determine the log event cluster to which the log belongs based on the word pair information corresponding to the log.

[0116] A log event cluster is a collection of logs that belong to the same log event.

[0117] The server determines the log event cluster to which a log belongs based on the word pairs corresponding to each log entry. Specifically, the server calculates the log's metric value based on the word pairs corresponding to the log entry and the word pairs included in the preset log event cluster, and determines the corresponding log event cluster based on the metric range to which each log's metric value belongs. It should be noted that the log's metric value can be understood as the similarity between the log's word pairs and the word pairs included in the preset log event cluster.

[0118] Step S1130: Extract log events from the log event cluster to obtain the log events of the log.

[0119] Since the same log event cluster contains the same log events, the server can extract the log events of a log cluster by retrieving the log events from the log event cluster. This can also be understood as the server determining the log events within a log event cluster based on the constant portions of the logs in that cluster, thus obtaining the log events of the logs contained within that cluster.

[0120] It should be noted that there may be instances where the constant portions of log events in two log event clusters are not significantly different, but they are divided into two separate log event clusters for separate log event extraction, leading to computational redundancy. Therefore, this embodiment can merge the log event clusters corresponding to log events with similar constant portions. Specifically, the server calculates the log event similarity between any two log event clusters and determines whether the calculated log event similarity is greater than or equal to a preset similarity threshold. If the log event similarity is greater than or equal to the preset similarity threshold, the corresponding two log event clusters are merged, and log events are extracted based on the merged log event cluster to obtain the log events in the merged log event cluster. If the log event similarity is less than the preset similarity threshold, the corresponding two log event clusters are not merged.

[0121] As can be seen, the log anomaly detection method in this embodiment determines the log event cluster to which the log belongs by using the word pair information contained in the log, and extracts the log events in the log event cluster to obtain the log events of the log. It can be seen that abstracting the log content of the log into the log event extraction method improves the intelligence of log anomaly detection.

[0122] As an example, based on the above embodiments, the window determined by the sliding window can be used as a sub-sequence; for details, please refer to [reference needed]. Figure 12 , Figure 12 This is a flowchart illustrating an anomaly detection method for logs, as shown in another exemplary embodiment of this application. Based on the above embodiment, the process of dividing the log sequence to be detected into multiple sub-sequences in step S210 further includes at least steps S1210 to S1220, which are detailed below:

[0123] Step S1210: The log sequence to be detected is divided into multiple windows by using a sliding window with specified sliding rules.

[0124] A sliding window is a pre-defined window that can slide according to specified sliding rules.

[0125] Specifying a sliding rule refers to setting a rule for sliding at preset intervals, where the preset interval can be set manually. It should be noted that the specified sliding rule can be sliding at the same interval or sliding with a gradually increasing interval; this embodiment does not limit this.

[0126] The server divides the log sequence into multiple windows by sliding a sliding window across it. In other words, each slide of the sliding window creates one new window.

[0127] Step S1220: The logs contained in multiple windows are respectively treated as multiple subsequences.

[0128] The server treats the logs contained in each window as a subsequence. It should be noted that the resulting windows may or may not overlap. If they overlap, it means that the two windows containing the overlapping portion contain some of the same logs. If they do not overlap, they may or may not contain the same logs.

[0129] As can be seen, the log anomaly detection method in this embodiment determines the subsequence through a sliding window, which facilitates the quick location of the window corresponding to the abnormal subsequence when determining the abnormal subsequence, and then performs log anomaly detection on the logs contained in the abnormal window, thereby improving the detection efficiency of log anomalies.

[0130] To elaborate on the anomaly detection method for logs in this application, Figure 13 The illustrated diagram further illustrates the anomaly detection method for the logs in this application, as detailed below:

[0131] The server acquires a log sequence to be inspected, which contains multiple logs. It then parses the logs to obtain information such as time, thread, session ID, process ID, log level, category label, and log content for each log. Next, based on the parsed log content, it performs log event statistics on the logs in the subsequences obtained from dividing the log sequence, generating a count vector for each subsequence. Log events can include Event1, Event2, Event3, Event4, and Event5 as shown in the diagram. The server then groups the logs in the sequence according to the parsed category labels, resulting in multiple log groups, such as Log Group 1, Log Group 2, and Log Group 3 as shown in the diagram. For each log group, it projects the count vectors corresponding to the logs in that group to obtain a projection vector. The server then determines abnormal subsequences based on the spatial distance between the count vectors and the projection vectors of the logs in the group. Finally, it inspects each log in the abnormal subsequences.

[0132] As an example, based on the above explanation, for methods of extracting log events, please refer to [link / reference]. Figure 14 , Figure 14 The diagram shows the effect of log event extraction. Figure 14The example shows a subsequence containing 5 log entries, namely log1, log2, log3, log4, log5… The server performs word pair parsing on log1, log2, log3, log4, and log5 to obtain the corresponding word pair information. Based on the word pair information, it determines the log event cluster to which log1, log2, log3, log4, and log5 belong, and then extracts the log events from the log event cluster to obtain the log events of the log. For example, if log3 is “genByteDataCintextrequestticket=null,execcreate Transfer Ticketagain”, the server, using the above extraction method, determines that the log event Event3 in the log event cluster to which log3 belongs is “genByteDataCintextrequestticket=*,execcreateTransfer Ticketagain”. For example, if log4 is “Greate Transfer Ticketticket=3 / 41DDF7959265054C7D34 / 1547532.”, the server uses the extraction method described above to determine that the log event Event4 in the log event cluster to which log4 belongs is “Greate Transfer Ticketticket=”.

[0133] See here for further reading. Figure 15 , Figure 15 The diagram illustrates a flowchart of the log anomaly detection method of this application. Specifically, the method includes: a log acquisition module (where the server acquires the log sequence to be detected); a log parsing module (where the server parses the logs in the acquired log sequence to obtain the time, thread, session ID, process ID, log level, category label, and log content contained in each log); a log event extraction module (where the server performs log event statistics on multiple logs contained in each subsequence based on the parsed log content, generating a count vector corresponding to each subsequence); a log grouping module (where the server groups the logs in the log sequence based on the parsed category labels, obtaining multiple log groups); and an anomaly log detection module (where the server determines the subsequences containing anomalies based on the count vectors and projection vectors corresponding to the logs in the log groups, and then detects the logs in the anomaly subsequences one by one to obtain the anomaly logs).

[0134] Figure 16 This is a block diagram illustrating an anomaly detection device for logs, as shown in an exemplary embodiment of this application. This log anomaly detection device can be applied to... Figure 1 The implementation environment shown is different from that described above. This log anomaly detection device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0135] like Figure 16 As shown, the exemplary log anomaly detection device 1600 includes: a segmentation module 1610, a counting vector generation module 1620, an anomaly detection module 1630, and a selection module 1640. Specifically:

[0136] The segmentation module 1610 is configured to divide the log sequence to be detected into multiple subsequences, each subsequence including multiple logs.

[0137] The counting vector generation module 1620 is configured to perform log event statistics on multiple logs contained in each subsequence and generate a counting vector corresponding to each subsequence.

[0138] The anomaly detection module 1630 is configured to perform anomaly detection on each subsequence based on the counting vector corresponding to each subsequence, and obtain the subsequences with anomalies.

[0139] Select module 1640 and configure it to select logs containing exceptions from the subsequences of exceptions.

[0140] In this exemplary log anomaly detection device, compared to performing anomaly detection on all logs in the log sequence one by one, by performing log event statistics on the logs contained in each subsequence, generating a count vector corresponding to each subsequence, and performing anomaly detection on each subsequence based on the count vector, identifying the subsequences with anomalies, and then detecting abnormal logs in the subsequences with anomalies, the number of logs to be detected can be reduced and the log detection efficiency can be improved.

[0141] Based on the above exemplary embodiments, the anomaly detection module 1630 includes: a projection module and an anomaly detection submodule. Specifically:

[0142] The projection module is configured to project the counting vector corresponding to each subsequence to obtain the projection vector corresponding to each subsequence.

[0143] The anomaly detection submodule is configured to perform anomaly detection on each subsequence based on the relationship between the counting vector and the projection vector corresponding to each subsequence, and to obtain the subsequences with anomalies.

[0144] In this exemplary log anomaly detection device, the detection of abnormal subsequences is improved by using the relationship between the counting vector corresponding to each subsequence and the projection vector corresponding to the counting vector.

[0145] Based on the above exemplary embodiments, the projection module includes: a grouping module, a projection submodule, and a projection vector determination module. Specifically:

[0146] The grouping module is configured to group logs within a log sequence based on category labels, resulting in multiple log groups. Logs within the same log group have the same category label.

[0147] The projection submodule is configured to project the counting vectors corresponding to the logs in each log group, thereby obtaining the projection vectors corresponding to the logs in the log group.

[0148] The projection vector determination module is configured to determine the projection vector corresponding to each subsequence based on the projection vectors corresponding to the logs contained in multiple log groups.

[0149] In this exemplary log anomaly detection device, logs in the log sequence are grouped according to category labels to obtain multiple log groups. The counting vectors corresponding to the logs in each log group are projected to obtain the corresponding projection vectors. Then, based on the projection vectors corresponding to the logs in the multiple log groups, the projection vector corresponding to each subsequence is determined. In this way, the logs in each log group are in the same dimension by grouping, reducing the interference of logs in other dimensions on the projection processing.

[0150] Based on the above exemplary embodiments, the projection vector determination module includes: an acquisition module and a projection vector averaging module. Specifically:

[0151] The acquisition module is configured to acquire the projection vectors corresponding to logs belonging to the same subsequence.

[0152] The projection vector averaging module is configured to perform an average operation on the projection vectors corresponding to logs belonging to the same subsequence, so as to obtain the projection vector corresponding to each subsequence.

[0153] In this exemplary log anomaly detection device, the projection vector corresponding to each subsequence is obtained by averaging the projection vectors corresponding to logs belonging to the same subsequence in each log group. This improves the accuracy of obtaining the projection vector corresponding to each subsequence based on the averaging of the projection vectors corresponding to different logs belonging to the same subsequence.

[0154] Based on the above exemplary embodiments, the anomaly detection submodule includes: a spatial distance value calculation module, a spatial distance value acquisition module, a spatial distance value averaging module, and an anomaly subsequence determination module. Specifically:

[0155] The spatial distance calculation module is configured to calculate the spatial distance between the counting vector and the projection vector corresponding to the logs in each log group, and obtain the spatial distance value corresponding to the logs in each log group.

[0156] The spatial distance value acquisition module is configured to acquire the spatial distance values ​​corresponding to logs belonging to the same subsequence.

[0157] The spatial distance averaging module is configured to calculate the average spatial distance values ​​of logs belonging to the same subsequence, thereby obtaining the average spatial distance between the counting vector and the projection vector for each subsequence.

[0158] The abnormal subsequence determination module is configured to perform anomaly detection on each subsequence based on the mean spatial distance between the counting vector and the projection vector corresponding to each subsequence, and obtain the subsequences with anomalies.

[0159] In this exemplary log anomaly detection device, the average spatial distance values ​​of logs belonging to the same subsequence in each log group are averaged to obtain the mean spatial distance between the counting vector and the projection vector of each subsequence. Anomaly subsequences are detected based on the mean spatial distance between the counting vector and the projection vector of each subsequence to obtain the subsequences with anomalies. This improves the accuracy of obtaining the spatial distance value of each subsequence by processing based on the mean spatial distance between the counting vector and the projection vector of each subsequence.

[0160] Based on the above exemplary embodiments, the anomaly detection submodule includes: a calculation module and a first judgment module. Specifically:

[0161] The calculation module is configured to calculate the spatial distance between the counting vector and the projection vector corresponding to each subsequence, and obtain the spatial distance value corresponding to each subsequence.

[0162] The first judgment module is configured to obtain a detection result that characterizes the existence of anomalies in the subsequence if the spatial distance value corresponding to the subsequence is greater than a preset spatial distance threshold.

[0163] In this exemplary log anomaly detection device, the spatial distance between the counting vector and the projection vector corresponding to each subsequence is calculated to obtain the spatial distance value corresponding to each subsequence. If the spatial distance value corresponding to the subsequence is greater than a preset spatial distance threshold, a detection result is obtained to characterize the anomaly of the subsequence, which can improve the accuracy of anomaly subsequence detection.

[0164] Based on the above exemplary embodiments, the counting vector generation module 1620 includes: a parsing module, a matrix generation module, and a counting vector generation submodule. Specifically:

[0165] The parsing module is configured to parse the category of log events corresponding to the logs contained in each subsequence, and count the number of log events belonging to the same category to obtain the statistical count corresponding to the subsequence.

[0166] The matrix generation module is configured to generate a matrix for each subsequence based on the statistical counts corresponding to each subsequence.

[0167] The counting vector generation submodule is configured to generate a counting vector for each subsequence based on the matrix corresponding to each subsequence.

[0168] In this exemplary log anomaly detection device, the number of log events belonging to the same category in a subsequence is counted to obtain the statistical quantity corresponding to the subsequence, and a matrix corresponding to each subsequence is generated based on the statistical quantity corresponding to the subsequence. Then, a counting vector corresponding to each subsequence is generated based on the matrix corresponding to each subsequence, thereby improving the accuracy of obtaining the counting vector corresponding to each subsequence.

[0169] Based on the above exemplary embodiments, the partitioning module 1610 includes: a window determination module and a control module. Specifically:

[0170] The window determination module is configured to divide the log sequence to be detected into multiple windows by using a sliding window with specified sliding rules.

[0171] As a module, it is configured to treat logs contained in multiple windows as multiple subsequences.

[0172] In this exemplary log anomaly detection device, a sliding window is used to determine the subsequence, which facilitates the quick location of the window corresponding to the abnormal subsequence when determining the abnormal subsequence, and then performs log anomaly detection on the logs contained in the abnormal window, thereby improving the detection efficiency of log anomalies.

[0173] Based on the above exemplary embodiments, prior to the counting vector generation module 1620, this device further includes: a word pair information determination module, a log event cluster determination module, and a log event determination module. Specifically:

[0174] The word pair information determination module is configured to parse word pairs of logs contained in the log sequence to obtain the word pair information corresponding to the logs.

[0175] The log event cluster determination module is configured to determine the log event cluster to which a log belongs based on the word pair information corresponding to the log.

[0176] The log event determination module is configured to extract log events from log event clusters to obtain the log events of the logs.

[0177] In this exemplary log anomaly detection device, the log event cluster to which the log belongs is determined by the word pairs contained in the log, and the log events in the log event cluster are extracted to obtain the log events of the log. It can be seen that the method of abstracting the log content of the log into the extraction of log events improves the intelligence of log anomaly detection.

[0178] It should be noted that the log anomaly detection device and the log anomaly detection method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the log anomaly detection device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0179] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the log anomaly detection method provided in the above embodiments.

[0180] Figure 17 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 17 The computer system 1700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0181] like Figure 17 As shown, the computer system 1700 includes a Central Processing Unit (CPU) 1701, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1702 or programs loaded from storage portion 1708 into Random Access Memory (RAM) 1703, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1703. The CPU 1701, ROM 1702, and RAM 1703 are interconnected via bus 1704. An Input / Output (I / O) interface 1705 is also connected to bus 1704.

[0182] The following components are connected to I / O interface 1705: an input section 1706 including a keyboard, mouse, etc.; an output section 1707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1708 including a hard disk, etc.; and a communication section 1709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 1709 performs communication processing via a network such as the Internet. Drive 1710 is also connected to I / O interface 1705 as needed. Removable media 1711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1710 as needed so that computer programs read from them can be installed into storage section 1708 as needed.

[0183] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1709, and / or installed from removable medium 1711. When the computer program is executed by central processing module (CPU) 1701, it performs various functions defined in the system of this application.

[0184] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0186] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0187] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the log anomaly detection method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0188] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the log anomaly detection method provided in the various embodiments described above.

[0189] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for anomaly detection in logs, characterized in that, The method includes: The log sequence to be detected is divided into multiple subsequences, each subsequence includes multiple logs, and the logs contained in the log sequence have category labels; Log event statistics are performed on multiple logs contained in each subsequence to generate a count vector corresponding to each subsequence; The logs in the log sequence are grouped according to the category labels to obtain multiple log groups; wherein the logs in the same log group have the same category labels. For each log group, the counting vectors corresponding to the logs contained in the log group are projected to obtain the projection vectors corresponding to the logs contained in the log group. Based on the projection vectors corresponding to the logs contained in the multiple log groups, determine the projection vector corresponding to each subsequence; Based on the relationship between the counting vector and the projection vector corresponding to each subsequence, anomaly detection is performed on each subsequence to obtain the subsequences with anomalies; Select logs containing anomalies from the subsequence of anomalies.

2. The method according to claim 1, characterized in that, The step of determining the projection vector corresponding to each subsequence based on the projection vectors corresponding to the logs contained in the plurality of log groups includes: Obtain the projection vectors corresponding to logs belonging to the same subsequence; The projection vectors corresponding to logs belonging to the same subsequence are averaged to obtain the projection vector corresponding to each subsequence.

3. The method according to claim 1, characterized in that, The step of performing anomaly detection on each subsequence based on the relationship between the counting vector and the projection vector corresponding to each subsequence to obtain the subsequences with anomalies includes: The spatial distance between the counting vector and the projection vector corresponding to the logs in each log group is calculated to obtain the spatial distance value corresponding to the logs in each log group. Get the spatial distance value corresponding to logs belonging to the same subsequence; The average spatial distance values ​​of logs belonging to the same subsequence are calculated to obtain the mean spatial distance between the counting vector and the projection vector corresponding to each subsequence. Based on the average spatial distance between the counting vector and the projection vector corresponding to each subsequence, anomaly detection is performed on each subsequence to obtain the subsequences with anomalies.

4. The method according to claim 1, characterized in that, The step of performing anomaly detection on each subsequence based on the relationship between the counting vector and the projection vector corresponding to each subsequence to obtain the subsequences with anomalies includes: The spatial distance between the counting vector and the projection vector corresponding to each subsequence is calculated to obtain the spatial distance value corresponding to each subsequence; If the spatial distance value corresponding to the subsequence is greater than the preset spatial distance threshold, a detection result is obtained to characterize that the subsequence has an anomaly.

5. The method according to claim 1, characterized in that, The step of performing log event statistics on multiple logs contained in each subsequence to generate a count vector corresponding to each subsequence includes: For each subsequence, the category of the log events corresponding to the logs contained in the subsequence is parsed, and the number of log events belonging to the same category is counted to obtain the statistical count corresponding to the subsequence. Generate a matrix corresponding to each subsequence based on the statistical counts corresponding to each subsequence; Generate a counting vector for each subsequence based on the matrix corresponding to each subsequence.

6. The method according to any one of claims 1 to 5, characterized in that, The log sequence to be detected is divided into multiple sub-sequences, including: The log sequence to be detected is divided into multiple windows by using a sliding window with specified sliding rules. The logs contained in the multiple windows are respectively regarded as the multiple sub-sequences.

7. The method according to any one of claims 1 to 5, characterized in that, Before performing log event statistics on the multiple logs contained in each subsequence and generating a count vector corresponding to each subsequence, the method further includes: The logs contained in the log sequence are parsed word pairs to obtain the word pair information corresponding to the logs; The log event cluster to which the log belongs is determined based on the word pair information corresponding to the log; Extract the log events from the log event cluster to obtain the log events of the log.

8. An anomaly detection device for logs, characterized in that, The device includes: The segmentation module is configured to divide the log sequence to be detected into multiple subsequences, each subsequence including multiple logs, and the logs contained in the log sequence have category labels; The counting vector generation module is configured to perform log event statistics on multiple logs contained in each subsequence and generate a counting vector corresponding to each subsequence; The anomaly detection module is configured to group the logs contained in the log sequence according to the category label to obtain multiple log groups; wherein the logs within the same log group have the same category label; for each log group, the counting vector corresponding to the logs contained in the log group is projected to obtain the projection vector corresponding to the logs contained in the log group; based on the projection vectors corresponding to the logs contained in the multiple log groups, the projection vector corresponding to each subsequence is determined; based on the relationship between the counting vector and the projection vector corresponding to each subsequence, anomaly detection is performed on each subsequence to obtain the subsequences containing anomalies; The selection module is configured to select logs containing anomalies from the subsequences of the anomalies.

9. An electronic device, characterized in that, include: Memory, which stores computer-readable instructions; A processor reads computer-readable instructions stored in memory to perform the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the processor of a computer, cause the computer to perform the method described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images