Method, device, computer equipment and storage medium for determining abnormal logs

Abstract

The present application discloses a method, device, computer equipment and storage medium for determining abnormal logs, the method comprising: obtaining log data to be analyzed, the log data comprising multiple logs; clustering the multiple logs in the log data according to the data composition characteristics of the data in the logs, obtaining multiple clustered log sets and the data composition characteristics corresponding to each log set, the log set comprising at least one log; for each log set, determining at least one abnormality detection rule matching the data composition characteristics corresponding to the log set; for each log set, using at least one abnormality detection rule corresponding to the log set, determining abnormal logs with abnormalities in the log set and abnormal data in the abnormal logs. The scheme of the present application can reduce the complexity of determining abnormal logs from a large amount of log data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, computer equipment and storage medium for determining abnormal logs. Background Art

[0002] During the operation of a software product (such as an application, or a system platform containing multiple applications, etc.), logs representing its operation status are recorded. Problems with the software product can be discovered in a timely manner by analyzing the logs of the software product.

[0003] Among them, software products generally contain program modules that implement different functional modes. Due to the differences between different program modules, the log forms generated by different program modules will be different. Therefore, the log data generated by the software product is not only large in volume, but also relatively complex and diverse. At the same time, since the abnormal manifestations of different program modules in the software product are different, the abnormal analysis methods for the log data of different program modules will also be different. Based on this, in order to locate the abnormal logs that characterize the abnormality of the software product from the log data of the software product, it is necessary to manually analyze each log in the log data one by one, resulting in a high complexity in determining the abnormal logs. Summary of the invention

[0004] In view of this, the present application provides a method, apparatus, computer device and storage medium for determining abnormal logs, so as to reduce the complexity of determining abnormal logs from a large amount of log data.

[0005] To achieve the above objectives, on the one hand, the present application provides a method for determining abnormal logs, including:

[0006] Obtaining log data to be analyzed, wherein the log data includes multiple logs;

[0007] Clustering multiple logs in the log data according to data composition features of the data in the logs to obtain multiple clustered log sets and data composition features corresponding to each of the log sets, wherein the log set includes at least one log;

[0008] For each of the log sets, determining at least one anomaly detection rule that matches the data composition feature corresponding to the log set;

[0009] For each of the log sets, at least one anomaly detection rule corresponding to the log set is used to determine an abnormal log in the log set and abnormal data in the abnormal log.

[0010] In a possible scenario, clustering the multiple logs in the log data according to the data composition features of the data in the logs includes:

[0011] Clustering is performed on multiple logs in the log data according to function identifiers and character features contained in the logs, wherein the function identifier in the log is used to characterize the type of function mode to which the running status information recorded in the log belongs.

[0012] In another possible scenario, it also includes:

[0013] In the case where there are abnormal logs in the log set, determining at least one target data item to which the abnormal data in each abnormal log in the log set belongs, wherein the target data item belongs to at least one data item in each log in the log set;

[0014] For each target data item corresponding to the log set, construct an abnormal identification curve graph of the target data item, wherein the abnormal identification curve graph includes a value change curve graph corresponding to the value of the target data item in each log of the log set, and information of abnormal data in the target data item is marked in the value change curve graph;

[0015] An abnormal identification curve graph of each target data item in the log set is displayed.

[0016] In another possible scenario, it also includes:

[0017] For each log set, determine a target log sample set that matches the data composition feature corresponding to the log set from multiple log sample sets corresponding to the stored log sample data, wherein the log sample data includes multiple log samples without abnormal data, and the multiple log sample sets are obtained by clustering multiple log samples of the log sample data based on the data composition feature of the data in the log samples;

[0018] For each log set, determine a first log occurrence frequency of the log set in the log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and in a case where it is determined based on the first log occurrence frequency and the second log occurrence frequency that the number of logs in the log set has an abnormal increase in number, determine the log set as a risk log set with an abnormal risk, wherein the first log occurrence frequency is a ratio of a first number of logs included in the log set to a first total number of logs included in the log data, and the second log occurrence frequency is a ratio of a second number of logs included in the target log sample set corresponding to the log set to a second total number of logs included in the log sample data;

[0019] Output prompt information for the risk log set.

[0020] In another possible scenario, before determining, for each of the log sets, at least one anomaly detection rule that matches the data composition feature corresponding to the log set, the method further includes:

[0021] Determine a set detection mode, where the detection mode is one of a manual detection mode and an automatic detection mode;

[0022] The step of determining, for each of the log sets, at least one anomaly detection rule that matches the data composition feature corresponding to the log set includes:

[0023] When the set detection mode is the automatic detection mode, for each of the log sets, determining at least one anomaly detection rule that matches the data composition feature corresponding to the log set;

[0024] The method further comprises:

[0025] When the set detection mode is the manual detection mode, a log display interface is displayed, wherein the log display interface displays a plurality of logs in the log data;

[0026] In response to the log presentation interface detecting a log selection operation, determining a target log selected by the log selection operation;

[0027] Determine the target log set to which the target log belongs;

[0028] Display the data of each log in the target log set.

[0029] In another aspect, the present application further provides a device for determining an abnormal log, comprising:

[0030] A log acquisition unit, used to acquire log data to be analyzed, wherein the log data includes a plurality of logs;

[0031] A log clustering unit, configured to cluster multiple logs in the log data according to data composition features of the data in the logs, to obtain multiple clustered log sets and data composition features corresponding to each of the log sets, wherein the log set includes at least one log;

[0032] A rule matching unit, configured to determine, for each of the log sets, at least one anomaly detection rule that matches a data composition feature corresponding to the log set;

[0033] The abnormality determination unit is used to determine, for each of the log sets, abnormal logs with abnormalities in the log set and abnormal data in the abnormal logs by using at least one abnormality detection rule corresponding to the log set.

[0034] In another aspect, the present application further provides a computer device, the computer device comprising: a processor and a memory;

[0035] The processor is used to call and execute the program stored in the memory;

[0036] The memory is used to store the program, and the program is at least used to implement any one of the methods for determining an abnormal log.

[0037] On the other hand, the present application further provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the method for determining an abnormal log as described in any of the above items is implemented.

[0038] From the above content, it can be seen that in the present application, multiple logs in the log data to be analyzed are clustered according to the data composition characteristics of the logs. Since the data composition characteristics of the data in the logs can reflect the operating status of the function of the software product for which the logs are generated, the logs recording the operating status of the same function can be clustered into the same log set based on the data composition characteristics; moreover, the data composition characteristics corresponding to the log set can reflect the functions targeted by the logs in the log set, therefore, the anomaly detection rules corresponding to the corresponding functions (i.e., the anomaly detection rules matched by the data composition characteristics of the log set) can be determined as the anomaly detection rules applicable to the logs in the log set, thereby determining the anomaly detection rules applicable to the logs in the log set, and then directly using the corresponding anomaly detection rules to perform anomaly detection on the logs in the log set, thereby avoiding the complexity caused by manually analyzing the log anomalies one by one, and thus reducing the complexity of determining abnormal logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0040] Figure 1 A schematic diagram of a composition architecture of a scenario to which a method for determining abnormal logs of the present application is applicable is shown;

[0041] Figure 2A schematic diagram of a process for determining abnormal logs provided by the present application is shown;

[0042] Figure 3 A schematic diagram of a process of outputting an abnormal identification curve graph for a log set with abnormal logs in the present application is shown;

[0043] Figure 4 A schematic diagram showing a numerical variation curve diagram of a data item in the present application;

[0044] Figure 5 A schematic diagram showing an abnormal identification curve diagram of a data item in the present application;

[0045] Figure 6 A schematic diagram of an abnormality display interface including an abnormality report and an abnormality identification curve graph in the present application is shown;

[0046] Figure 7 Another flow chart of a method for determining abnormal logs of the present application is shown;

[0047] Figure 8 A schematic diagram of an abnormal summary report in this application is shown;

[0048] Fig. 9 Another flow chart of a method for determining abnormal logs of the present application is shown;

[0049] Fig.10 A schematic diagram of the structure of a device for determining abnormal logs of the present application is shown;

[0050] Fig.11 A schematic diagram of the composition architecture of a computer device in the present application is shown. DETAILED DESCRIPTION

[0051] The solution of the present application is suitable for determining abnormal logs from a large amount of log data to reduce the complexity of determining abnormal logs. The logs mentioned in the present application can be operation logs generated during the operation of software products such as applications. By performing abnormal analysis on the logs, problems such as poor performance and failures in the software products can be discovered in a timely manner.

[0052] The solution of the present application can be applied to a personal computer, a server, or a server system composed of multiple servers. In order to improve the efficiency of determining abnormal logs, the present application can also be applied to a cloud platform or other computing systems.

[0053] For ease of understanding, the application of the solution of this application to a cloud platform is used as an example. Figure 1 As shown, it shows a schematic diagram of the composition architecture of a scenario to which the present application is applicable.

[0054] Depend on Figure 1 It can be seen that the scenario includes: a cloud platform 10 , which may include multiple cloud servers 101 .

[0055] The scenario may also include multiple application clients 20, in which the application to be monitored may be run. Accordingly, during the application running process, the application client will generate logs related to the application running, such as the usage of memory and processor during the application running, and some data response of the application.

[0056] For example, assuming that it is necessary to analyze whether a game application is abnormal based on the log of the game application, then the application client can be a game client, and the game client will generate and store relevant logs during the game operation.

[0057] The application client 20 may send logs generated during the operation of the application to the cloud platform 10 .

[0058] Accordingly, the cloud platform can obtain the log data corresponding to different application clients, obtain multiple copies of log data, and perform anomaly analysis on each copy of log data in turn.

[0059] The cloud platform is also called cloud computing platform, which is a network platform built based on cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data calculation, storage, processing and sharing.

[0060] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and is flexible and convenient. The background services of the technical network system require a large amount of computing and storage resources, such as image storage and encoding. With the rapid development and application of the Internet industry, each item may have its own identification mark in the future, and all of them need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0061] Among them, cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0062] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0063] exist Figure 1 In the scenario, the cloud server on the cloud platform can complete the collection and analysis of log data and the identification of abnormal logs.

[0064] Understandably, Figure 1 The example above takes the case where the cloud platform directly obtains log data generated by the application client from the application client. In actual applications, the cloud platform can also obtain log data through other devices or through other network channels without any restrictions.

[0065] in addition, Figure 1 The example of obtaining the log data of the application client is used for explanation. In actual applications, if it is necessary to analyze a certain system or network platform, the cloud platform can also obtain log data related to the operation of the system or network platform.

[0066] It should be noted that Figure 1 For ease of understanding, this application is applied to a cloud platform as an example. When a single computer device, server, or server cluster is used to analyze log data, only Figure 1 The cloud platform in the scenario shown can be replaced with corresponding devices or server clusters, etc. The scenarios are similar and will not be repeated here.

[0067] The solution of this application is introduced below with reference to the flow chart.

[0068] like Figure 2 As shown, it shows a flow chart of a method for determining abnormal logs in the present application. The method of this embodiment can be applied to the aforementioned computer device, server or cloud platform, etc. The method of this embodiment may include:

[0069] S201, obtaining log data to be analyzed, where the log data includes multiple logs.

[0070] For example, the log data to be analyzed may be the operation log of a software product, and each log in the log data may record the operation status data of the software product during operation, etc. For example, the operation log of an application in a client, or the operation log of a system or network platform, etc.

[0071] It is understandable that different logs may be generated at different times for applications or software systems, and therefore, each log data actually contains multiple logs generated at different times. For example, the log data may be a log data text, which may contain multiple lines of logs, and the multiple lines of logs may be regarded as a sequence of logs generated at multiple times.

[0072] Optionally, in order to distinguish each log in the log data, each log may correspond to a log identifier, which can be used to uniquely identify a log in the log data. For example, the log identifier may be a log sequence number. For example, when the log data consists of multiple lines of logs, each line of log corresponds to a line number, which is the identifier of the log.

[0073] It should be noted that in actual applications, multiple log data from different device ends may be obtained at the same time, but the analysis process for each log data is the same, and the subsequent steps in this application can be used to determine abnormal logs.

[0074] S202 , clustering multiple logs in the log data according to the data composition features of the data in the logs, to obtain multiple clustered log sets and the data composition features corresponding to each log set.

[0075] The data composition feature of the log refers to the features of the data content contained in the log. For example, the data composition feature of the log can represent the number and type of data items contained in the log, as well as the specific form of the value corresponding to each data item, etc.

[0076] It is understandable that since software products generally have multiple different functional modes, such as software products include different functional modules, different functional modules implement different functional modes. For example, a game application may include data encoding and decoding functions and image rendering functions. The game application will obtain logs generated by each of these two functional modes during the process of executing these two functional modes respectively.

[0077] Correspondingly, the logs recorded by the software product may include logs of the operating conditions of different functional modes, and the data content and specific forms contained in the logs corresponding to different functional modes may be quite different. However, the data content and specific forms in the logs for the same functional mode are the same or almost the same (that is, the data composition characteristics of the functional mode are the same or similar). Therefore, it is possible to determine which logs in the log data are generated for the same functional mode based on the data composition characteristics. Based on this, the logs for the same functional mode can be clustered into a log set based on the data composition characteristics.

[0078] Each log set may include at least one log. It is understandable that logs with the same data composition characteristics will be clustered into the same log set. Therefore, the data composition characteristics generally applicable to each log in the log set are the data composition characteristics corresponding to the log set. Similarly, the data composition characteristics corresponding to the log set reflect the functional mode targeted by the logs in the log set.

[0079] It is understandable that, since the same log set contains logs with the same data composition characteristics, a log can only belong to a certain log set, and logs in different log sets do not overlap.

[0080] It is understandable that the data composition characteristics of the log can be reflected through multiple dimensions. As a possible scenario, the data composition characteristics of the log can be the function identifier and character features contained in the log. Among them, the function identifier in the log is used to characterize the type of functional mode to which the operating status information recorded in the log belongs. For example, the function identifier can be the name of the functional mode, etc. Character features may include: one or more of the features such as the number of real numbers contained in the log, the number of characters other than real numbers, and the specific content of characters other than real numbers. Accordingly, multiple logs in the log data can be clustered according to the function identifier and character features contained in the log.

[0081] For example, taking the log data of a game application as an example, the game application involves the rendering function mode, so the log of the rendering function mode will record: the file name representing the rendering mode, the resolution, and the transmission rate of the rendering data. The corresponding log can be: **.render; resolution: 1920*180; transmission rate: 25bps, then the function identifier contained in the log is "**.render", and it contains three real numbers, namely 1980, 180 and 25, and the characters other than the real numbers include at least "resolution" and "transmission rate". Correspondingly, if other logs also have the function identifier "**.render", have the two strings of "resolution" and "transmission rate", and have three real numbers, then it can be considered that these logs are also logs generated for the rendering mode, and thus these logs are clustered into a log set.

[0082] S203: For each log set, determine at least one anomaly detection rule that matches the data composition feature corresponding to the log set.

[0083] Among them, the anomaly detection rule that matches the data composition feature corresponding to the log set is actually the anomaly detection rule applicable to the functional mode corresponding to the log in the log set.

[0084] For example, at least one anomaly detection rule may be pre-configured for each data composition feature. Accordingly, according to the correspondence between the data composition feature and the anomaly detection rule, only one anomaly detection rule suitable for anomaly detection of each log in the log set may be determined.

[0085] For another example, at least one anomaly detection rule applicable to each of the different functional modes may be pre-configured. Accordingly, for each log set, the functional mode corresponding to the logs in the log set may be determined based on the data composition features corresponding to the log set, and then at least one anomaly detection rule suitable for the functional mode may be searched.

[0086] Among them, the anomaly detection rules may include the conditions that different data items in the log need to meet. Among them, the data items in the log can also be called parameter items, and different data items represent a status indicator of the functional mode recorded in the log. For example, the data items in the log may include: CPU usage, memory occupancy, etc. Correspondingly, the anomaly detection rules may include, CPU usage is greater than the set CPU usage, memory occupancy is greater than the set occupancy, etc.

[0087] It is understandable that the anomaly detection rules can be set according to different software products and in combination with actual needs, and there is no limitation on this.

[0088] S204: for each log set, using at least one anomaly detection rule corresponding to the log set, determine an abnormal log with abnormalities in the log set and abnormal data in the abnormal log.

[0089] It is understandable that if a log in a log set meets the abnormal condition in the abnormality detection rule, it means that the log is an abnormal log. The log being an abnormal log means that the value of at least one data item in the log is abnormal. Therefore, based on the abnormality detection rule, the abnormal data in the abnormal log can be located. For example, the abnormal data can be the data item with abnormality in the abnormal log and the value of the data item with abnormality.

[0090] In one possible case, in order to facilitate cached data and anomaly detection of data, for each log set, at least one data item in the logs in the log set can be extracted, and each extracted data item can be used as a field in the data table to be constructed, and a data table can be constructed based on the respective values ​​of the at least one data item in each log in the log set.

[0091] The data items of each log in the log set are the same, so all the data items of any log in the log set can be extracted. For example, if each log in the log set contains two data items, memory usage and CPU usage, these two data items will be extracted as two fields of the data table.

[0092] Each record in the data table constructed based on the log set corresponds to a log in the log set, and different logs correspond to different records in the data table. For example, if a log in the log set has a unique row number in the log data, then the record corresponding to the log can be identified in the data table by the row number corresponding to the log.

[0093] For example, suppose the log set contains two logs with log line numbers 11 and 13, and the fields extracted from the logs in the log set are CPU usage and memory usage, then the data table constructed based on the log set can have CPU usage and memory usage, and a record in the data table is a log with log line number 11, and the record contains the values ​​corresponding to the CPU usage and the values ​​corresponding to the memory usage. Correspondingly, there is also a record with log line number 13 in the data table.

[0094] After caching the data of each log in the log set through the data table, after determining at least one anomaly detection rule applicable to the log set, anomaly detection can be performed on each record in the data table based on the at least one anomaly detection rule to detect abnormal records with abnormal values ​​of fields. Accordingly, the log corresponding to the abnormal record is the abnormal log, and the data of the abnormal field in the abnormal record is the abnormal data in the abnormal log.

[0095] It can be seen that in the embodiment of the present application, multiple logs in the log data to be analyzed can be clustered according to the data composition characteristics of the logs. Since the data composition characteristics of the data in the logs can reflect the operating status of the function of the software product for which the logs are generated, the logs recording the operating status of the same function can be clustered into the same log set based on the data composition characteristics; and, the data composition characteristics corresponding to the log set can reflect the functions targeted by the logs in the log set, so the anomaly detection rules corresponding to the corresponding functions (i.e., the anomaly detection rules matched by the data composition characteristics of the log set) can be determined as the anomaly detection rules applicable to the logs in the log set, thereby making it possible to determine the anomaly detection rules applicable to the logs in the log set, and then the corresponding anomaly detection rules can be directly used to perform anomaly detection on the logs in the log set, avoiding the complexity caused by manually analyzing the log anomalies one by one, thereby reducing the complexity of determining abnormal logs.

[0096] It is understandable that after determining the abnormal logs in the log set in step S204, in order to facilitate the user to intuitively understand which data items in which logs have abnormal values, the present application can also output a curve chart corresponding to the data items in the abnormal logs for the log set. Figure 3 , which shows a schematic diagram of a process of outputting an abnormal identification curve graph for a log set with abnormal logs in the present application. The process of this embodiment may include:

[0097] S301 : for each log set, if there is an abnormal log in the log set, determine at least one target data item to which abnormal data in each abnormal log in the log set belongs.

[0098] Among them, for any abnormal log, the abnormal log may have at least one data item with abnormal data, and the data item with data anomaly and its value are abnormal data, so each abnormal data corresponds to a data item. For the convenience of distinction, the data item with data anomaly in the abnormal log is called the target data item.

[0099] For example, the abnormal logs in the log set include log 1 and log 2. The data of data item a1 in log 1 is abnormal, and the data of data item a2 in log 2 is abnormal. Then for log 1, the target data item to which its abnormal data belongs is data item a1; and the target data item to which the abnormal data in log 2 belongs is data item a2.

[0100] Among them, there can be one or more abnormal logs in the log set. When there are multiple abnormal logs in the log set, the types and numbers of target data items with data anomalies in different abnormal logs are different. This application will count all target data items with data anomalies to obtain at least one target data item. As in the example above, although there is a data anomaly in data item a1 in log 1, data item a1 belongs to the target data item with anomalies; although there is no data anomaly in data item a2 in log 1, there is an anomaly in the data of data item a2 in log 2, therefore, data item a2 also belongs to the target data item with anomalies.

[0101] S302: For each target data item corresponding to the log set, construct an abnormal identification curve graph of the target data item.

[0102] The abnormal identification curve graph of the target data item includes a numerical value change curve graph of the numerical value of the target data item in each log of the log set, and information of abnormal data in the target data item is marked in the numerical value change curve graph.

[0103] It can be understood that each target data item corresponds to a numerical change curve graph, and the numerical change curve graph of the target data item is constructed based on the numerical value of the target data item in each log in the log set. The numerical change curve graph can present the numerical value of the target data item in each log in the log set.

[0104] For example, the coordinate system in the numerical change curve graph of the target data item may include two mutually perpendicular coordinate axes, where each value in one coordinate axis identifies a log in a log set, and different values ​​in the other coordinate axis represent the numerical values ​​of the target data item. Accordingly, the values ​​of the target data item of each log in the log set may be marked in the coordinate system in turn, thereby obtaining a numerical change curve graph reflecting the numerical changes of the target data item of each log in the log set.

[0105] For easier understanding, see Figure 4 , Figure 4 A schematic diagram showing a numerical variation curve of a data item. Figure 4 In the example, the data item contained in each log of the log set is the CPU usage, and Figure 4 A numerical change curve graph of CPU usage is shown in FIG.

[0106] exist Figure 4 The horizontal axis in the middle represents the identification number of each log, such as the line number of each log contained in the log set in the log data. Therefore, Figure 4 The horizontal coordinate of each point in the numerical change curve corresponds to a log line number. Figure 4 , the log set at least includes the log with line number 12, the log with line number 20, the log with line number 25, and so on. Of course, the logs with other line numbers included in the log set cannot be listed one by one.

[0107] at the same time, Figure 4 The vertical axis indicates the specific value of CPU usage. Correspondingly, according to the CPU usage recorded in each log in the log set, the corresponding value can be marked on the vertical axis at the corresponding position of the log in the coordinate system. For example, if the CPU usage in the log with line number 12 is 40%, then the CPU usage at the position corresponding to the horizontal axis 20 in the value change curve is 60%. Figure 4 The numerical change curve is shown at the point (20, 60) in the graph.

[0108] It is understandable that, in order to enable the user to intuitively see which logs have abnormal values ​​of the target data item based on the value change curve of the target data item, the present application can mark the information of abnormal data in the target data item in the value change curve of the target data item, thereby obtaining an abnormal identification curve. For example, the coordinate point where the abnormality exists is marked in the value change curve of the target data item, or the horizontal coordinate of the log where the target data item has abnormal data.

[0109] For example, see Figure 5 , which shows a schematic diagram of an abnormal identification curve diagram of the present application.

[0110] Figure 5 is Figure 5 The abnormal identification curve is obtained based on the numerical change curve of CPU usage. Figure 4 and Figure 5 It can be seen that in Figure 5 For logs with abnormal CPU usage, the coordinate position 501 corresponding to the abnormal log will be marked on the horizontal axis of the abnormal identification curve graph, such as Figure 5 In order to visually see the horizontal axis position of each abnormal log, Figure 5 In the example, only two logs with abnormal CPU usage are taken, namely, the logs with line numbers 20 and 1001. Correspondingly, the horizontal axes of the two abnormal logs are 20 and 1001 respectively.

[0111] It is understandable that in the embodiment of the present application, after determining the abnormal logs in the log set, the value change curve graphs of the target data items corresponding to the abnormal target data items in the log set can be respectively constructed. Alternatively, after clustering to obtain the log set, the value change curve graphs are respectively constructed for each data item involved in each log in the log set. Accordingly, after determining the abnormal logs in the log set, the value change curve graph of the target data item can be determined directly from the constructed value change curve graphs of each data item for each target data item with abnormal data.

[0112] There are multiple possible specific ways to construct a value change curve graph of each data item in any log set:

[0113] For example, in a possible implementation, the value of the data item in each log in the log set may be extracted, and based on the value of the data item in each log, a value change curve graph of the data item may be constructed.

[0114] For another example, in another possible implementation, when a data table corresponding to each log set is constructed respectively, a value change curve of the data item can be constructed directly based on the value of the corresponding field of the data item in each record in the data table. In this case, the column corresponding to each field in the data table is the value of each log in the data item, so the value change curve of the data item can be generated directly based on the value of each record in the column.

[0115] S303, displaying an abnormal identification curve graph of each target data item in the log set.

[0116] In a possible implementation, for each log set, after the abnormal log analysis of the log set is completed, an abnormal identification curve graph for each target data item in the log set can be output. In this case, for each log set, when displaying the abnormal identification curve graph of the target data item in the log of the log set, the data combination feature corresponding to the target log set can also be identified, so that the user can intuitively understand in which functional mode of the software product the abnormal identification curve graph of the target data item is, and understand which logs in the logs corresponding to the functional mode have abnormalities based on the abnormal identification curve graph.

[0117] In yet another possible implementation, the present application may uniformly output an abnormal identification curve graph of each target data item in each log set having abnormal logs after analyzing all log sets.

[0118] It can be understood that in actual applications, while outputting the exception identification curve graph of the target data item in the log set, the present application can also provide an exception report for the log set, which can indicate the existence of anomaly logs in the log set and specific information about the anomaly in the anomaly log.

[0119] For example, after analyzing each log set, an abnormality display interface may be outputted, in which an abnormality report of each log set is displayed, and an abnormality identification curve graph of each target data item of each log set is presented in the abnormality display interface.

[0120] like Figure 6 As shown, it shows a schematic diagram of the abnormal display interface in the present application.

[0121] Depend on Figure 6It can be seen that the exception report 601 containing information such as various specific exception causes for the abnormal logs in the log set can be displayed in the exception display interface. At the same time, the exception identification curve 602 of each target data item with an exception in the log set can be presented below the exception report, so that the user can intuitively understand the specific exception cause and determine the log with an exception based on the exception identification curve.

[0122] It is understandable that in actual applications, if under normal circumstances, the number of logs generated for a certain functional mode is generally small, then if the number of logs of this functional mode detected in the log data is large, it means that this functional mode may be abnormal. Therefore, in order to be able to promptly discover the abnormality of a certain functional mode, the present application can also analyze whether the log set belongs to a risk log set with abnormalities based on whether there is an abnormality in the frequency of logs appearing in the log set, and then promptly determine the functional mode with abnormal risks based on the risk log set.

[0123] Among them, the operation of detecting whether a log set is a risk log set can be two parallel ways of determining abnormal logs with the operation of performing abnormal log detection on the log set in the previous embodiment. Of course, it can also be based on the previous abnormal log detection on the log set to analyze whether the log set belongs to the risk log set.

[0124] For ease of understanding, the following is an example of a situation. Figure 7 , which shows another implementation process of a method for determining abnormal logs in the present application. The process of this embodiment may include:

[0125] S701, obtaining a set of log data to be analyzed, where the log data includes a plurality of logs.

[0126] S702 , clustering multiple logs in the log data according to the data composition features of the data in the logs, to obtain multiple clustered log sets and the data composition features corresponding to each log set.

[0127] S703: For each log set, use at least one data item of each log in the log set as a field in a data table to be constructed, and construct a data table based on respective values ​​of the at least one data item in each log in the log set.

[0128] S704, for each log set, determine at least one anomaly detection rule that matches the data composition feature corresponding to the log set, and based on the at least one anomaly detection rule, determine that there are abnormal records and abnormal data in the abnormal records in the data table of the log set.

[0129] Among them, since each record in the data table corresponding to the log set corresponds to a log in the log set, the log corresponding to the abnormal record is the abnormal log, and accordingly, the abnormal data in the abnormal record is determined as the abnormal data in the abnormal log corresponding to the abnormal record.

[0130] For details of the above steps S701 to S704, please refer to the relevant introduction of the previous embodiment, which will not be repeated here.

[0131] It should be noted that, in this application, after clustering log sets, a data table is generated for each log set, and the data of each log in each log set is cached through the data table. However, if the data table is not generated, and the anomaly detection rules are used to directly perform anomaly detection on each log in the log set, it is also applicable to this embodiment.

[0132] S705, output an abnormal display interface.

[0133] Among them, the exception display interface can display the exception report corresponding to each log set, and the exception report can prompt the reason and abnormal situation of the abnormal log in the log set; at the same time, the exception display interface can also present the exception identification curve of the target data item with abnormalities in the log set.

[0134] For example, a user can click on an exception log indicated as having an exception in an exception report, and then a corresponding exception curve identification graph of the target data item of the exception log in the log set where the exception log is located can be presented in the exception display interface, or an exception identification curve graph of each target data item in the log set where the exception log is located can be presented.

[0135] Of course, one case of an abnormal display interface is taken as an example here, and the other cases of the abnormal display interface mentioned above are also applicable to this embodiment.

[0136] S706 , for each log set, determine a target log sample set that matches the data composition feature corresponding to the log set from multiple log sample sets corresponding to the stored log sample data.

[0137] The log sample data is the log data obtained when there is no abnormal operation of the software product. Accordingly, the log sample data includes a plurality of log samples without abnormal data, and the plurality of log samples in the log sample data are clustered into a plurality of log sample sets.

[0138] For example, multiple log samples in the log sample data can be clustered in advance based on the data composition features of the data in the log sample to obtain multiple log sample sets and the data composition features corresponding to each log sample set. On this basis, for the clustered log sets, based on the data composition features corresponding to the log sets, the log sample sets corresponding to the same data composition features can be queried.

[0139] For the sake of distinction, the log sample set that matches the data composition features of the log set is called the target log sample set.

[0140] S707, for each log set, determine the first log occurrence frequency of the log set in the log data and the second log occurrence frequency of the target log sample set corresponding to the log set in the log sample data; if it is determined based on the first log occurrence frequency and the second log occurrence frequency that the number of logs in the log set has increased abnormally, determine the log set as a risk log set with abnormal risk.

[0141] The first log occurrence frequency is the ratio of the first number of logs included in the log set to the first total number of logs included in the log data. For example, if the log set contains 50 logs, and the log data obtained in step S701 contains 1000 logs, then the first log occurrence frequency corresponding to the log set is 1 / 20.

[0142] Correspondingly, the second log occurrence frequency is a ratio between a second number of logs included in the target log sample set corresponding to the log set and a second total number of logs included in the log sample data.

[0143] There are multiple ways to determine that the number of logs in the log set has increased abnormally based on the first log occurrence frequency and the second log occurrence frequency. For example, for a log set, if the ratio of the first log occurrence frequency to the second log occurrence frequency is greater than a set threshold, it is determined that the number of logs in the log set has increased abnormally. For another example, for a log set, if the difference between the first log occurrence frequency and the second log occurrence frequency is greater than a set value, it is determined that the number of logs in the log set has increased abnormally.

[0144] It can be understood that, for a log set, the frequency of occurrence of the second log of the target log sample set corresponding to the log set represents the frequency of occurrence of the functional mode corresponding to the log in the log set under normal circumstances. Therefore, if the frequency of occurrence of the first log corresponding to the log set is much greater than the frequency of occurrence of the second log, it means that more logs have appeared for the functional mode, which is different from the normal situation, thereby indicating that the possibility of abnormality in the functional mode has increased. Therefore, the log set corresponding to the functional mode is determined as a risk log set, which is conducive to users to promptly discover the possible risks of the functional mode.

[0145] S708: Output prompt information for the risk log set.

[0146] The prompt information for the risk log set is used to prompt the user that the function mode targeted by the log of the risk log set has abnormal risk. If there are multiple risk log sets, corresponding prompt information can be output for the multiple risk log sets.

[0147] For example, the prompt information may include the functional mode corresponding to the risk log set and prompt information that the functional mode has risks.

[0148] For another example, as an optional method, the prompt information may include a rule supplement prompt for the risk log set, and the rule supplement prompt is used to prompt the user to add anomaly detection rules for the data composition characteristics corresponding to the risk log set. It is understandable that if the log set belongs to the risk log set, it means that the functional mode corresponding to the log set has an abnormal risk. In order to timely discover this abnormal risk, the user can be prompted to deploy the anomaly detection rules for the functional mode in a more targeted and detailed manner, so as to more timely and reliably discover the possible anomalies of the functional mode based on the logs corresponding to the functional mode.

[0149] Correspondingly, after obtaining at least one anomaly detection rule set by the user for the data combination feature (or functional mode) corresponding to the risk log set, at least one anomaly detection rule corresponding to the data combination feature (or functional mode) can be stored.

[0150] It should be noted that in this embodiment, after clustering each log set and generating the corresponding data table, the operation of performing anomaly detection on the log set based on the anomaly detection rules corresponding to the log set and detecting whether the log set is a risk log set is taken as an example, that is, steps S704-S705 and steps S706 to S708 are performed in parallel. However, it can be understood that in actual applications, steps S706 to S708 can be performed after determining the abnormal logs in each log set based on the anomaly detection rules of each log set; or, after determining the abnormal logs in each log set, steps S706 to S708 can be performed for the log set without abnormal logs, and there is no restriction on this.

[0151] It is understandable that in the embodiments of the present application, multiple log data may be obtained at the same time, for example, log data may be obtained from different application clients respectively, thereby obtaining multiple log data from different application clients. In the case of obtaining multiple log data, each log data may be processed in sequence in accordance with any of the above embodiments of the present application.

[0152] In particular, after obtaining multiple log data and analyzing the abnormal logs in each log set in each log, an abnormal summary report can be generated for the multiple log data. The abnormal summary report shows the abnormal situation in each log data, wherein the abnormal situation in each log data is a summary of the abnormal situations corresponding to the abnormal logs in each log set clustered from the log data.

[0153] For example, see Figure 8 , which shows a schematic diagram of an abnormal summary report in this application.

[0154] exist Figure 8 Each field in the exception summary report represents various possible abnormal situations. For example, abnormal situations include: high CPU usage, low memory, and high sending / receiving bit rate, etc.

[0155] exist Figure 8 Each row in the exception summary report represents the exception summary corresponding to a log data, where the log data can be identified by the corresponding file name, etc., for ease of description and more intuitiveness. Figure 8 Log data 1, log data 2, etc. are used to represent different logs.

[0156] If there is an abnormality represented by a field in the log data, the value of the field in the row of the log data is 1, otherwise it is 0. For example, if there is a high CPU usage in log data 1, but there is no abnormality of low memory, then the value of the "high memory usage" field in log data 1 in the abnormal summary report is 1, and the value of the "low memory" field is 0.

[0157] It is understandable that after completing the analysis of multiple log data and generating the exception summary report, the exception summary report can be directly displayed; alternatively, the exception summary report can be displayed after detecting that the user inputs an instruction to display the summary report.

[0158] It is understandable that in the above embodiments, the abnormal log detection of log data is automatically completed by a computer or a cloud platform, etc. However, it is understandable that in actual applications, by clustering each log in the log data, it is also possible to assist users in more effectively and efficiently determining abnormal logs.

[0159] Specifically, after clustering to obtain multiple log sets, if it is determined that the manual detection mode needs to be entered, a log display interface can be displayed, and the log display interface displays multiple logs in the log data. On this basis, in response to the log display interface detecting a log selection operation, the target log selected by the log selection operation is determined, and the target log set to which the target log belongs is determined, so as to display the data of each log in the target log set.

[0160] Since logs belonging to the same functional mode are clustered into the same log set through clustering, when a user needs to view a log, the data of the logs belonging to the same functional mode as the log will be displayed, so that the user can view multiple logs in the functional mode at the same time, thereby more conveniently and efficiently analyzing whether there are abnormalities in the functional mode.

[0161] There are many specific ways to display the data of each log in the target log set. The following is an example. Fig. 9 , which shows another flow chart of a method for determining abnormal logs in the present application. The method of this embodiment may include:

[0162] S901, obtaining a set of log data to be analyzed, where the log data includes multiple logs.

[0163] S902 , clustering multiple logs in the log data according to the data composition features of the data in the logs, to obtain multiple clustered log sets and the data composition features corresponding to each log set.

[0164] S903, for each log set, use at least one data item of each log in the log set as a field in a data table to be constructed, and construct a data table based on respective values ​​of the at least one data item in each log in the log set.

[0165] As an optional method, after constructing the data table of the log collection, you can also generate a numerical change curve graph of the data item for each data item represented by each field in the data table. The process of generating the numerical change curve of the data item can be found in the previous related introduction, which will not be repeated here. On this basis, when you need to display the numerical change curve graph of a data item later, you can directly display the numerical change curve graph of the data item; or, when you need to generate a common identification curve graph for the data item, you can directly construct it on the numerical change curve graph of the data item.

[0166] S904, determining whether the set detection mode is the automatic detection mode, if yes, executing step S905; if no, executing step S909;

[0167] Among them, in step S904, the set detection mode can be determined, and the detection mode is one of the manual detection mode and the automatic detection mode. If the set detection mode is the automatic detection mode, the operations of steps S905 to S908 are executed; otherwise, step S909 is executed to assist manual detection.

[0168] It can be understood that the detection mode can be a detection mode that the user is prompted to input or select after clustering the various log sets; or it can be a detection mode presentation interface that is displayed after obtaining the log data, in which the user can select or input the user's desired detection mode. Accordingly, the set detection mode is determined in response to the mode selection or input operation in the display mode presentation interface.

[0169] S905, for each log set, determine at least one anomaly detection rule that matches the data composition feature corresponding to the log set, and based on the at least one anomaly detection rule, determine that there are abnormal records and abnormal data in the abnormal records in the data table of the log set.

[0170] S906 , for each log set, if there is an abnormal log in the log set, determine at least one target data item to which the abnormal data in each abnormal log in the log set belongs.

[0171] S907: For each target data item corresponding to the log set, construct an abnormal identification curve graph of the target data item.

[0172] The abnormal identification curve graph is a numerical value change curve graph of the numerical values ​​of the target data items in each log of the log set, and the information of abnormal data in the target data items is marked in the numerical value change curve graph.

[0173] For example, for the target data item in the log set, a value change curve graph of the target data item may be constructed first, or if the value change curve graph of the target data item has been constructed after S903, the value change curve graph of the target data item may be directly obtained. Then, an abnormal identification curve graph may be constructed based on the value change curve graph of the target data item.

[0174] S908, displaying an abnormal identification curve graph of each target data item in the log set.

[0175] S909, when the set detection mode is the manual detection mode, displaying the log display interface.

[0176] The log display interface displays multiple logs in the log data.

[0177] Among them, in the log display interface, the log logo can also be displayed in a set display mode, which can include bolding the font or enhancing the brightness to highlight the log logo, so that the user can intuitively understand that they can click on the log to view the relevant information of the log.

[0178] Optionally, the log displayed in the log display interface may also indicate the value of each data item in the log, so that the user can intuitively see the data of each data item involved in the log. For example, the data of the data item in the log may be displayed in a different color or brightness than other data in the log.

[0179] S910: In response to a log selection operation being detected on a log presentation interface, a target log selected by the log selection operation is determined.

[0180] The log selection operation may be a click on a log in the log display interface, or other operations that trigger the selection of a log.

[0181] For the sake of distinction, the log selected by the log selection operation is called the target log.

[0182] For example, when the log identifier is displayed in a specific display manner on the log display interface, in response to an operation in which the log identifier is clicked, the log corresponding to the clicked log identifier can be determined as a target log.

[0183] S911, determining the target log set to which the target log belongs, and respectively determining the value change curves of various data items corresponding to the target log set.

[0184] The log set to which the target log is clustered is the target log set to which the target log belongs.

[0185] The data items corresponding to the target log set are the data items contained in the logs of the target log set, and the value change curve of each data item is a value change curve constructed based on the value of the data item in each log in the target log set.

[0186] It is understandable that, if after clustering the log set, only a data table of the log set is generated, but a value change curve is not generated for each data item involved in the log set based on the data table, then step S911 can be based on the data table corresponding to the target log set, and the value change curve corresponding to each data item involved in the target log set is constructed. The process of constructing the value change curve of the data item can refer to the previous related introduction.

[0187] If the value change curve corresponding to each data item in the target log set has been generated before entering the manual detection mode, the corresponding value change curve can be directly queried and obtained.

[0188] S912, displaying a curve graph display interface, where the curve graph display interface displays value change curves of various data items corresponding to the target log set.

[0189] The value change curve of each data item displayed on the curve graph display interface can be seen in Figure 4 As shown, no further details are given here.

[0190] In this embodiment, if the set detection mode is the manual detection mode, then the present application can display the log display interface. On this basis, if the user clicks on a log in the log display interface, the log set to which the log is clustered will be determined, and the numerical change curve corresponding to the data items involved in each log in the log set will be displayed, so that the user can intuitively see the value changes of each data item recorded in the functional mode corresponding to the log in different logs, which is conducive to users to more efficiently discover abnormal data and efficiently locate abnormal logs.

[0191] Corresponding to a method for determining an abnormal log in the present application, the present application also provides a device for determining an abnormal log. Fig.10 As shown, it shows a schematic diagram of the composition structure of an embodiment of a device for determining an abnormal log of the present application. The device of this embodiment may include:

[0192] The log acquisition unit 1001 is used to acquire log data to be analyzed, where the log data includes multiple logs;

[0193] The log clustering unit 1002 is used to cluster multiple logs in the log data according to the data composition characteristics of the data in the log, to obtain multiple clustered log sets and the data composition characteristics corresponding to each log set, wherein the log set includes at least one log;

[0194] A rule matching unit 1003 is used to determine, for each log set, at least one anomaly detection rule that matches the data composition feature corresponding to the log set;

[0195] The anomaly determination unit 1004 is configured to determine, for each log set, an abnormal log in the log set and abnormal data in the abnormal log by using at least one abnormality detection rule corresponding to the log set.

[0196] In a possible implementation, the log clustering unit includes:

[0197] The log clustering subunit is used to cluster multiple logs in the log data according to the function identifier and character features contained in the log, wherein the function identifier in the log is used to characterize the type of function mode to which the running status information recorded in the log belongs.

[0198] In yet another possible implementation, the device may further include:

[0199] A data item determination unit, configured to determine, when there are abnormal logs in the log set, at least one target data item to which the abnormal data in each abnormal log in the log set belongs, wherein the target data item belongs to at least one data item in each log in the log set;

[0200] An abnormal curve construction unit is used to construct an abnormal identification curve graph of each target data item corresponding to the log set, wherein the abnormal identification curve graph includes a value change curve graph corresponding to the value of the target data item in each log of the log set, and information of abnormal data in the target data item is marked in the value change curve graph;

[0201] The curve display unit is used to display the abnormal identification curve graph of each target data item in the log set.

[0202] In another possible implementation, the device of the embodiment of the present application may further include:

[0203] A sample set matching unit is used to determine, for each log set, a target log sample set that matches the data composition feature corresponding to the log set from multiple log sample sets corresponding to the stored log sample data, wherein the log sample data includes multiple log samples without abnormal data, and the multiple log sample sets are obtained by clustering multiple log samples of the log sample data based on the data composition feature of the data in the log samples;

[0204] a risk log set determination unit, for determining, for each log set, a first log occurrence frequency of the log set in the log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and determining the log set as a risk log set with an abnormal risk when it is determined based on the first log occurrence frequency and the second log occurrence frequency that the number of logs in the log set has an abnormal increase in number, wherein the first log occurrence frequency is a ratio of a first number of logs included in the log set to a first total number of logs included in the log data, and the second log occurrence frequency is a ratio of a second number of logs included in the target log sample set corresponding to the log set to a second total number of logs included in the log sample data;

[0205] The risk warning unit is used to output warning information for the risk log set.

[0206] Optionally, the risk warning unit includes:

[0207] The rule supplement prompt unit is used to output a rule supplement prompt for the risk log set, and the rule supplement prompt is used to prompt the user to add anomaly detection rules according to the data composition characteristics corresponding to the risk log set.

[0208] In yet another possible implementation, the device may further include:

[0209] A mode determination unit, used to determine a set detection mode before the rule matching unit determines at least one anomaly detection rule that matches the data composition feature corresponding to the log set, the detection mode being one of a manual detection mode and an automatic detection mode;

[0210] The rule matching unit is specifically used to determine, for each log set, at least one anomaly detection rule that matches the data composition feature corresponding to the log set when the set detection mode is the automatic detection mode;

[0211] The device may also include:

[0212] A log display unit, used to display a log display interface when the set detection mode is a manual detection mode, and the log display interface displays multiple logs in the log data;

[0213] A log selection unit, configured to determine a target log selected by the log selection operation in response to the log presentation interface detecting the log selection operation;

[0214] A set determining unit, used to determine a target log set to which a target log belongs;

[0215] The data display unit is used to display the data of each log in the target log set.

[0216] Optionally, the device may further include:

[0217] A curve determination unit, used to determine the respective value change curves of various data items corresponding to the target log set before the data display unit displays the data of each log in the target log set, wherein the data items corresponding to the target log set are the data items contained in the logs of the target log set, and the value change curve of each data item is a value change curve constructed based on the value of the data item in each log of the target log set;

[0218] The data display unit is specifically used to display a curve chart display interface, and the curve chart display interface displays the value change curves of various data items corresponding to the target log set.

[0219] In another aspect, the present application also provides a computer device, which may be a personal computer, a server, or a node in a cloud platform, etc. Fig.11 , which shows a schematic diagram of the composition architecture of a computer device provided by the present application. Fig.11 In the embodiment, the computer device 1100 may include: a processor 1101 and a memory 1102 .

[0220] Optionally, the computer device may further include: a communication interface 1103 , an input unit 1104 , a display 1105 and a communication bus 1106 .

[0221] The processor 1101 , the memory 1102 , the communication interface 1103 , the input unit 1104 and the display 1105 all communicate with each other via the communication bus 1106 .

[0222] In the embodiment of the present application, the processor 1101 may be a central processing unit, a specific application integrated circuit, etc.

[0223] The processor may call the program stored in the memory 1102. Specifically, the processor may execute the operations performed by the cloud computer device side in the above embodiments.

[0224] The memory 1102 is used to store one or more programs, and the program may include a program code, and the program code includes computer operation instructions. In the embodiment of the present application, the memory at least stores a program for implementing the method for determining an abnormal log in any of the above embodiments.

[0225] In one possible implementation, the memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, the programs mentioned above, and application programs required for functions such as image playback, etc.; the data storage area may store data created during the use of the computer device.

[0226] The communication interface 1103 may be an interface of a communication module.

[0227] The present application may further include an input unit 1104, which may include a touch sensing unit, a keyboard, and the like.

[0228] The display 1105 includes a display panel, such as a touch display panel.

[0229] certainly, Fig.11 The computer device structure shown does not constitute a limitation on the computer device in the embodiment of the present application. In actual applications, the computer device may include Fig.11 More or fewer components than shown, or combinations of certain components.

[0230] On the other hand, the present application further provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the method for determining an abnormal log in any one of the above embodiments is implemented.

[0231] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other, and different embodiments can be combined with each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0232] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0233] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0234] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for determining abnormal logs, characterized in that: include: Obtaining log data to be analyzed, wherein the log data includes multiple logs; Clustering multiple logs in the log data according to the function identifier and character features contained in the log to obtain multiple clustered log sets and data composition features corresponding to each log set, wherein the log set includes at least one log, the function identifier in the log is used to characterize the type of functional mode to which the running status information recorded in the log belongs, the data composition features characterize the number and type of data items contained in the log, and the value corresponding to each data item, and the character features include one or more of the number of real numbers contained in the log, the number of characters other than real numbers, and the specific content of the characters other than real numbers; Determine a set detection mode, where the detection mode is one of a manual detection mode and an automatic detection mode; In the case where the set detection mode is the automatic detection mode, for each of the log sets, based on the data composition features corresponding to the log set, the functional mode corresponding to the logs in the log set is determined, and at least one anomaly detection rule matching the functional mode is searched according to the correspondence between the data composition features and the anomaly detection rules, wherein the anomaly detection rules include conditions that different types of data items in the log need to satisfy; For each of the log sets, using at least one anomaly detection rule corresponding to the log set, determine an abnormal log in the log set and abnormal data in the abnormal log; When the set detection mode is the manual detection mode, a log display interface is displayed, wherein the log display interface displays a plurality of logs in the log data; In response to the log presentation interface detecting a log selection operation, determining a target log selected by the log selection operation; Determine the target log set to which the target log belongs; Display the data of each log in the target log set.

2. The method according to claim 1, characterized in that Also includes: In the case where there are abnormal logs in the log set, determining at least one target data item to which the abnormal data in each abnormal log in the log set belongs, wherein the target data item belongs to at least one data item in each log in the log set; For each target data item corresponding to the log set, construct an abnormal identification curve graph of the target data item, wherein the abnormal identification curve graph includes a value change curve graph corresponding to the value of the target data item in each log of the log set, and information of abnormal data in the target data item is marked in the value change curve graph; An abnormal identification curve graph of each target data item in the log set is displayed.

3. The method according to claim 1, characterized in that: Also includes: For each log set, determine a target log sample set that matches the data composition feature corresponding to the log set from multiple log sample sets corresponding to the stored log sample data, wherein the log sample data includes multiple log samples without abnormal data, and the multiple log sample sets are obtained by clustering multiple log samples of the log sample data based on the data composition feature of the data in the log samples; For each log set, determine a first log occurrence frequency of the log set in the log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and in a case where it is determined based on the first log occurrence frequency and the second log occurrence frequency that the number of logs in the log set has an abnormal increase in number, determine the log set as a risk log set with an abnormal risk, wherein the first log occurrence frequency is a ratio of a first number of logs included in the log set to a first total number of logs included in the log data, and the second log occurrence frequency is a ratio of a second number of logs included in the target log sample set corresponding to the log set to a second total number of logs included in the log sample data; Output prompt information for the risk log set.

4. The method according to claim 3, characterized in that The outputting of prompt information for the risk log set includes: A rule supplement prompt for the risk log set is output, where the rule supplement prompt is used to prompt a user to add an anomaly detection rule for a data composition feature corresponding to the risk log set.

5. The method according to claim 1, characterized in that Before displaying the data of each log in the target log set, the method further includes: Determine the respective value change curves of various data items corresponding to the target log set, the data items corresponding to the target log set are the data items contained in the logs of the target log set, and the value change curve of each data item is a value change curve constructed based on the value of the data item in each log of the target log set; The displaying of data of each log in the target log set includes: A curve chart display interface is displayed, wherein the curve chart display interface displays value change curves of various data items corresponding to the target log set.

6. A device for determining abnormal logs, characterized in that: include: A log acquisition unit, used to acquire log data to be analyzed, wherein the log data includes a plurality of logs; A log clustering unit, configured to cluster multiple logs in the log data according to data composition features of the data in the logs, to obtain multiple clustered log sets and data composition features corresponding to each of the log sets, wherein the log set includes at least one log; A mode determination unit, used to determine a set detection mode, wherein the detection mode is one of a manual detection mode and an automatic detection mode; A rule matching unit, configured to determine, for each of the log sets, a functional mode corresponding to the logs in the log set based on the data composition features corresponding to the log set, and search for at least one anomaly detection rule matching the functional mode according to the correspondence between the data composition features and the anomaly detection rules, wherein the anomaly detection rules include conditions that different types of data items in the log need to satisfy; an abnormality determination unit, configured to determine, for each of the log sets, an abnormal log having an abnormality in the log set and abnormal data in the abnormal log by using at least one abnormality detection rule corresponding to the log set; The log clustering unit comprises: The log clustering subunit is used to cluster multiple logs in the log data according to the function identifier and character features contained in the log, wherein the function identifier in the log is used to characterize the type of the function mode to which the running status information recorded in the log belongs, the data composition feature of the log characterizes the number and type of data items contained in the log, and the value corresponding to each data item, and the character feature includes one or more of the number of real numbers contained in the log, the number of characters other than real numbers, and the specific content of characters other than real numbers; A log display unit, used for displaying a log display interface when the set detection mode is a manual detection mode, wherein the log display interface displays a plurality of logs in the log data; A log selection unit, configured to determine a target log selected by the log selection operation in response to the log presentation interface detecting the log selection operation; A set determining unit, used to determine the target log set to which the target log belongs; The data display unit is used to display the data of each log in the target log set.

7. A computer device, characterized in that: The computer device comprises: a processor and a memory; The processor is used to call and execute the program stored in the memory; The memory is used to store the program, and the program is at least used to implement the method for determining an abnormal log as described in any one of claims 1 to 5 above.

8. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the method for determining an abnormal log as described in any one of claims 1 to 5 above is implemented.

Images