Abnormality detection method and system for application program

Abstract

The present specification provides an application exception detection method and system, after obtaining the log data set of the target system in the target application, determining the target error type and the N characteristics corresponding to the log data set of the target error type, determining the target feature set related to the target error type from the N characteristics, and outputting the target feature set, so that the operation and maintenance personnel can locate the problem based on the target feature set. In this scheme, the target feature set related to the target error type is determined based on the characteristics, so as to determine the system related to the target error type in the multiple systems of the target application, which can improve the speed and accuracy of the application exception detection.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and in particular to a method and system for detecting anomalies in applications. Background Technology

[0002] Internet applications (APPs) need to iterate based on constantly changing user needs. An APP comprises multiple systems, with each iteration adding more systems. As the number of iterations increases, the number of internal systems and their code size also increase. Currently, when users encounter problems while using the APP, technical staff need to investigate multiple systems based on user feedback. After determining the general scope, they then examine the internal code logic of those systems to determine if the problem exists. Only after successfully locating the problem can it be fixed.

[0003] Therefore, there is a need for an application anomaly detection method and system that can quickly locate problems in an app. Summary of the Invention

[0004] This manual provides a method and system for detecting anomalies in applications, which can quickly locate problems in an APP.

[0005] In a first aspect, this specification provides an anomaly detection method for an application, comprising: acquiring a log dataset of a target system within a target application, the target application including multiple systems connected via data, the multiple systems including the target system; determining a target error type, wherein at least a portion of the log data in the log dataset belongs to the target error type; determining N features corresponding to the log dataset, where N is a positive integer; and determining a target feature set related to the target error type from the N features, and outputting the target feature set, wherein the target feature set characterizes the systems related to the target error type among the multiple systems.

[0006] In some embodiments, the target application includes a payment program, and the log data in the log dataset includes transaction data.

[0007] In some embodiments, the data connection includes a series connection and a parallel connection, the plurality of systems transmit data based on the data connection, and the target system includes the last system in the data transmission.

[0008] In some embodiments, the N features are mapped to the plurality of systems, and the mapping relationship includes data production relationship and data transmission relationship.

[0009] In some embodiments, obtaining the log dataset of the target system in the target application includes: periodically obtaining an initial log dataset output by the target system from the target system; cleaning the initial log dataset to obtain the log dataset; identifying error log data in the log dataset; and classifying the error log data to determine at least one error type, wherein the at least one error type includes the target error type.

[0010] In some embodiments, determining the N features corresponding to the log dataset includes: converting the log data in the log dataset into structured data; determining the N features corresponding to the structured data; and determining the feature values ​​of the N features corresponding to the log data.

[0011] In some embodiments, determining a target feature set related to the target error type from the N features includes: performing at least one round of data partitioning on the log dataset based on the information entropy corresponding to the N features to obtain at least one target log dataset, wherein the proportion of target error log data in the target log dataset is greater than a preset threshold, and the target error log data includes log data belonging to the target error type; and using features related to the at least one target log dataset as the target feature set.

[0012] In some embodiments, the step of performing at least one round of data partitioning on the log dataset based on the information entropy corresponding to the N features to obtain at least one target log dataset includes, for each round of data partitioning: determining the log dataset to be partitioned, the log dataset to be partitioned including the log dataset or at least a portion of the sub-log datasets obtained from the previous round of data partitioning; for each log dataset to be partitioned: determining the corresponding target partitioning method and target segmentation features based on the information entropy corresponding to the N features, and partitioning the current log dataset to be partitioned into multiple corresponding sub-log datasets based on the target partitioning method; determining the target log dataset and non-target log datasets from the multiple sub-log datasets, the non-target log datasets including sub-log datasets that are not the target log dataset; and determining whether to perform the next round of data partitioning on the non-target log datasets.

[0013] In some embodiments, determining the corresponding target partitioning method and target segmentation feature based on the information entropy corresponding to the N features includes: determining the feature value corresponding to each of the N features based on each log data; for each feature, pre-partitioning the log dataset to be partitioned using multiple pre-partitioning methods based on the feature value, determining multiple information entropies corresponding to the multiple pre-partitioning methods, wherein each pre-partitioning method divides the log data to be partitioned into multiple pre-partitioned sub-log datasets whose feature values ​​do not overlap; determining the minimum information entropy from the information entropies corresponding to the N features; using the pre-partitioning method corresponding to the minimum information entropy as the target partitioning method, and using the feature corresponding to the minimum information entropy as the target segmentation feature.

[0014] In some embodiments, determining whether to perform the next round of data partitioning on the non-target log dataset includes one of the following: if the non-target log dataset does not contain log data belonging to the target error type, then the non-target log dataset is not partitioned in the next round; if the number of data partitioning rounds between the non-target log dataset and the log dataset reaches a preset round number threshold, then the non-target log dataset is not partitioned in the next round; if the non-target log dataset includes log data belonging to the target error type, and the number of data partitioning rounds between the non-target log dataset and the log dataset is less than the round number threshold, then the non-target log dataset is partitioned in the next round.

[0015] In some embodiments, the step of using features related to the at least one target log dataset as the target feature set includes: using the target segmentation features corresponding to the log dataset to be segmented that includes the target log dataset in at least some of the data segmentations of the at least one round of data segmentation as the target feature set.

[0016] Secondly, this specification also provides an application anomaly detection system, comprising: at least one storage medium storing at least one instruction set for detecting adversarial attacks; and at least one processor communicatively connected to the at least one storage medium, wherein, when the application anomaly detection system is running, the at least one processor reads the at least one instruction set and executes the method described in any of the first aspects according to the instructions of the at least one instruction set.

[0017] As can be seen from the above technical solutions, the application anomaly detection method and system provided in this specification acquires the log dataset of the target system in the target application, determines the target error type and its corresponding N features in the log dataset, and determines the target feature set related to the target error type from the N features, and outputs the target feature set. In this solution, the target feature set related to the target error type is determined based on features. The target feature set is a description of the features of the log data related to the target error type. Since there is a mapping relationship between the features and multiple systems of the target application, the systems related to the target error type in the multiple systems of the target application can be indirectly derived based on the target feature set, thereby facilitating subsequent problem localization and improving the speed and accuracy of application anomaly detection.

[0018] The anomaly detection methods and other functions of the system provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the anomaly detection methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram illustrating an application scenario of an anomaly detection system for an application provided according to an embodiment of this specification is shown.

[0021] Figure 2 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0022] Figure 3 A software architecture diagram corresponding to an anomaly detection system for an application provided according to an embodiment of this specification is shown;

[0023] Figure 4 A flowchart of an anomaly detection method for an application provided according to an embodiment of this specification is shown;

[0024] Figure 5 A system architecture diagram of a target application provided according to an embodiment of this specification is shown; and

[0025] Figure 6A flowchart of each round of data partitioning provided according to embodiments of this specification is shown. Detailed Implementation

[0026] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0027] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0028] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0029] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0030] For ease of description, the terms used in this specification are explained as follows:

[0031] Application: refers to a computer program designed to perform one or more specific tasks. It runs in user mode, can interact with the user, and has a visual interface.

[0032] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0033] The application anomaly detection method provided in this specification can be applied to any application. For example, in scenarios such as facial recognition payment or QR code payment, the method can be used to detect payment anomalies in the collected log dataset of payment applications; in gaming scenarios, it can be used to detect game operation anomalies in the collected log dataset of gaming applications; in scenarios such as social apps, news apps, utility apps, and online shopping apps, it can be used to detect anomalies in the collected log dataset of the target application; it can also be applied to anomaly detection in other apps, which will not be elaborated here. For ease of description, this application will use the application anomaly detection method applied to payment recognition scenarios for anomaly detection in payment applications as an example.

[0034] Those skilled in the art should understand that the anomaly detection methods and systems for applications described in this specification, when applied to other usage scenarios, are also within the scope of protection of this specification.

[0035] Figure 1 A schematic diagram illustrating an application scenario of an anomaly detection system 100 for an application provided according to an embodiment of this specification is shown. For example... Figure 1 As shown, the application scenario may include user 200, client 300, server 400, application anomaly detection system 100 (hereinafter referred to as system 100) and network 500.

[0036] User 200 can be a user of client 300. Client 300 can communicate with server 400. User 200 can interact with server 400 through client 300. In some embodiments, server 400 can communicate with multiple clients 300. In some embodiments, client 300 can interact with server 400 through network 500 to receive or send messages, such as sending log data. In some embodiments, client 300 can have one or more applications installed. The application provides the ability and interface to interact with the outside world through network 500. The application includes, but is not limited to: web browser applications, search applications, chat applications, shopping applications, video applications, financial management applications, instant messaging tools, email clients, social platform software, etc. Client 300 can have a target application installed. The target application can be any of the above-mentioned applications. The target application can generate log data. In some embodiments, the target application can be a payment application. User 200 can make mobile payments through the target application. In this case, the log data can be transaction data.

[0037] In some embodiments, client 300 may include a mobile device, tablet, laptop, built-in device in a motor vehicle, or similar content, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or similar device, or any combination thereof. In some embodiments, the smart home device may include a smart TV, a desktop computer, etc., or any combination thereof. In some embodiments, the smart mobile device may include a smartphone, a personal digital assistant, a gaming device, a navigation device, etc., or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include a virtual reality headset, virtual reality glasses, a virtual reality patch, an augmented reality headset, augmented reality glasses, an augmented reality patch, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, a head-mounted display, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an in-vehicle computer, an in-vehicle TV, etc. In some embodiments, client 300 may be a device with positioning technology for locating the position of client 300.

[0038] Server 400 can be a server providing various services. In some embodiments, server 400 can communicate with client 300. Server 400 can provide corresponding services for applications installed on client 300. In some embodiments, server 400 can be a server providing corresponding services for the target application. For example, when the target application is a payment application, server 400 can be a server providing mobile payment functionality for the payment application. In some embodiments, server 400 can communicate with multiple clients 300 and receive data sent by clients 300, such as log data generated by the target application during use. When user 200 fails to make a mobile payment through the target application, client 300 generates error log data based on the erroneous transaction data. When user 200 successfully makes a mobile payment through the target application, client 300 generates normal log data based on the normal payment transaction data and periodically reports the collected log data to server 400 so that server 400 can perform anomaly detection on the target application based on the log data.

[0039] System 100 may be a system for anomaly detection of the target application. In some embodiments, the anomaly detection method for the application may be executed on system 100. In this case, system 100 may store data or instructions for executing the anomaly detection method for the application described herein, and may execute or be used to execute the data or instructions. In some embodiments, system 100 may include a hardware device with data processing capabilities and necessary programs for driving the hardware device. System 100 may communicate with server 400 and receive log data sent by server 400 to perform anomaly detection on the target application based on the log data.

[0040] It should be noted that system 100 can be integrated into server 400 as a module of server 400, or share computing devices with server 400, or it can be independent of server 400.

[0041] Network 500 serves as a medium to provide a communication connection between client 300, server 400, and system 100. Network 500 facilitates the exchange of information or data. For example... Figure 1As shown, client 300, server 400, and system 100 can connect to network 500 and transmit information or data to each other through network 500. In some embodiments, network 500 can be any type of wired or wireless network, or a combination thereof. For example, network 500 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), and Bluetooth networks. TM ZigBee TM A network, a near-field communication (NFC) network, or a similar network. In some embodiments, network 500 may include one or more network access points. For example, network 500 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of client 300, server 400, and system 100 can connect to network 500 to exchange data or information.

[0042] It should be understood that Figure 1 The number of clients 300, servers 400, and systems 100 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 300, servers 400, and systems 100.

[0043] Figure 2 A hardware structure diagram of a computing device 600 provided according to an embodiment of this specification is shown. The computing device 600 can execute the anomaly detection method for the application described in this specification. The anomaly detection method for the application is described in other parts of this specification. As previously mentioned, the system 100 can execute the anomaly detection method for the application described in this specification. In some embodiments, the system 100 can be integrated on a server 400, the anomaly detection method for the application can be executed on the server 400, and the computing device 600 can be the server 400. In some embodiments, the system 100 can be independent of the server 400, the anomaly detection method for the application can be executed independently on the system 100, and the computing device 600 can be the hardware device where the system 100 resides.

[0044] like Figure 2 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. Additionally, the computing device 600 may include I / O components 660.

[0045] The internal communication bus 610 can connect different system components, including storage medium 630, processor 620 and communication port 650.

[0046] I / O component 660 supports input / output between computing device 600 and other components.

[0047] Communication port 650 is used for data communication between computing device 600 and external sources. For example, communication port 650 can be used for data communication between computing device 600 and network 500. Communication port 650 can be a wired communication port or a wireless communication port.

[0048] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 also includes at least one set of instructions stored in the data storage device. Instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that perform the liveness detection methods provided in this specification.

[0049] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. At least one processor 620 is used to execute at least one instruction set described above. When the computing device 600 is running, at least one processor 620 reads at least one instruction set and, according to the instructions of at least one instruction set, executes the anomaly detection method of the application provided in this specification. Processor 620 can execute all the steps included in the anomaly detection method of the application. Processor 620 can be in the form of one or more processors. In some embodiments, processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in the computing device 600 described in this specification. However, it should be noted that the computing device 600 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0050] Figure 3 A software architecture diagram corresponding to an anomaly detection system 100 for an application provided according to an embodiment of this specification is shown. Figure 3 As shown, the system 100 may include: an information acquisition system 110, a data cleaning system 120, and a problem localization system 130. The information acquisition system 110 periodically acquires the initial log dataset of the target system in the target application collected by the client, and uploads it to the data cleaning system 120 at regular intervals. The data cleaning system 120 cleans the initial log dataset to obtain the log dataset. After data cleaning is completed, the problem localization system 130 can perform anomaly detection on the target application based on the log dataset.

[0051] It should be noted that the software system corresponding to the anomaly detection method of the application can be deployed entirely on server 400 or on computing device 600, existing independently of server 400.

[0052] Figure 4 A flowchart of an application anomaly detection method P100 according to an embodiment of this specification is shown. As previously described, computing device 600 can execute the application anomaly detection method P100 of this specification. Specifically, processor 620 can read an instruction set stored in its local storage medium and then execute the application anomaly detection method P100 of this specification according to the instructions in the instruction set. Figure 4 As shown, method P100 may include:

[0053] S120: Obtain the log dataset of the target system in the target application.

[0054] Figure 5 A system architecture diagram of a target application provided according to an embodiment of this specification is shown. The target application may include multiple systems connected by data, such as system A, system B, system C, system D, ..., system M. The multiple systems may include the target system. Data connections between the multiple systems may include series connections and / or parallel connections. The series connection may be a head-to-tail connection, that is, the input and output connections of two systems connected in series. That is, the output of the previous system is the input of the next system connected in series with it. For example, Figure 5 The examples shown are systems A and B, system B and system D, system B and system C, etc. The parallel connection can be a head-to-head connection, meaning the two systems in the parallel connection have the same input. For example... Figure 5 Systems C and D are shown in the diagram. Multiple systems transmit data based on the data connection. The target system may include the last system in the data transmission, i.e., the output data of the target system is no longer used as input data for other systems.

[0055] During the use of the target application, each of the multiple systems can generate corresponding log data. Log data generated by different systems can be transmitted via the data connection. Different systems may generate different log data and may also receive different log data. For example, system A can generate log data A, which can be passed to system B. System B can then generate corresponding log data B' based on log data A and its own generated log data B. System B can also pass log data B' to system D and system C, and so on. Therefore, to obtain more log information, method P100 can use the last system in the target application as the target system and perform anomaly detection on the target application based on the log data generated by the target system. It should be noted that those skilled in the art should understand that the target system can also be any other system in the target application, and even if the target system is any other system, it is still within the scope of protection of this specification.

[0056] The target application can be any application, such as at least one of the following: payment apps, game apps, social apps, news apps, utility apps, and online shopping apps. Taking a payment scenario as an example, when the target application can include a payment application, the log data in the log dataset can include transaction data. At least one client 300 can obtain the log dataset by recording transaction data generated by at least one user's payment behavior during a payment process over a period of time. The log dataset can include at least one correct transaction and at least one incorrect transaction. Correct transaction data refers to transaction data where the user successfully paid. Incorrect transaction data refers to transaction data where the user failed to pay. Reasons for payment failure include network problems, insufficient balance, and security issues. When a transaction fails, the client 300 records the transaction data as incorrect. Similarly, when a transaction succeeds, the client 300 may record the transaction data as correct, or it may not record the transaction data at all; for example, the client 300 may not record the transaction data as incorrect.

[0057] Client 300 can send log data to server 400 in real time or periodically. Processor 620, when performing anomaly detection on the target application, can obtain the log dataset from server 400 in real time or periodically.

[0058] The processor 620 can acquire the log dataset of the target system in the target application in the following ways: For example, the processor 620 can periodically acquire the initial log dataset output by the target system; perform data cleaning on the initial log dataset to obtain the log dataset; determine the error log data in the log dataset; and classify the error log data to determine at least one error type, wherein the at least one error type includes the target error type.

[0059] When the target application runs, it generates an initial log dataset. The processor 620 periodically retrieves this initial log dataset and performs data cleaning to obtain the final log dataset. Data cleaning may include at least one of the following: checking data consistency, handling invalid values, and handling missing values.

[0060] After the data cleaning is completed, the processor 620 can identify the erroneous log data in the log dataset based on the log dataset.

[0061] In some embodiments, each log entry in the log dataset may have a corresponding error identifier to indicate whether the log entry is an erroneous log entry. The processor 620 can determine erroneous log entries in the log dataset based on the error identifier.

[0062] In some embodiments, error log data may also correspond to an error type identifier to indicate the error type of the error log data, such as network problem, insufficient balance, and security problem. Processor 620 can determine the error log data in the log dataset based on the error type.

[0063] After identifying the error log data in the log dataset, the processor 620 can classify the error log data to determine at least one error type. Specifically, the processor 620 can group erroneous transaction data corresponding to the same error type into one category based on the error type identifier of the error log data, thereby determining at least one error type.

[0064] like Figure 4 As shown, method P100 may also include:

[0065] S140: Determine the target error type.

[0066] The target error type is one of at least one error type. At least a portion of the log data in the log dataset contains error types that belong to the target error type.

[0067] After determining at least one error type, the processor 620 can determine a target error type based on the at least one error type. For example, the processor 620 sequentially determines each error type from at least a portion of the error types of the at least one error type as the target error type. Alternatively, the processor 620 randomly selects any one error type from the at least one error type and determines it as the target error type. Alternatively, when the at least one error type is multiple error types, the processor 620 can also randomly select at least two error types from the at least one error type and sequentially determine each of the at least two error types as the target error type. Alternatively, the processor 620 determines the target error type from the at least one error type according to a preset selection rule. The preset rule may include: the number of erroneous transaction data is greater than or equal to a preset value.

[0068] like Figure 4 As shown, method P100 may also include:

[0069] S160: Determine the N features corresponding to the log dataset.

[0070] Where N is a positive integer. The N features corresponding to the log dataset refer to information describing a transaction in the log dataset from N different dimensions. The N features may include, but are not limited to, transaction object, transaction amount, transaction method, transaction product, transaction time, transaction address, transaction device, and transaction risk. Transaction object can include merchants and users, etc. Payment methods can include facial recognition payment, QR code payment, and mobile payment, etc. Payment products can include account balance and UnionPay cards, etc.

[0071] There are mapping relationships between N features and multiple systems, including data production relationships and data transmission relationships. The mapping relationships are illustrated below with reference to the accompanying diagram:

[0072] Continue reading Figure 5 Multiple systems have data transmission directions. These systems can be labeled as starting systems, ending systems, intermediate systems, etc., according to the data transmission direction. For example, if the data transmission direction indicated by the arrow in the diagram is from system A to system M, then system A, located at the beginning of the data transmission direction, can be called the starting system, and system M, located at the end of the data transmission direction, can be called the ending system. Other systems besides the starting and ending systems can be called intermediate systems. The target system can be any system among the multiple systems except the first system, such as system B, system C, system D…system M. For ease of description, systems will be labeled as the preceding or following system based on the data transmission direction. For example, if the arrow in the diagram indicates a flow from system A to system B, then system A is the preceding system of system B, and system B is the following system of system A.

[0073] Each of the multiple systems corresponds to a task. These tasks may include order processing tasks, checkout tasks, etc. When executing their respective tasks, each system generates at least one feature, or it may not generate a new feature but instead modify the feature values ​​of features transmitted from the upstream system. In some embodiments, the features generated by the multiple systems may overlap.

[0074] During the payment process, a transaction passes through multiple systems according to system logic, such as the acquiring system and the POS system. These upstream and downstream systems are interconnected and interdependent. After a transaction order passes through a system, it undergoes corresponding processing. For example, after an order is initiated, an order number is generated in the acquiring system and associated with the payment information. This data is then processed and transferred to the next system. The downstream system further processes the data based on the input from the upstream system. While the data is being processed by each system, its characteristics gradually become richer. For instance, the original transaction data may contain characteristics a and b. After passing through system I, the characteristics may change to a, b, c, and d. After passing through system J, it may further change to a, b, e, and f.

[0075] Since each system processes data, retaining all features would lead to feature explosion in the log dataset of the end system, resulting in high storage costs. Therefore, transaction data features can be selectively retained. For example, each system can receive at least one feature from the previous system based on data transmission relationships, retain at least some features from the previous system, generate new features based on data production relationships, and then transmit the retained features from the previous system and the newly generated features to the next system based on data transmission relationships. Thus, the end system possesses features from at least some of the systems upstream of the end system. Log datasets from the end system enable comprehensive and accurate anomaly detection. The processor 620 can determine the N features corresponding to the log dataset in various ways, such as converting log data in the log dataset into structured data, determining the N features corresponding to the structured data, and determining the feature values ​​of the N features corresponding to the log data.

[0076] The log data in the log dataset is unstructured. Converting it to structured data facilitates subsequent processing by the processor 620. After converting the log data to structured data, the processor 620 also needs to determine the N features corresponding to the structured data, and the feature values ​​of these N features. For example, if one of the N features is "user," then the feature value can be a specific user, such as user 1, user 2, user 3... user n, where n is a positive integer. If one of the N features is "merchant," then the feature value can be a specific merchant, such as merchant 1, merchant 2, merchant 3... merchant k, where k is a positive integer. If one of the N features is "payment method," then the feature value can be a specific payment method, such as facial recognition payment, card payment, or QR code payment.

[0077] like Figure 4 As shown, method P100 may also include:

[0078] S180: Determine the target feature set related to the target error type from N features and output the target feature set.

[0079] As mentioned earlier, there is a mapping relationship between N features and multiple systems. When a user encounters an error while using the target application, it is often because a problem has occurred in a system within the target application, such as system C. When system C encounters a problem, it generates an error message, which is then passed to system D, causing system D to report an error, and ultimately causing system M to report an error, allowing the user to see the failure. In this situation, maintenance personnel cannot directly determine from the error message that system C caused the failure, because it is possible that system B or even system A made an error in transmitting information, leading to system C's error. Therefore, in order to quickly locate the abnormal system in the target application, the method P100 can extract features from the log dataset and perform feature analysis to determine the target feature set related to the target error type among the N features. Based on the mapping relationship between features and systems, the abnormal system can be quickly located, thereby improving the efficiency of anomaly detection. The target feature set represents the systems related to the target error type among multiple systems. Step S180 may include the following steps: The processor 620 performs at least one round of data partitioning on the log dataset based on the information entropy corresponding to N features to obtain at least one target log dataset, and uses the features related to at least one target log dataset as the target feature set.

[0080] The processor 620 performs at least one round of data partitioning of the log dataset based on the information entropy corresponding to N features. The principle is as follows: information entropy represents the amount of information a feature provides for anomaly detection, or its importance in anomaly detection. The amount of information provided for anomaly detection can be the amount of information provided for locating the abnormal system under the target error type. Information entropy is negatively correlated with the amount of information it provides for anomaly detection. The larger the information entropy, the smaller the amount of information it provides for anomaly detection. The smaller the information entropy, the larger the amount of information it provides for anomaly detection. The larger the information entropy of a feature, the smaller the correlation between the feature and the target error type. The smaller the information entropy of a feature, the greater the correlation between the feature and the target error type. Based on information entropy, the scenario in which the problem occurred can be gradually refined, i.e., which erroneous transaction data caused the target error type. Information entropy allows the sub-log datasets obtained in each round of partitioning to gradually approximate the target log dataset, thus facilitating problem localization.

[0081] Based on the information entropy corresponding to N features, the processor 620, after at least one round of data partitioning of the log dataset, ensures that the proportion of target error log data in each of the resulting target log datasets is greater than a preset threshold. Target error log data includes log data belonging to the target error type. This concentrates erroneous transaction data corresponding to the target error type within the target log dataset, facilitating subsequent problem localization.

[0082] Figure 6 A flowchart illustrating each round of data partitioning provided according to embodiments of this specification is shown. Figure 6 As shown, when the processor 620 performs at least one round of data partitioning on the log dataset based on the information entropy corresponding to N features, the implementation method for each round of data partitioning can be as follows:

[0083] S620: Determine the log dataset to be partitioned.

[0084] The log dataset to be partitioned includes the log dataset itself or at least a portion of the sub-log datasets obtained from the previous round of data partitioning. For ease of description, we define the partitioned dataset obtained after each data partition as the sub-log dataset obtained from the current data partition. During each round of data partitioning, the processor 620 can determine the log dataset to be partitioned based on whether the current partition is the first round. For example, if the current data partition is the first round, the processor 620 can use the acquired log dataset as the log dataset to be partitioned. When the current data partition is a partition after the first round, the processor 620 uses at least a portion of the sub-log datasets obtained from the previous round of data partitioning as the log dataset to be partitioned; that is, the processor 620 uses the sub-log datasets from the previous round that need to be further partitioned as the log dataset to be partitioned.

[0085] S640: For each log dataset to be partitioned, based on the information entropy corresponding to N features, determine the corresponding target partitioning method and target segmentation features, and divide the current log dataset to be partitioned into multiple corresponding sub-log datasets based on the target partitioning method.

[0086] The specific implementation of step S640 can be as follows: For example, the processor 620 determines the feature value corresponding to each of the N features based on each log data; for each feature, based on the feature value, pre-divides the log dataset to be divided in multiple pre-division methods, determines multiple information entropies corresponding to the multiple pre-division methods, and determines the minimum information entropy from the information entropies corresponding to the N features; and uses the pre-division method corresponding to the minimum information entropy as the target division method, and uses the feature corresponding to the minimum information entropy as the target segmentation feature.

[0087] Each log data point corresponds to N features, and each of these N features has a corresponding feature value. The feature value represents the value of the log data under that feature. Let's continue with the payment scenario as an example. For instance, if the feature is the payment object, and the payment object is a merchant, the feature value could be Merchant 1, Merchant 2, Merchant 3, ..., Merchant n, etc. Similarly, if the feature is the payment object, and the payment object is a user, the feature value could be User 1, User 2, User 3, ..., User n, etc. If the feature is the payment method, the feature value could be facial recognition payment, QR code payment, or card payment, etc. Finally, if the feature is the payment amount, the feature value could be any payment value within a certain precision range.

[0088] For each of the N features, the processor 620 can employ various pre-partitioning methods to pre-partition the feature value corresponding to that feature. Each pre-partitioning method can divide the feature value corresponding to that feature into multiple pre-partitioned sub-log datasets. Based on the multiple pre-partitioned sub-log datasets obtained by this pre-partitioning method, the processor 620 can determine the information entropy corresponding to this pre-partitioning method. Specifically, each pre-partitioning method divides the log dataset to be partitioned into multiple pre-partitioned sub-log datasets with mutually disjoint feature values, and the union of the multiple pre-partitioned sub-log datasets is the log dataset to be partitioned. The mutually disjoint feature values ​​can mean that for a single feature, the feature values ​​contained in the multiple pre-partitioned sub-log datasets are different, and the same feature value does not appear in multiple different pre-partitioned sub-log datasets. For example, the N features include feature i, feature j, and feature k. The processor 620 can denote the log dataset to be partitioned as S, and based on the feature value corresponding to each of the N features, iterates through multiple possible partitioning methods of S, i.e., iterates through multiple pre-partitioning methods of S, and determines multiple information entropies corresponding to the multiple pre-partitioning methods. For example, the processor 620 can perform various pre-partitions on S based on multiple feature values ​​i1, i2, i3, ..., iL corresponding to feature i, including at least the following pre-partition methods:

[0089] Pre-partitioning method S1: The processor 620 divides S into pre-partitioned sub-log datasets A1(i1, i2) and B1(i3, ..., iL). A1 and B1 are disjoint, and the union of A1 and B1 is S.

[0090] Pre-partitioning method S2: The processor 620 divides S into pre-partitioned sub-log datasets A2 (i1, i2, i3) and B2 (i4, ..., iL). A2 and B2 are disjoint, and the union of A2 and B2 is S.

[0091] Pre-partitioning method S3: The processor 620 divides S into pre-partitioned sub-log datasets A3 (i1, i2, i3, i4) and B3 (i5, ..., iL). A3 and B3 are disjoint, and the union of A3 and B3 is S.

[0092] It should be understood that the above pre-partitioning method is an illustrative example, and the pre-partitioning does not have to be based on the order of feature values. For example, in pre-partitioning method S1, S can also be divided into pre-partitioned sub-log datasets A1(i1, i3) and B1(i2, i4, ..., iL).

[0093] For the three pre-partitioning methods mentioned above, the processor 620 calculates the information entropy corresponding to pre-partitioning method 1, pre-partitioning method 2, and pre-partitioning method 3 respectively, obtaining three information entropies corresponding to feature i, denoted as information entropy i1, information entropy i2, and information entropy i3 respectively. The information entropy can be determined using the following formula (1):

[0094]

[0095] In equation (1), Entropy j,At Let represent the information entropy of the pre-partitioned sub-log dataset At corresponding to the pre-partitioning of S based on feature j, where t can take any value from the set A1, B1, A2, B2, A3, and B3; Err y The target error type y represents the target error type; X represents all feature values ​​in the pre-partitioned sub-log dataset At; x is the feature value variable; p x,j This represents the error probability corresponding to feature value x under feature j, under the target error type.

[0096] The processor 620 can determine p in the following way. x,j For example, processor 620 determines the proportion of error log data corresponding to feature value x under feature j in all log data corresponding to feature value x under feature j, based on the target error type. For example, if feature j is user and feature value x is user 2, then p x,j This represents the percentage of error log data of the target error type in all log data of User 2 in the log dataset.

[0097] Based on the principle of formula (1) above, processor 620 can determine the information entropy of each pre-partitioned sub-log dataset in multiple pre-partitioned sub-log datasets under each pre-partitioning method. And based on the information entropy of each pre-partitioned sub-log dataset in multiple pre-partitioned sub-log datasets under each pre-partitioning method, the information entropy under each pre-partitioning method is determined. Specifically, it can be determined using the following formula (2):

[0098]

[0099] In equation (2), St represents the t-th pre-partitioning method of the dataset S to be partitioned; a represents the number of feature values ​​in set At, b represents the number of feature values ​​in set Bt, and the rest of the characters can be found in the introduction of equation (1).

[0100] Similarly, the processor 620 can also obtain the pre-partitioning method and its information entropy corresponding to feature j, such as 4 pre-partitioning methods and their 4 information entropies, denoted as information entropy j1, information entropy j2, information entropy j3, and information entropy j4, respectively. It can also obtain the pre-partitioning method and its information entropy corresponding to feature k, such as 5 pre-partitioning methods and their 5 information entropies, denoted as information entropy k1, information entropy k2, information entropy k3, information entropy k4, and information entropy k5, respectively.

[0101] After the processor 620 has traversed all N features, it can obtain the information entropy corresponding to the N features. Then, it determines the minimum information entropy from among the information entropies corresponding to the N features. There are several ways to determine the minimum information entropy from among the information entropies corresponding to the N features, as follows:

[0102] In some embodiments, the processor 620 can select the minimum information entropy from all information entropies corresponding to N features, and use it as the minimum information entropy corresponding to the N features. For example, the processor 620 sorts the 12 information entropies corresponding to features i, j, and k in ascending order, and determines the information entropy at the top of the list as the minimum information entropy. Alternatively, the processor 620 sorts the 12 information entropies corresponding to features i, j, and k in descending order, and determines the information entropy at the bottom of the list as the minimum information entropy.

[0103] In some embodiments, the processor 620 can further select the minimum information entropy from multiple information entropies corresponding to each of the N features, and use it as the information entropy to be selected. Thus, N information entropies to be selected can be obtained from the N features. Then, the processor 620 determines the minimum information entropy from the N information entropies to be selected, and uses it as the minimum information entropy corresponding to the N features. For example, the processor 620 can determine the minimum information entropy from the 3 information entropies corresponding to feature i, as information entropy to be selected 1; determine the minimum information entropy from the 4 information entropies corresponding to feature j, as information entropy to be selected 2; and determine the minimum information entropy from the 5 information entropies corresponding to feature k, as information entropy to be selected 3. The processor 620 then selects the minimum information entropy to be selected from information entropy to be selected 1, information entropy to be selected 2, and information entropy to be selected 3, and uses it as the minimum information entropy corresponding to the N features.

[0104] After determining the minimum information entropy, the processor 620 can use the pre-partitioning method corresponding to the minimum information entropy as the target partitioning method and the feature corresponding to the minimum information entropy as the target segmentation feature. For example, if the minimum information entropy among the information entropies corresponding to N features is information entropy i3, then the processor 620 determines the pre-partitioning method 3 corresponding to i3 as the target partitioning method and determines feature i as the target segmentation feature. Based on the feature value corresponding to feature i, the processor 620 performs this round of partitioning on the log dataset to be partitioned using the target partitioning method. That is, in this round of partitioning, the processor 620 divides the log dataset S into sub-log dataset A3 and sub-log dataset B3.

[0105] The above embodiments describe at least one round of data partitioning of the log dataset based on information entropy. In some embodiments, the processor 620 can also perform at least one round of data partitioning of the log dataset based on information gain. The process for determining information gain can be expressed by the following formula (3):

[0106]

[0107] In equation (3), Gain represents information gain; Entropy(Err) y ) represents the information entropy corresponding to the target error type y, which can be determined based on the proportion of error log data corresponding to the target error type y in all log data in the log dataset to be divided.

[0108] Unlike the entropy-based dataset partitioning method, when pre-partitioning the log dataset to be partitioned using various pre-partitioning methods based on feature values ​​for each feature, the processor 620 determines the maximum information entropy from the information gains corresponding to N features; and uses the pre-partitioning method corresponding to the maximum information gain as the target partitioning method, and the feature corresponding to the maximum information gain as the target segmentation feature. The remaining processes are the same as the entropy-based dataset partitioning method, and can be referred to the aforementioned introduction on the implementation process of the entropy-based dataset partitioning method, which will not be repeated here.

[0109] Continue reading Figure 6 After step S640, steps S660 and S680 may also be included.

[0110] S660: Determine the target log dataset and non-target log datasets from multiple sub-log datasets; and determine whether to proceed with the next round of data partitioning for the non-target log datasets.

[0111] Non-target log datasets include sub-log datasets that are not part of the target log dataset.

[0112] Processor 620 can determine the target log dataset for each sub-log dataset within multiple sub-log datasets when the proportion of error log data in that sub-log dataset exceeds a preset threshold. It then designates the remaining sub-log datasets (excluding the target dataset) as non-target datasets. Continuing with the example above, if processor 620 determines that A3 is the target log dataset and B3 is a non-target log dataset from A3 and B3, then A3 no longer needs to be partitioned, while B3 requires further determination to determine whether a next round of data partitioning is necessary.

[0113] S680: Determine whether to proceed with the next round of data partitioning for the non-target log dataset.

[0114] There are several ways to implement step S680, which can be as follows:

[0115] In some embodiments, the processor 620 may not perform the next round of data partitioning on the non-target log dataset if it determines that the non-target log dataset does not contain log data of the target error type. For example, when the non-target log dataset B3 contains only log data of the non-target error type, it indicates that the partitioning of the log dataset is complete.

[0116] In some embodiments, the processor 620 may, if it determines that the number of data partitioning rounds between the non-target log dataset and the log dataset has reached a preset round threshold, not proceed with the next round of data partitioning for the non-target log dataset. For example, if the total number of partitioning rounds for the log dataset reaches the preset round threshold during the current partitioning round, then the partitioning of the log dataset ends.

[0117] In addition to the two scenarios described in the above embodiments, the processor 620 needs to perform another round of data partitioning on the non-target log dataset. For example, the processor 620 may perform another round of data partitioning on the non-target log dataset when it determines that the non-target log dataset includes log data of the target error type, and the number of data partitioning rounds between the non-target log dataset and the log dataset is less than a round number threshold. For example, when the non-target log dataset B3 includes log data of the target error type, and the total number of partitioning rounds for the log dataset in this round has not reached the preset round number threshold, the processor 620 performs another round of data partitioning on the non-target log dataset. When it is determined that the non-target log dataset needs to be partitioned in the next round, the processor 620 may determine the non-target log dataset as the log dataset to be partitioned and execute steps S640, S660, and S680 until the partitioning is completed.

[0118] When processor 620 performs at least one round of partitioning on the log dataset and ends the partitioning process, it can use the target segmentation features corresponding to the log dataset to be partitioned that include the target log dataset from at least some of the data partitioning rounds performed in the at least one round of partitioning as the target feature set. For example, if processor 620 performs 100 rounds of partitioning on the log dataset, it can determine the target segmentation features corresponding to the log dataset to be partitioned that include the target log dataset from some or all of the 100 rounds of partitioning as the target feature set. The target feature set can be understood as a description of the problem scenario, which includes the target segmentation features and their corresponding feature values.

[0119] After the processor 620 generates the target feature set, maintenance personnel can use it to investigate the error and its corresponding problem location. If the investigation confirms a real problem, the problem is fixed, and feedback is given to the data cleaning system to emphasize the importance of the features used to locate the problem. Simultaneously, the anomaly detection is recorded in the experience database so that repair suggestions can be directly provided when the same or similar problems occur again. If the investigation confirms a false alarm, feedback is given to the problem location system, indicating that the problem location was incorrect, allowing it to self-adjust. Upon receiving feedback, the problem location system will weaken the importance of this feature based on the specific circumstances, and if it confirms that the feature is useless, it will be fed back to the data cleaning system for pre-removal of the feature.

[0120] In some embodiments, the processor 620 can also establish a causal relationship graph through the log dataset, that is, construct a causal relationship graph between the target error type and the feature. For example, a causal relationship graph is established between the target segmentation feature and the target error type corresponding to the log dataset to be segmented, which contains the target log dataset, so as to discover and locate the problem.

[0121] In summary, the application anomaly detection method P100 and system 100 provided in this specification collect log data from the target system within the target application, determine the target error type based on the error type corresponding to the log dataset, identify N features corresponding to the log dataset, and determine the target feature set related to the target error type from the N features. This target feature set is then output so that operations and maintenance personnel can locate problems based on it. In this scheme, the target feature set related to the target error type is determined based on features. The target feature set describes the characteristics of the log data related to the target error type, and there is a mapping relationship between these features and multiple systems within the target application. Therefore, based on the target feature set, the systems within the target application related to the target error type can be indirectly deduced, facilitating subsequent problem location and improving the speed and accuracy of application anomaly detection.

[0122] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing anomaly detection of an application. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the anomaly detection method P100 of the application described in this specification. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 600, the program code causes the computing device 600 to perform the steps of the anomaly detection method P100 of the application described in this specification. The program product for implementing the above method may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing device 600. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 600, partially on computing device 600, as a standalone software package, partially on computing device 600 and partially on a remote computing device, or entirely on a remote computing device.

[0123] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0124] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0125] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0126] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0127] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. All contents used for all purposes, except for any history of prosecution documents relating to it, that may be inconsistent with or conflict with this document, or any such history of prosecution documents that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0128] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. An anomaly detection method for an application, comprising: Obtain the log dataset of the target system in the target application, wherein the target application includes multiple systems connected to data, and the multiple systems include the target system; Determine the target error type, wherein at least some log data in the log dataset has an error type that belongs to the target error type; Determine N features corresponding to the log dataset, where N is a positive integer; and Based on the information entropy corresponding to the N features, the log dataset is divided into at least one round of data partitioning to obtain at least one target log dataset. The proportion of target error log data in the target log dataset is greater than a preset threshold. The target error log data includes log data belonging to the target error type. The information entropy is determined based on the error probability corresponding to each feature value of the feature under the target error type. From the N features, features related to the at least one target log dataset are selected as the target feature set related to the target error type; and Output the target feature set, wherein the target feature set represents the systems in the plurality of systems that are related to the target error type.

2. The method according to claim 1, wherein, The target application includes payment applications, and the log data in the log dataset includes transaction data.

3. The method according to claim 1, wherein, The data connection includes serial connection and parallel connection, the multiple systems transmit data based on the data connection, and the target system includes the last system in the data transmission.

4. The method according to claim 1, wherein, The N features are mapped to the multiple systems, and the mapping relationship includes data production relationship and data transmission relationship.

5. The method according to claim 1, wherein, The acquisition of the target system's log dataset in the target application includes: Periodically obtain the initial log dataset output by the target system; The initial log dataset is cleaned to obtain the log dataset. Determine the error log data in the log dataset; and The error log data is classified to determine at least one error type, and the at least one error type includes the target error type.

6. The method according to claim 1, wherein, The determination of the N features corresponding to the log dataset includes: Convert the log data in the log dataset into structured data; Determine the N features corresponding to the structured data; and Determine the feature values ​​of the N features corresponding to the log data.

7. The method according to claim 1, wherein, The process of partitioning the log dataset at least once based on the information entropy corresponding to the N features to obtain at least one target log dataset includes each round of data partitioning: Determine the log dataset to be partitioned, wherein the log dataset to be partitioned includes the log dataset or at least a portion of the sub-log datasets obtained from the previous round of data partitioning; For each of the log datasets to be partitioned: Based on the information entropy corresponding to the N features, the corresponding target partitioning method and target segmentation features are determined, and the current log dataset to be partitioned is divided into multiple corresponding sub-log datasets based on the target partitioning method. The target log dataset and non-target log dataset are determined from the plurality of sub-log datasets, wherein the non-target log dataset includes sub-log datasets that are not the target log dataset from the plurality of sub-log datasets; as well as Determine whether to proceed with the next round of data partitioning for the non-target log dataset.

8. The method according to claim 7, wherein, The step of determining the corresponding target partitioning method and target segmentation features based on the information entropy corresponding to the N features includes: Based on each log data, determine the feature value corresponding to each of the N features; For each feature, based on the feature value, the log dataset to be divided is pre-divided in multiple pre-division methods to determine multiple information entropies corresponding to the multiple pre-division methods, wherein each pre-division method divides the log dataset to be divided into multiple pre-divided sub-log datasets whose feature values ​​do not overlap. Determine the minimum information entropy from the information entropy corresponding to the N features; and The pre-segmentation method corresponding to the minimum information entropy is used as the target segmentation method, and the feature corresponding to the minimum information entropy is used as the target segmentation feature.

9. The method according to claim 7, wherein, The step of determining whether to proceed with the next round of data partitioning for the non-target log dataset includes one of the following: If it is determined that the non-target log dataset does not contain log data of the target error type, then the non-target log dataset will not be partitioned in the next round. If the number of data partitioning rounds between the non-target log dataset and the log dataset reaches a preset round threshold, then the non-target log dataset will not be partitioned in the next round. If it is determined that the non-target log dataset includes log data belonging to the target error type, and the number of data partitioning rounds between the non-target log dataset and the log dataset is less than the number of rounds threshold, then the non-target log dataset will be partitioned into the next round.

10. The method according to claim 7, wherein, The step of using features related to the at least one target log dataset as the target feature set includes: The target feature set is defined as the target segmentation feature corresponding to the log dataset to be segmented, which is included in at least some of the data segmentation rounds of the at least one round of data segmentation.

11. An anomaly detection system for an application, comprising: At least one storage medium storing at least one instruction set for performing anomaly detection of the application; as well as At least one processor is communicatively connected to the at least one storage medium. When the anomaly detection system of the application is running, the at least one processor reads the at least one instruction set and executes the method according to any one of claims 1-10 according to the instructions of the at least one instruction set.

Images