Data processing method and device, equipment, medium and program product

Abstract

The invention provides a data processing method which can be applied to the technical field of artificial intelligence. The data processing method comprises the steps that multiple sets of associated log data are obtained, the multiple sets of associated log data indicate at least one behavior chain of at least one user, the behavior chain comprises multiple behavior nodes, and the behavior nodes represent operation behaviors of the user for service data; determining an abnormal operation behavior in the plurality of operation behaviors of the at least one user and a target behavior node corresponding to the abnormal operation behavior based on the plurality of groups of associated log data; determining target associated log data related to the abnormal operation behavior from the multiple groups of associated log data; and determining a plurality of behavior nodes before the target behavior node in the behavior chain where the target behavior node is located based on the target associated log data by using a pre-trained graph neural network model to obtain an abnormal behavior chain. The invention further provides a data processing device and equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and more particularly to a data processing method, device, equipment, medium and program product. BACKGROUND

[0002] In the related art, the verification for sensitive data leakage is often limited to independent analysis of a single log type. Such one-sided verification method can only capture partial fragments in the leakage chain and lacks correlation and integration of log data from different sources and different time points, thus it is difficult to restore the complete leakage chain. SUMMARY

[0003] In view of the above problems, the present application provides a data processing method, device, equipment, medium and program product.

[0004] According to a first aspect of the present application, a data processing method is provided, comprising: obtaining a plurality of sets of associated log data, the plurality of sets of associated log data indicating at least one behavior chain of at least one user, the behavior chain comprising a plurality of behavior nodes, the behavior nodes representing operation behaviors of the user on business data; determining, based on the plurality of sets of associated log data, an abnormal operation behavior in the plurality of operation behaviors of the at least one user and a target behavior node corresponding to the abnormal operation behavior; determining, from the plurality of sets of associated log data, target associated log data related to the abnormal operation behavior; and determining, based on the target associated log data, a plurality of behavior nodes before the target behavior node in the behavior chain in which the target behavior node is located, using a pre-trained graph neural network model, to obtain an abnormal behavior chain.

[0005] According to an embodiment of the present application, the determining, based on the plurality of sets of associated log data, of the abnormal operation behavior in the plurality of operation behaviors of the at least one user comprises: performing feature extraction on the plurality of sets of associated log data to obtain an associated feature sequence; inputting each associated feature of the associated feature sequence into a pre-trained isolation forest model to obtain an abnormal score group corresponding to the associated feature sequence, the abnormal score group comprising a plurality of abnormal scores sorted according to a time sequence; and determining the at least one abnormal behavior based on the abnormal score group.

[0006] According to an embodiment of the present application, the determining, based on the plurality of sets of associated log data, of the abnormal operation behavior in the plurality of operation behaviors of the at least one user comprises: performing feature extraction on the plurality of sets of associated log data to obtain an associated feature sequence; inputting each associated feature of the associated feature sequence into a pre-trained isolation forest model to obtain an abnormal score group corresponding to the associated feature sequence, the abnormal score group comprising a plurality of abnormal scores sorted according to a time sequence; and determining the at least one abnormal behavior based on the abnormal score group.

[0007] According to an embodiment of the present application, the method further comprises: in the case that each of the groups of abnormal scores in the time dimension exceeds the preset threshold, sending an alarm information to a preset user.

[0008] According to an embodiment of the present application, the pre-trained graph neural network model is obtained by the following method: obtaining sample correlation log data, the sample log correlation data comprising at least one sample abnormal behavior chain, a sample abnormal behavior node of the at least one sample abnormal behavior chain, and a real predecessor node of the sample abnormal behavior node; constructing a sample adjacency matrix based on the sample correlation log data, the sample adjacency matrix representing the correlation relationship between systems corresponding to each sample log in the sample log correlation data; inputting the sample adjacency matrix and the sample abnormal behavior node into the graph neural network model to obtain a sample predecessor node of the sample abnormal behavior node and a predicted value of the sample predecessor node; determining a loss value based on the predicted value of the sample predecessor node and the predicted value of the real predecessor node; updating the graph neural network model using the loss value to obtain the pre-trained graph neural network model.

[0009] According to an embodiment of the present application, the pre-trained graph neural network model is obtained by the following method: obtaining sample correlation log data, the sample log correlation data comprising at least one sample abnormal behavior chain, a sample abnormal behavior node of the at least one sample abnormal behavior chain, and a real predecessor node of the sample abnormal behavior node; constructing a sample adjacency matrix based on the sample correlation log data, the sample adjacency matrix representing the correlation relationship between systems corresponding to each sample log in the sample log correlation data; inputting the sample adjacency matrix and the sample abnormal behavior node into the graph neural network model to obtain a sample predecessor node of the sample abnormal behavior node and a predicted value of the sample predecessor node; determining a loss value based on the predicted value of the sample predecessor node and the predicted value of the real predecessor node; updating the graph neural network model using the loss value to obtain the pre-trained graph neural network model.

[0010] According to an embodiment of the present application, the sample correlation log comprises a session identifier, a user identifier and an operation time, and the constructing a sample adjacency matrix based on the sample correlation log data comprises: determining a candidate correlation log group related to the user identifier from the sample correlation log based on the user identifier; screening the candidate correlation log group by using a preset time interval to determine a first log correlation relationship between the candidate correlation log groups, wherein the first log correlation relationship represents that the operation time interval of each two candidate logs is less than the preset time interval; determining a second log correlation relationship between the sample correlation logs based on the session identifier; and constructing the adjacency matrix based on the first log correlation relationship and the second log correlation relationship.

[0011] A second aspect of the present application provides a data processing apparatus, comprising: an acquisition module configured to acquire a plurality of sets of correlation log data, wherein the plurality of sets of correlation log data indicates at least one behavior chain of at least one user, the behavior chain comprises a plurality of behavior nodes, and each behavior node represents an operation behavior of the user on service data; an abnormal operation behavior determination module configured to determine, based on the plurality of sets of correlation log data, an abnormal operation behavior in a plurality of operation behaviors of the at least one user and a target behavior node corresponding to the abnormal operation behavior; a target correlation log data determination module configured to determine, from the plurality of sets of correlation log data, target correlation log data related to the abnormal operation behavior; and an abnormal behavior chain determination module configured to determine, based on the target correlation log data, a plurality of behavior nodes before the target behavior node in the behavior chain in which the target behavior node is located by using a pre-trained graph neural network model, to obtain an abnormal behavior chain.

[0012] According to an embodiment of the present application, the abnormal operation behavior determination module comprises: a feature extraction sub-module configured to perform feature extraction on the plurality of sets of correlation log data to obtain a correlation feature sequence; an abnormal score group determination sub-module configured to input each correlation feature of the correlation feature sequence into a pre-trained isolation forest model to obtain an abnormal score group corresponding to the correlation feature sequence, wherein the abnormal score group comprises a plurality of abnormal score groups sorted according to a time sequence; and a first abnormal behavior determination sub-module configured to determine the at least one abnormal behavior based on the abnormal score group.

[0013] According to an embodiment of the present application, the correlation log data comprises domain name data, and the abnormal operation behavior determination module comprises: a target correlation log data determination sub-module configured to determine the target correlation log data based on the domain name data in a case where a preset list comprises the domain name data; and a second abnormal behavior determination sub-module configured to determine an operation behavior indicated by the target correlation log data as an abnormal behavior.

[0014] According to an embodiment of the present application, the device further comprises an alarm module configured to send alarm information to a preset user if a plurality of groups of abnormal scores in the abnormal score group that are continuous in the time dimension all exceed a preset threshold.

[0015] According to an embodiment of the present application, the device further comprises a sample obtaining module configured to obtain sample association log data, wherein the sample log association data comprises at least one sample abnormal behavior chain, a sample abnormal behavior node of the at least one sample abnormal behavior chain, and a real predecessor node of the sample abnormal behavior node; an adjacency matrix constructing module configured to construct a sample adjacency matrix based on the sample association log data, wherein the sample adjacency matrix represents an association relationship between systems corresponding to each sample log in the sample log association data; a predicted value determining module configured to input the sample adjacency matrix and the sample abnormal behavior node into a graph neural network model to obtain a sample predecessor node of the sample abnormal behavior node and a predicted value of the sample predecessor node; a loss value determining module configured to determine a loss value based on the predicted value of the sample predecessor node and the predicted value of the real predecessor node; and an updating module configured to update the graph neural network model using the loss value to obtain the pre-trained graph neural network model.

[0016] According to an embodiment of the present application, the sample association log comprises a session identifier, a user identifier, and an operation time, and the adjacency matrix constructing module comprises: a candidate association log group determining submodule configured to determine a candidate association log group related to the user identifier from the sample association log based on the user identifier; a first log association relationship determining submodule configured to filter the candidate association log group using a preset time interval to determine a first log association relationship between the candidate association log group, wherein the first log association relationship represents that an operation time interval of each two candidate logs is less than the preset time interval; a second log association relationship determining submodule configured to determine a second log association relationship between the sample association logs based on the session identifier; and a matrix constructing submodule configured to construct the adjacency matrix based on the first log association relationship and the second log association relationship.

[0017] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0018] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0019] The fifth aspect of the present application also provides a computer program product comprising computer programs or instructions, which, when executed by a processor, implement the steps of the above method.

[0020] According to the embodiments of the present application, by tracing the multiple sets of association log data indicating at least one behavior chain of at least one user as the trace data, the limitation of single log data is avoided, and then by determining the abnormal operation behavior of at least one user and the target behavior node corresponding to the abnormal operation behavior from the association log data, the abnormal behavior can be accurately located, and then the abnormal behavior chain is constructed, which provides a construction basis for subsequent construction of the abnormal behavior chain, further, by means of the pre-trained graph neural network, the abnormal behavior chain is obtained by taking the target behavior node as a starting point for forward tracing, which improves the accuracy and response efficiency of system security check, reduces the manual troubleshooting burden of related operation and maintenance personnel, and improves the tracing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other objects, features and advantages of the present application will become more apparent from the following description of the embodiments of the present application taken with reference to the accompanying drawings, in which:

[0022] Figure 1 An application scenario diagram of a data processing method and device according to an embodiment of the present application is schematically shown;

[0023] Figure 2 A flowchart of a data processing method according to an embodiment of the present application is schematically shown;

[0024] Figure 3 A schematic diagram of constructing an abnormal behavior chain according to an embodiment of the present application is schematically shown;

[0025] Figure 4 An architecture diagram of a graph neural network model according to an embodiment of the present application is schematically shown;

[0026] Figure 5 An architecture diagram of a system constructed by a data processing method according to an embodiment of the present application is schematically shown;

[0027] Figure 6 A structural block diagram of a data processing device according to an embodiment of the present application is schematically shown;

[0028] Figure 7 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0029] Embodiments of the present application will be described below with reference to the accompanying drawings. However, it should be understood that the description is merely exemplary and is not intended to limit the scope of the present application. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to one skilled in the art that one or more embodiments can be practiced without these specific details. In other instances, well-known structures and techniques have been omitted in order to avoid obscuring the concepts of the present application.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, means the inclusion of the stated features, steps, operations, and / or components but not to the exclusion of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0032] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should be generally interpreted as including at least one of the items enumerated after the term, unless otherwise defined (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0033] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user equipment information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal.

[0034] In a scenario of making an automated decision by using personal information, the method, device and system provided by the embodiments of the present application provide a corresponding operation entry for a user to select to agree or reject the automated decision result; if the user selects to reject, an expert decision process is entered. The expression "automated decision" herein refers to an activity of making a decision by automatically analyzing and evaluating, by a computer program, a user's behavior habit, interest and hobby, or economic, health and credit status, etc. The expression "expert decision" herein refers to an activity of making a decision by a person who is engaged in a certain field, has special experience, knowledge and skills and reaches a certain professional level.

[0035] In the related art, the verification of sensitive data leakage is often performed by matching a single type of log data through a predefined rule, or by determining a local segment in a leakage chain through setting a user behavior baseline in a system, and it is difficult to effectively associate and integrate log data of different sources and different time points, so it is difficult to restore a complete leakage chain and provide a reliable basis for subsequent traceability and evidence collection.

[0036] Therefore, the embodiments of the present application provide a data processing method, which acquires a plurality of sets of associated log data, the plurality of sets of associated log data indicating at least one behavior chain of at least one user, the behavior chain including a plurality of behavior nodes, the behavior node representing an operation behavior of the user on business data; based on the plurality of sets of associated log data, determining an abnormal operation behavior in a plurality of operation behaviors of the at least one user and a target behavior node corresponding to the abnormal operation behavior; determining, from the plurality of sets of associated log data, target associated log data related to the abnormal operation behavior; and based on the target associated log data, determining, by using a pre-trained graph neural network model, a plurality of behavior nodes before the target behavior node in the behavior chain in which the target behavior node is located, to obtain an abnormal behavior chain.

[0037] Figure 1 An application scenario diagram of the data processing method and device according to the embodiments of the present application is schematically shown.

[0038] As shown in Figure 1 According to the application scenario 100 of the embodiments, the first terminal device 101, the second terminal device 102, the third terminal device 103, the network 104 and the server 105 are included. The network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0039] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0040] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0041] The server 105 can be a server providing various services, such as a background management server supporting websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as examples). The background management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0042] It should be noted that the data processing method provided by the embodiments of the present application can generally be executed by the server 105. Correspondingly, the data processing apparatus provided by the embodiments of the present application can generally be arranged in the server 105. The data processing method provided by the embodiments of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the data processing apparatus provided by the embodiments of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0043] It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0044] The data processing method according to the embodiments of the present application will be described in detail based on the scenarios described below. Figure 1 Figures 2-5

[0045] Figure 2 The flowchart of the data processing method according to the embodiments of the present application is schematically shown.

[0046] As​​Figure 2 As shown, the data processing method of this embodiment includes operation S210 to operation S240.

[0047] In operation S210, a plurality of sets of associated log data are acquired.

[0048] The plurality of sets of associated log data indicate at least one behavior chain of at least one user, and the behavior chain includes a plurality of behavior nodes, and each behavior node represents an operation behavior of the user on business data.

[0049] It should be noted that before the associated log data is acquired, the consent or authorization of the user is obtained. That is, after the consent or authorization of the user is obtained, operation S210 is performed.

[0050] According to an embodiment of the present application, the above-mentioned associated log data can be log data in a plurality of systems, which are associated through user operation behaviors. For example, the above-mentioned associated log data can include database audit logs, file access logs, network proxy logs, etc.

[0051] For example, for the log data "20XX-7-20 18:00 User A downloaded a piece of data xxxxxxxx" in system A, and the log data "20XX-7-20 18:01 User A transmitted a piece of data xxxxxxxx to User B" in system B, since the above two pieces of log data reflect the operation behaviors of User A within a certain time, the log data corresponding to system A and the log data corresponding to system B can be used as the associated log data in the present application for tracing.

[0052] According to an embodiment of the present application, since the above-mentioned associated log data is collected from different systems, and the log formats in each system are different, the log format can be cleaned after collection, and the associated log data can be converted into a unified log format.

[0053] In operation S220, based on the plurality of sets of associated log data, an abnormal operation behavior in a plurality of operation behaviors of at least one user and a target behavior node corresponding to the abnormal operation behavior are determined.

[0054] According to an embodiment of the present application, the abnormal operation behavior can be determined by a pre-trained classification model, or the abnormal operation behavior can be determined according to statistical data, which is not limited in the present application.

[0055] For example, the system constructed based on the method in the present application can check the associated log data in a recent time window at regular intervals. The aforementioned time window can be 10 minutes, for example. When the system detects that the user checks the sensitive data table at a non-working time, the operation behavior can be determined as an abnormal operation behavior.

[0056] In operation S230, target association log data related to the abnormal operation behavior is determined from the multiple sets of association log data.

[0057] In operation S240, the pre-trained graph neural network model is used to determine, based on the target association log data, multiple behavior nodes in the behavior chain in which the target behavior node is located and which precede the target behavior node, to obtain an abnormal behavior chain.

[0058] Figure 3 An illustrative diagram of constructing an abnormal behavior chain according to an embodiment of the present application is schematically shown.

[0059] As Figure 3 shown, in which the black dashed line in the diagram can represent an adjacency relationship, i.e., the association log data corresponding to the behavior nodes have an association relationship, and the solid line represents an abnormal behavior chain, the abnormal behavior chain in the diagram is behavior node D - behavior node C - behavior node B - behavior node A, and behavior node D is the target behavior node. In the process of determining the abnormal behavior chain, behavior node D can be first input into the pre-trained graph neural network, so as to determine the target predecessor behavior node from the two adjacent behavior nodes behavior node C and behavior node E (for ease of display, only two behavior nodes are shown here, and in the actual process, there can be multiple adjacent behavior nodes). Assuming that the pre-trained graph neural network determines that the target predecessor behavior node is behavior node C, then in the second step, behavior node C is input into the pre-trained graph neural network to determine the target predecessor behavior node of behavior node C. In this way, until the last determined behavior node (here, behavior node A) does not exist a forward adjacency relationship, the abnormal behavior chain is obtained.

[0060] According to embodiments of the present application, by tracing at least one behavior chain of at least one user indicated by the multiple sets of association log data as the trace data, the limitations of single log data are avoided, and then by determining the abnormal operation behavior of at least one user and the target behavior node corresponding to the abnormal operation behavior from the association log data, the abnormal behavior can be accurately located, and then the abnormal behavior chain can be constructed based on the construction basis. Further, by means of the pre-trained graph neural network, the target behavior node is taken as the starting point for forward tracing to obtain the abnormal behavior chain, which improves the accuracy and response efficiency of system security checking, reduces the manual troubleshooting burden of related operation and maintenance personnel, and improves the tracing efficiency.

[0061] According to an embodiment of the present application, the above determining the abnormal operation behavior from the plurality of operation behaviors of the at least one user based on the plurality of sets of associated log data comprises: performing feature extraction on the plurality of sets of associated log data to obtain a sequence of associated features; inputting each associated feature of the sequence of associated features into a pre-trained isolation forest model to obtain an abnormal score group corresponding to the sequence of associated features, the abnormal score group comprising a plurality of abnormal scores sorted according to a time sequence; and determining at least one abnormal behavior based on the abnormal score group.

[0062] The isolation forest is an unsupervised anomaly detection algorithm that isolates abnormal points by dividing the data space, and the core idea is that abnormal samples are more likely to be quickly isolated due to their small quantity and large feature value difference.

[0063] According to an embodiment of the present application, the parameters of the isolation forest model can include the number of random trees, the number of samples for constructing a sub-tree, and the expected abnormal proportion, wherein the number of random trees can be set to 100, the number of samples for constructing a sub-tree can be set to 256, and the expected abnormal proportion can be set to 0.05.

[0064] For example, for a sequence of associated features "A1, A2, A3, A4", inputting the sequence of associated features into the pre-trained isolation forest model obtains an abnormal score group corresponding to the sequence of associated features as "A1: 0.43; A2: 0.24; A3: 0.65; A4: 0.25", and then the operation behavior indicated by the associated log data corresponding to the associated feature A3 can be determined as an abnormal operation behavior according to a preset abnormal threshold (which can be set to 0.65 here).

[0065] According to an embodiment of the present application, the above method further comprises: in a case where a plurality of abnormal scores in the abnormal score group that are continuous in the time dimension all exceed a preset threshold, sending an alarm information to a preset user.

[0066] According to an embodiment of the present application, the preset threshold can be 0.6, for example. For example, when three continuous abnormal scores all exceed 0.6, an alarm information can be sent to a preset user (which can be a system administrator or a related operation and maintenance personnel) in the form of a short message, an email, or the like, and the alarm information can include an abnormal operation user identifier and an abnormal behavior chain.

[0067] According to an embodiment of the present application, when an abnormal score greater than a preset blocking threshold is detected, the corresponding operation behavior of the abnormal operation user can be immediately blocked, and the preset blocking threshold can be 0.8, for example.

[0068] According to the embodiment of the present application, the abnormal score group is determined from the correlation feature sequence determined by the isolation forest model from the multiple sets of correlation log data, and then at least one abnormal behavior is determined. The abnormal behavior can be determined from a large amount of correlation log data without manual annotation, reducing the workload of relevant personnel. The alarm is given when the abnormal scores in multiple time dimensions are continuously above the threshold value, avoiding frequent alarms caused by individual misoperations, improving processing efficiency, and directly blocking the abnormal scores above the blocking threshold to avoid causing greater losses and enhancing the reliability of the system.

[0069] According to the embodiment of the present application, the correlation log data includes domain name data. Based on the multiple sets of correlation log data, the abnormal operation behavior in the multiple operation behaviors of at least one user is determined, including: in the case that the preset list includes domain name data, determining target correlation log data based on the domain name data; and determining the operation behavior indicated by the target correlation log data as an abnormal behavior.

[0070] For example, for a piece of correlation log data "20XX-7-20 18:00 User A uploads data xxxxxxxxx to the website https: / / *.*.*", the system detects that the domain name data "https: / / *.*.*" is in the preset list, and then determines that "20XX-7-20 18:00 User A uploads data xxxxxxxxx to the website https: / / *.*.*" is the target correlation log data, and the corresponding abnormal behavior is that User A uploads data xxxxxxxxx to the website https: / / *.*.*".

[0071] According to the embodiment of the present application, in the case that the preset list includes domain name data, the abnormal behavior is directly determined, and the isolation forest model is combined for abnormal determination, which can determine the abnormal behavior in multiple dimensions and then take corresponding measures to improve the stability of the system.

[0072] According to the embodiment of the present application, the pre-trained graph neural network model is trained by the following method: obtaining sample correlation log data, the sample log correlation data including at least one sample abnormal behavior chain, at least one sample abnormal behavior node of the sample abnormal behavior chain, and a real predecessor node of the sample abnormal behavior node; constructing a sample adjacency matrix based on the sample correlation log data, the sample adjacency matrix representing the correlation relationship between systems corresponding to each sample log in the sample log correlation data; inputting the sample adjacency matrix and the sample abnormal behavior node into the graph neural network model to obtain a sample predecessor node of the sample abnormal behavior node and a predicted value of the sample predecessor node; determining a loss value based on the predicted value of the sample predecessor node and the predicted value of the real predecessor node; updating the graph neural network model using the loss value to obtain the pre-trained graph neural network model.

[0073] For example, assuming that there are three systems, system A, system B and system C, and there is a correlation between the sample logs corresponding to system A and system B and system B and system C, the adjacency matrix constructed can be .

[0074] Figure 4 An architecture diagram of a graph neural network model according to an embodiment of the present application is schematically shown.

[0075] As shown in Figure 4 , the graph neural network model can include a convolution layer 1, an aggregation layer, an activation layer and a convolution layer 2. The graph neural network receives a feature vector of a sample abnormal behavior node at an input layer, and performs neighborhood message passing (neighbor node feature aggregation) through a sample adjacency matrix in the convolution layer 1 and the aggregation layer. Further, the graph neural network performs a nonlinear transformation on the aggregation result in the activation layer, obtains an updated node feature in the convolution layer 2, and generates a sample predecessor node and a predicted value of the sample predecessor node through the updated node feature.

[0076] According to an embodiment of the present application, the loss value can be determined by calculating the binary cross entropy between the predicted value of the sample predecessor node and the predicted value of the real predecessor node.

[0077] The process of neighbor node feature aggregation in the convolution layer 1 can be described by the following formula (1).

[0078] (1)

[0079] wherein h i （l+1） represents a feature vector of the (l+1)th behavior node i, W self represents a self-connection weight matrix, j is all neighbor behavior nodes of the behavior node i, W edge represents an edge weight matrix, c ij represents an edge normalization factor between the neighbor behavior node j and the behavior node i, which is weighted according to the operation type between the behavior nodes.

[0080] According to an embodiment of the present application, a sample adjacency matrix representing the correlation between systems is constructed from a sample correlation log, and the sample adjacency matrix and a sample abnormal behavior node are input into a graph neural network model for training. The loss value is determined based on the predicted value of the real predecessor node and the predicted value of the sample predecessor node output by the graph neural network model to forward propagate and adjust the graph neural network model, so as to train a pre-trained graph neural network model capable of accurately restoring an abnormal behavior chain, which can effectively reduce the workload of relevant personnel and improve the efficiency of tracing.

[0081] According to the embodiment of the present application, the sample correlation log comprises a session identifier, a user identifier and an operation time. The sample adjacency matrix is constructed based on the sample correlation log data, comprising: determining a candidate correlation log group related to the user identifier from the sample correlation log based on the user identifier; filtering the candidate correlation log group by using a preset time interval to determine a first log correlation relationship between the candidate correlation log groups, wherein the first log correlation relationship represents that the operation time interval of each two candidate logs is less than the preset time interval; determining a second log correlation relationship between the sample correlation logs based on the session identifier; and constructing the adjacency matrix based on the first log correlation relationship and the second log correlation relationship.

[0082] According to the embodiment of the present application, the preset time interval may be 5 minutes, for example.

[0083] For example, there are multiple pieces of sample correlation log data (hereinafter referred to as log data for convenience of description). Log data 1: “20XX-7-20 18:00; userid: 001; xxxxxxxxxxxx”; log data 2: “20XX-7-20 18:02; userid: 001; xxxxxxxxxxxx; sessionid: 01A”; log data 3: “20XX-7-20 18:02; userid: 002; xxxxxxxxxxxx”; and log data 4: “20XX-7-20 18:08; userid: 001; xxxxxxxxxxxx; sessionid: 01A”. First, the candidate log correlation group can be determined as the log data 1, the log data 2 and the log data 4 according to the user identifier. Since the operation time interval of the log data 1 and the log data 2 is less than the preset time interval (set to 5 minutes), it can be determined that the log data 1 and the log data 2 have a first log correlation relationship. Since the session identifiers of the log data 2 and the log data 4 are consistent, it can be determined that the log data 2 and the log data 4 have a second log correlation relationship. Further, the correlation relationship between the systems corresponding to the logs can be determined according to the first log correlation relationship and the second log correlation relationship, and then the adjacency matrix is constructed.

[0084] According to statistics, compared with the traceability method in the prior art, the data processing method proposed in the present application improves the timeliness of event analysis by about 24 times, reduces the false positive rate by at least 70%, reduces the operation and maintenance cost by 90%, and at the same time constructs an end-to-end complete abnormal behavior chain to achieve full coverage of the evidence chain.

[0085] Figure 5 An architecture diagram of a system constructed by the data processing method according to the embodiment of the present application is schematically shown.

[0086] As Figure 5As shown, the log collection layer collects logs from different systems, cleans and standardizes them, and then transmits them to the intelligent analysis layer. The intelligent analysis layer associates logs based on session identifiers, judges abnormal behavior through the isolated forest model, and uses a pre-trained graph neural network to trace the source to determine the abnormal behavior chain and sends it to the source tracing display layer, graph storage layer, and alarm engine. The graph storage layer stores the relationships between systems, the alarm engine sends alarms to preset users via email, SMS, etc., and the source tracing display layer displays the abnormal behavior chain and the abnormal scores of each operation.

[0087] Based on the above data processing method, this application also provides a data processing apparatus. The following will be combined with... Figure 6 The device is described in detail.

[0088] Figure 6 A schematic block diagram of a data processing apparatus according to an embodiment of this application is shown.

[0089] like Figure 6 As shown, the data processing device 600 in this embodiment includes an acquisition module 610, an abnormal operation behavior determination module 620, a target associated log data determination module 630, and an abnormal behavior chain determination module 640.

[0090] The acquisition module 610 is used to acquire multiple sets of associated log data. These multiple sets of associated log data indicate at least one behavior chain for at least one user. The behavior chain includes multiple behavior nodes, and each behavior node represents a user's operation behavior on business data. In one embodiment, the acquisition module 610 can be used to execute the operation S210 described above, which will not be repeated here.

[0091] The abnormal operation behavior determination module 620 is used to determine, based on multiple sets of associated log data, abnormal operation behaviors among multiple operation behaviors of at least one user, as well as the target behavior nodes corresponding to the abnormal operation behaviors. In one embodiment, the abnormal operation behavior determination module 620 can be used to execute the operation S220 described above, which will not be repeated here.

[0092] The target associated log data determination module 630 is used to determine the target associated log data related to the abnormal operation behavior from multiple sets of associated log data. In one embodiment, the target associated log data determination module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0093] The abnormal behavior chain determination module 640 is configured to determine, based on the target association log data, a plurality of behavior nodes before the target behavior node in a behavior chain in which the target behavior node is located, by using the pre-trained graph neural network model, to obtain an abnormal behavior chain. In an embodiment, the abnormal behavior chain determination module 640 can be configured to perform the operation S240 described above, and details are not repeated here.

[0094] According to an embodiment of the present application, the abnormal operation behavior determination module comprises: a feature extraction submodule configured to perform feature extraction on the plurality of sets of association log data to obtain a sequence of association features; an abnormal score group determination submodule configured to input each association feature of the sequence of association features into a pre-trained isolation forest model to obtain an abnormal score group corresponding to the sequence of association features, the abnormal score group comprising a plurality of sets of abnormal scores sorted according to a time sequence; and a first abnormal behavior determination submodule configured to determine the at least one abnormal behavior based on the abnormal score group.

[0095] According to an embodiment of the present application, the association log data comprises domain name data, and the abnormal operation behavior determination module comprises: a target association log data determination submodule configured to determine the target association log data based on the domain name data in a case where the preset list comprises the domain name data; and a second abnormal behavior determination submodule configured to determine the operation behavior indicated by the target association log data as an abnormal behavior.

[0096] According to an embodiment of the present application, the device further comprises an alarm module configured to send alarm information to a preset user in a case where a plurality of sets of abnormal scores that are continuous in the time dimension in the abnormal score group all exceed a preset threshold.

[0097] According to an embodiment of the present application, the device further comprises: a sample acquisition module configured to acquire sample association log data, the sample log association data comprising at least one sample abnormal behavior chain, a sample abnormal behavior node of the at least one sample abnormal behavior chain, and a real predecessor node of the sample abnormal behavior node; an adjacency matrix construction module configured to construct a sample adjacency matrix based on the sample association log data, the sample adjacency matrix representing an association relationship between systems corresponding to each sample log in the sample log association data; a predicted value determination module configured to input the sample adjacency matrix and the sample abnormal behavior node into a graph neural network model to obtain a sample predecessor node of the sample abnormal behavior node and a predicted value of the sample predecessor node; a loss value determination module configured to determine a loss value based on the predicted value of the sample predecessor node and the predicted value of the real predecessor node; and an update module configured to update the graph neural network model by using the loss value to obtain the pre-trained graph neural network model.

[0098] According to an embodiment of the present application, the sample correlation log comprises a session identifier, a user identifier and an operation time, and the adjacency matrix construction module comprises: a candidate correlation log group determination submodule, configured to determine a candidate correlation log group related to the user identifier from the sample correlation log based on the user identifier; a first log correlation relationship determination submodule, configured to filter the candidate correlation log group by using a preset time interval to determine a first log correlation relationship between the candidate correlation log group, the first log correlation relationship representing that an operation time interval of each two candidate logs is less than the preset time interval; a second log correlation relationship determination submodule, configured to determine a second log correlation relationship between the sample correlation logs based on the session identifier; and a matrix construction submodule, configured to construct the adjacency matrix based on the first log correlation relationship and the second log correlation relationship.

[0099] According to an embodiment of the present application, any multiple modules of the acquisition module 610, the abnormal operation behavior determination module 620, the target correlation log data determination module 630 and the abnormal behavior chain determination module 640 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the acquisition module 610, the abnormal operation behavior determination module 620, the target correlation log data determination module 630 and the abnormal behavior chain determination module 640 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging a circuit, etc. hardware or firmware, or implemented in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 610, the abnormal operation behavior determination module 620, the target correlation log data determination module 630 and the abnormal behavior chain determination module 640 can be at least partially implemented as a computer program module which can perform corresponding functions when the computer program module is run.

[0100] Figure 7 A block diagram of an electronic device suitable for implementing the data processing method according to an embodiment of the present application is schematically shown.

[0101] As Figure 7As shown, the electronic device 700 according to embodiments of the present application includes a processor 701 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 702 or a program loaded into a random access memory (RAM) 703 from a storage section 708. The processor 701 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chip set, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 701 can also include an on-board memory for cache use. The processor 701 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present application.

[0102] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations of the method processes according to embodiments of the present application by executing the programs in the ROM 702 and / or the RAM 703. Note that the programs can also be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 can also perform various operations of the method processes according to embodiments of the present application by executing the programs stored in the one or more memories.

[0103] According to embodiments of the present application, the electronic device 700 can also include an input / output (I / O) interface 705 which is also connected to the bus 704. The electronic device 700 can also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as necessary. A removable medium 711 such as a magnetic disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 710 as necessary, so that a computer program read out therefrom is installed in the storage section 708 as necessary.

[0104] The application further provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist independently without being assembled into the device / apparatus / system. The computer readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the application.

[0105] According to the embodiments of the application, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In this application, a computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to the embodiments of the application, the computer readable storage medium can include the ROM 702 and / or the RAM 703 described above, and / or one or more memory other than the ROM 702 and the RAM 703.

[0106] The embodiments of the application also include a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the data processing method provided by the embodiments of the application.

[0107] The above functions defined in the system / apparatus of the embodiments of the application are performed when the computer program is executed by the processor 701. According to the embodiments of the application, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0108] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium, and be downloaded and installed through the communication part 709, and / or installed from the detachable medium 711. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0109] In such embodiments, the computer program can be downloaded and installed from the network via the communication section 709, and / or installed from the removable media 711. When the computer program is executed by the processor 701, the above-described functions defined in the system of the embodiments of the present application are executed. According to the embodiments of the present application, the system, device, apparatus, module, unit, and the like described above can be realized by the computer program module.

[0110] According to the embodiments of the present application, the program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. The programming language includes, but is not limited to, such as Java, C++, python, "C" language, or similar programming language. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0111] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0112] Those skilled in the art can understand that the features described in various embodiments of the present application can be combined and / or integrated in various combinations and / or integrations, even if such combinations or integrations are not explicitly described in the present application. In particular, the features described in various embodiments of the present application can be combined and / or integrated in various combinations and / or integrations without departing from the spirit and teachings of the present application. All such combinations and / or integrations fall within the scope of the present application.

Claims

1. A data processing method, characterized by, The method comprises: obtaining a plurality of sets of associated log data, the plurality of sets of associated log data indicating at least one behavior chain of at least one user, the behavior chain comprising a plurality of behavior nodes, the behavior nodes representing operation behaviors of the user on business data; determining, based on the plurality of sets of associated log data, an abnormal operation behavior in the plurality of operation behaviors of the at least one user and a target behavior node corresponding to the abnormal operation behavior; determining, from the plurality of sets of associated log data, target associated log data related to the abnormal operation behavior; determining, based on the target associated log data, a plurality of behavior nodes before the target behavior node in the behavior chain in which the target behavior node is located, by using a pre-trained graph neural network model, to obtain an abnormal behavior chain.

2. The method of claim 1, wherein, The determination of the abnormal operation behavior in the plurality of operation behaviors of the at least one user based on the plurality of sets of associated log data comprises: performing feature extraction on the plurality of sets of associated log data to obtain a sequence of associated features; inputting each associated feature of the sequence of associated features into a pre-trained isolation forest model to obtain a group of abnormal scores corresponding to the sequence of associated features, the group of abnormal scores comprising a plurality of groups of abnormal scores sorted according to time sequence; determining the at least one abnormal behavior based on the group of abnormal scores.

3. The method according to any one of claims 1 or 2, characterized in that, The associated log data comprises domain name data, and the determination of the abnormal operation behavior in the plurality of operation behaviors of the at least one user based on the plurality of sets of associated log data comprises: in a case where a preset list comprises the domain name data, determining the target associated log data based on the domain name data; determining the operation behavior indicated by the target associated log data as an abnormal behavior.

4. The method of claim 2, wherein, The method further comprises: in a case where a plurality of groups of abnormal scores that are continuous in the time dimension in the group of abnormal scores all exceed a preset threshold, sending alarm information to a preset user.

5. The method of claim 1, wherein, The pre-trained graph neural network model is obtained by the following method: obtaining sample associated log data, the sample log associated data comprising at least one sample abnormal behavior chain, sample abnormal behavior nodes of the at least one sample abnormal behavior chain, and real predecessor nodes of the sample abnormal behavior nodes; constructing a sample adjacency matrix based on the sample associated log data, the sample adjacency matrix representing the association relationship between systems corresponding to each sample log in the sample log associated data; inputting the sample adjacency matrix and the sample abnormal behavior nodes into a graph neural network model to obtain sample predecessor nodes of the sample abnormal behavior nodes and predicted values of the sample predecessor nodes; determining a loss value based on the predicted values of the sample predecessor nodes and the predicted values of the real predecessor nodes; updating the graph neural network model by using the loss value to obtain the pre-trained graph neural network model.

6. The method of claim 5, wherein, The sample associated log comprises a session identifier, a user identifier, and an operation time, and the construction of the sample adjacency matrix based on the sample associated log data comprises: determining, based on the user identifier, a candidate associated log group related to the user identifier from the sample associated log; The candidate associated log groups are filtered by using a preset time interval to determine a first log association relationship between the candidate associated log groups, the first log association relationship representing that an operation time interval of each two candidate logs is less than the preset time interval; determine a second log association relationship between the sample associated logs based on the session identifier; construct the adjacency matrix based on the first log association relationship and the second log association relationship.

7. A data processing apparatus, characterized by The device comprises: An acquisition module is configured to acquire a plurality of sets of associated log data, the plurality of sets of associated log data indicating at least one behavior chain of at least one user, the behavior chain comprising a plurality of behavior nodes, the behavior nodes representing operation behaviors of the user on service data; An abnormal operation behavior determination module is configured to determine, based on the plurality of sets of associated log data, an abnormal operation behavior in a plurality of operation behaviors of the at least one user and a target behavior node corresponding to the abnormal operation behavior; A target associated log data determination module is configured to determine, from the plurality of sets of associated log data, target associated log data related to the abnormal operation behavior; An abnormal behavior chain determination module is configured to determine, based on the target associated log data, a plurality of behavior nodes before the target behavior node in a behavior chain in which the target behavior node is located, by using a pre-trained graph neural network model, to obtain an abnormal behavior chain.

8. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instruction is executed by the processor to implement the steps of the method according to any one of claims 1-6.

10. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instruction is executed by the processor to implement the steps of the method according to any one of claims 1-6. The computer program or instruction is executed by the processor to implement the steps of the method according to any one of claims 1-6.

Images