A data traceability method, apparatus, equipment, medium, and product
By processing operation log data through graph convolutional neural networks and constructing a relation matrix, the problem of inaccurate data tracing caused by incomplete knowledge graphs is solved, and efficient and accurate data tracing is achieved.
Patent Information
- Application Number
- CN202510601440.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2026-11-10
AI Technical Summary
In existing technologies, knowledge graphs may be incomplete, resulting in incomplete and inaccurate data tracing results, difficulties in disambiguating feature entities, and an inability to accurately identify the meaning of features, thus affecting the accuracy of data tracing.
A graph convolutional neural network is used to process operation log data, construct a feature matrix and an adjacency matrix, and perform operations through the graph convolutional neural network to generate a relation matrix, thereby enabling data traceability.
It improves the accuracy and completeness of data traceability, enabling accurate identification of features and relationships, and ensuring the integrity of the data chain.
Smart Images

Figure CN123069359A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of big data technology, and in particular relates to a data traceability method, apparatus, equipment, medium and product. Background Technology
[0002] The demand for data security and privacy protection in the field of information security is constantly growing. With the increasing level of informatization, massive amounts of data are generated, stored, transmitted, and processed, potentially containing sensitive information such as important business secrets and personal privacy. However, data is often difficult to effectively track, monitor, and trace during its flow, leading to frequent data leaks, tampering, and misuse. Therefore, data tracing is a crucial means of ensuring data security. Currently, knowledge graphs are often used to reconstruct data history based on tracing paths to achieve tracing, utilizing the tagging information in data packets to determine the path forwarded by the data packets during network transmission. However, knowledge graphs may not be complete and may not contain all feature entities and relationship information, resulting in incomplete and inaccurate tracing results. Furthermore, feature entity disambiguation is difficult; when feature entities have similar characteristics or are ambiguous, entity extraction techniques may fail to accurately identify the meaning of the features, further reducing the accuracy of tracing results. Summary of the Invention
[0003] This application provides a data traceability method, apparatus, device, medium, and product that can process operation log data based on graph convolutional neural networks to obtain a relationship matrix, thereby achieving accurate traceability.
[0004] On the one hand, embodiments of this application provide a data tracing method, including:
[0005] Retrieve operation log data;
[0006] Extract multiple features and the relationships between these features from the operation log data; the features include events and event attributes.
[0007] A feature matrix is constructed based on the multiple features, and an adjacency matrix is constructed based on the association relationships between the multiple features;
[0008] Based on a graph convolutional neural network, the adjacency matrix and the feature matrix are processed to obtain a relation matrix that represents the multiple features and the relationships between the multiple features.
[0009] Based on the relationship matrix, data tracing is performed on the first target feature to obtain a second target feature that is related to the first target feature.
[0010] On the other hand, the graph convolutional neural network-based operation on the adjacency matrix and the feature matrix to obtain a relation matrix representing the multiple features and the relationships between them includes:
[0011] For each layer of the graph convolutional neural network, perform the following operations:
[0012] Obtain the target matrix corresponding to the current layer of the network; for the first layer of the graph convolutional neural network, the target matrix is the feature matrix; for each subsequent layer of the graph convolutional neural network, the target matrix is determined based on the output matrix of the previous layer.
[0013] Based on the adjacency matrix and the target matrix corresponding to the current layer network, the input matrix of the current layer network is determined;
[0014] Based on the set function of the current layer network, the input matrix of the current layer network is processed to obtain the output matrix of the current layer network; the output matrix of the last layer network of the graph convolutional neural network is the relation matrix.
[0015] On the other hand, for each subsequent layer of the graph convolutional neural network, before determining the input matrix of the current layer based on the adjacency matrix and the target matrix corresponding to the current layer, the method further includes:
[0016] Based on the reversibility adjustment factor and the first output matrix of other layers preceding the current layer, the target matrix corresponding to the current layer is determined so that the operation performed by the current layer is reversible; the first output matrix includes at least the output matrix of the previous layer.
[0017] On the other hand, obtaining the target matrix corresponding to the current layer network includes:
[0018] Obtain the degree matrix of the multiple features;
[0019] Correspondingly, based on the adjacency matrix and the target matrix corresponding to the current layer network, the input matrix of the current layer network is determined, including:
[0020] The adjacency matrix and the target matrix are processed based on the degree matrix;
[0021] Based on the processed adjacency matrix and target matrix, the input matrix of the current layer network is determined.
[0022] On the other hand, before determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network, the method further includes:
[0023] Determine the weight matrix corresponding to each layer of the graph convolutional neural network;
[0024] Correspondingly, based on the adjacency matrix and the target matrix corresponding to the current layer network, the input matrix of the current layer network is determined, including:
[0025] The input matrix of the current layer network is determined based on the weight matrix, the adjacency matrix, and the target matrix.
[0026] On the other hand, based on the association relationships between the multiple features, an adjacency matrix is constructed, including:
[0027] Based on the relationships between these features, an original adjacency matrix is constructed;
[0028] The adjacency matrix is obtained by adding the original adjacency matrix and the identity matrix.
[0029] On the other hand, before extracting multiple features from the operation log data and the correlations between these features, the method further includes:
[0030] Retrieve multiple feature matching fields used for feature matching;
[0031] Based on the aforementioned multiple feature matching fields, a data parsing template library is constructed;
[0032] Correspondingly, multiple features and the relationships between these features are extracted from the operation log data, including:
[0033] Based on the multiple feature matching fields in the data parsing template library, multiple target fields are matched in the operation log data;
[0034] Based on the multiple target fields, the multiple features are determined respectively; and based on the position of the target fields in the operation log data, the association relationship between the multiple features is determined.
[0035] On the other hand, based on the multiple feature matching fields in the data parsing template library, multiple target fields are matched in the operation log data, including:
[0036] Based on the feature matching fields, construct a regular expression;
[0037] Based on the regular expression, the multiple target fields are matched in the operation log data.
[0038] On the other hand, this application also provides a data traceability device, the device comprising:
[0039] The acquisition module is used to acquire operation log data;
[0040] The extraction module is used to extract multiple features from the operation log data and the relationships between these features; the features include events and event attributes.
[0041] A construction module is used to construct a feature matrix based on the multiple features, and to construct an adjacency matrix based on the association relationships between the multiple features;
[0042] The computation module is used to perform operations on the adjacency matrix and the feature matrix sequentially based on a graph convolutional neural network to obtain a relation matrix that represents the multiple features and the association relationships between the multiple features.
[0043] The tracing module is used to trace the data of the first target feature based on the relationship matrix to obtain the second target feature that is related to the first target feature.
[0044] On the other hand, this application also provides a data traceability device, the device comprising: a processor and a memory storing computer program instructions;
[0045] When the processor executes the computer program instructions, it implements the data tracing method described above.
[0046] On the other hand, this application also provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the data tracing method described above.
[0047] On the other hand, this application also provides a computer program product in which the instructions, when executed by the processor of an electronic device, cause the electronic device to perform the data tracing method described above.
[0048] The data tracing method in this application employs a graph convolutional neural network (GNN) for data tracing. Specifically, this application extracts multiple features and their relationships from operation log data; then constructs a feature matrix and an adjacency matrix (i.e., graph data); and processes the graph data using a GNN to construct a relation matrix that represents multiple features and their relationships. Based on this, data tracing can be performed on a first target feature using the relation matrix to obtain a second target feature that is related to the first target feature. It is evident that this application, by constructing graph data and transforming it into a corresponding relation matrix using a GNN, leverages the strong expressive power of the GNN to ensure data integrity and improve tracing accuracy. Furthermore, by constructing graph data by converting features and relationships into nodes and edges, this application can accurately identify all features, further improving the accuracy of the tracing results. Attached Figure Description
[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 A flowchart illustrating a data tracing method provided in one embodiment of this application is shown;
[0051] Figure 2 A flowchart illustrating a data tracing method provided in another embodiment of this application is shown;
[0052] Figure 3 A flowchart illustrating a data tracing method provided in another embodiment of this application is shown;
[0053] Figure 4 A flowchart illustrating a data tracing method provided in another embodiment of this application is shown;
[0054] Figure 5 This is a schematic diagram of the structure of a data traceability device provided in another embodiment of this application;
[0055] Figure 6 A schematic diagram of the data traceability device provided in an embodiment of this application is shown. Detailed Implementation
[0056] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.
[0057] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0058] The demand for data security and privacy protection in the field of information security is constantly growing. With the increasing level of informatization, massive amounts of data are generated, stored, transmitted, and processed, potentially containing important trade secrets, personal privacy, and other sensitive information. However, data is often difficult to effectively track, monitor, and trace during its flow, leading to frequent data leaks, tampering, and misuse. Traditional security hardware products (such as firewalls) and data loss prevention systems cannot effectively monitor and manage the dynamic dissemination of sensitive corporate data, lack continuous management methods for security incidents before, during, and after they occur. This makes it difficult for enterprises to predict potential data security risks in advance, obtain complete security incident information in a timely manner, lack security incident response capabilities, and accurately locate and trace the source of incidents after they happen. Data security incidents involving confidential or sensitive information are closely integrated with business operations and can easily cause significant losses to individuals and organizations, impacting organizational business development.
[0059] Currently, methods such as knowledge graphs are often used to reconstruct historical data based on tracing paths to achieve source tracing, utilizing the tagging information in data packets to determine the path the data packets were forwarded during network transmission. However, knowledge graphs may not be complete; for example, they may not contain all feature entities and relationship information, leading to incomplete and inaccurate source tracing results. Furthermore, this method faces difficulties in entity disambiguation; when feature entities have similar characteristics or exhibit ambiguity, entity extraction techniques may fail to accurately identify the meaning of the features, further reducing the accuracy of the source tracing results.
[0060] Therefore, this application proposes a novel data tracing method. Considering the strong expressive power of graph convolutional neural networks (GCNNs) and their ability to effectively guarantee data integrity, this application utilizes GCNNs to preserve the relationship links between data, thereby achieving fast and accurate tracing. Because GCNNs can learn the feature representations of nodes and edges in a graph, they can model the structure of the graph and the relationships between nodes, making them excellent at tasks such as classification, clustering, and prediction of graph data. Therefore, this application uses GCNNs to construct a relationship matrix representing multiple features and the relationships between them, thereby achieving accurate tracing.
[0061] Based on this, embodiments of this application provide a data traceability method, apparatus, device, computer storage medium, and computer program product. The data traceability method provided in the embodiments of this application will be described first below. Figure 1 A flowchart illustrating a data tracing method provided in one embodiment of this application is shown. Figure 1 As shown, the method includes the following steps: S101 to S107.
[0062] S101: Get operation log data.
[0063] This application does not limit the specific content of the operation log data or the specific method of obtaining the operation log data. As some optional implementation methods, operation log data generated by the system, network, application, etc. can be collected using various methods, such as File Transfer Protocol (FTP) / Secure Shell File Transfer Protocol (SFTP), standard logging protocol SYSLOG, Transmission Control Protocol (TCP), Open Database Connectivity (ODBC) / Java Database Connectivity (JDBC), Simple Object Access Protocol (SOAP), FlumeAPI, etc.
[0064] S102: Extract multiple features from the operation log data and the relationships between these features.
[0065] Furthermore, it is necessary to extract multiple features from the operation log data and the relationships between these features. The specific extraction method can be selected according to the actual situation. As some optional embodiments, a data parsing template library can be constructed based on the fields corresponding to the features to be extracted. Based on the feature matching fields in this data parsing template library, the corresponding features can be matched from the operation log data. At the same time, during the feature matching process, the relationships between the features can also be obtained. For example, multiple features extracted from the same log segment can be considered to have a relationship.
[0066] The features in this application include events and event attributes. Events can typically include actions such as logging in or out of a system, application, or webpage, while event attributes can include the object to which the action is performed, the subject of the action, the execution time, and so on.
[0067] S103: Construct a feature matrix based on multiple features, and construct an adjacency matrix based on the association relationships between multiple features.
[0068] This involves constructing graph data by using features as nodes and the relationships between features as edges connecting the nodes. Based on the aforementioned data parsing template library, structured data information can be obtained, which further needs to be processed to extract features and relationships. Since graph convolutional neural networks need to process graph data, this application requires constructing graph data based on operation log data, i.e., extracting features and the relationships between features; and then constructing graph data through nodes and edges to facilitate graph data processing by the graph convolutional neural network.
[0069] Specifically, as some optional implementation methods, an n-dimensional network graph G(V,E,D) can be formed based on structured data, where V is the set of nodes forming features, E is the edge formed by the relationships between nodes, D is the n-dimensional feature relationship, and n is the representation of the event dimension of the network space nodes (the value of n depends on the size of the data), ultimately forming a V×V×D 3-dimensional matrix M. G In some optional embodiments, the login username, destination resource address, operation data content, etc. can be represented as feature nodes. If the value of the corresponding position in the matrix is 1, it indicates that there is a relationship between the nodes. If the value of the corresponding position in the matrix is 0, it indicates that there is no relationship between the nodes.
[0070] For the n-dimensional graph data G = (V, E), where V and E represent the sets of nodes and edges, respectively, the feature representation of each node forms an n×d matrix X, where n represents the number of nodes and d represents the feature dimension of each node. Simultaneously, the adjacency matrix of the graph data is an n×n matrix A used to represent the connections between nodes. After constructing the feature matrix and adjacency matrix, the adjacency matrix and feature matrix are then processed sequentially using the layers of a graph convolutional neural network.
[0071] S104: Based on graph convolutional neural networks, adjacency matrices and feature matrices are processed to obtain a relation matrix that represents multiple features and the relationships between them.
[0072] Graph convolutional neural networks typically use adjacency matrices to represent the connections between nodes in a graph, and feature matrices to represent the features of each node. The core of convolution operations on a graph is aggregating the features of a node and its neighbors. For each node in the graph, its own features and the features of its neighbors are weighted and summed to obtain new node features. This weighted summation process can be seen as information propagation, enabling nodes to acquire information from their neighbors. The output of each layer in a graph convolutional neural network is used to determine the input of the next layer. Through multi-layer information propagation and transformation, nodes can capture information from further distances in the graph.
[0073] Based on this, this application uses the set functions corresponding to each layer of the graph convolutional neural network to perform corresponding operations on the adjacency matrix and feature matrix in sequence. Through multi-layer information propagation and transformation, a relation matrix is finally obtained to represent multiple features and the correlation between multiple features.
[0074] Ultimately, the relation matrix will store multiple different relation links, that is, multiple features that are related form a relation link; for example, if an event occurs: user A logs into system B at time X, then user A, time X, login, system B, etc. are related features, and there is a relationship between them. The relation matrix will store this relation link.
[0075] S105: Based on the relationship matrix, perform data tracing on the first target feature to obtain the second target feature that is related to the first target feature.
[0076] The data tracing method provided in this application first structures the data, then constructs a graph based on the structured data to form graph data; then it can represent the graph data based on a graph convolutional neural network to obtain connection information; finally, it can use the graph convolutional neural network to trace the data.
[0077] Specifically, assuming it's necessary to trace the source of a first target feature, then based on the relationship matrix, it's necessary to match a second target feature that is related to the first target feature. For example, if the first target feature is "User A, login event, system B", then the relevant relationship links can be quickly and accurately matched in the relationship matrix. That is, all relevant relationship links of User A logging into system B can be retrieved, thereby generating tracing results to complete data tracing.
[0078] Furthermore, in practical applications, a reversible module can be added to the graph convolutional neural network to form a "reversible histogram convolutional neural network," which has stronger expressive power. It can achieve high accuracy not only during the pre-training stage but also ensure no information loss, resulting in better performance in downstream tasks. In addition, the "reversible histogram convolutional neural network" provided in this application has higher memory and backpropagation efficiency. It can completely recover data during both forward and backward propagation without the need for additional storage of intermediate results, thus exhibiting high memory efficiency. During backpropagation, it eliminates the need to save intermediate gradient information, reducing computational and storage burdens and improving backpropagation efficiency.
[0079] The data tracing method in this application employs a graph convolutional neural network (GNN) for data tracing. Specifically, this application extracts multiple features and their relationships from operation log data; then constructs a feature matrix and an adjacency matrix (i.e., graph data); and processes the graph data using a GNN to construct a relation matrix that represents multiple features and their relationships. Based on this, data tracing can be performed on a first target feature using the relation matrix to obtain a second target feature that is related to the first target feature. It is evident that this application, by constructing graph data and transforming it into a corresponding relation matrix using a GNN, leverages the strong expressive power of the GNN to ensure data integrity and improve tracing accuracy. Furthermore, by constructing graph data by converting features and relationships into nodes and edges, this application can accurately identify all features, further improving the accuracy of the tracing results.
[0080] As mentioned above, the output of each layer in a graph convolutional neural network is used to determine the input of the next layer. Through multi-layer information propagation and transformation, nodes can capture information from further distances within the graph. Based on this, this application provides a specific implementation method. Figure 2 A flowchart illustrating step S105 of one embodiment of this application is shown. Figure 2 As shown, S105 may include: performing S1051 and S1052 for each layer of the graph convolutional neural network.
[0081] S1051: Determine the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network.
[0082] Before determining the input matrix, the target matrix of the current layer of the network needs to be obtained. For the first layer of a graph convolutional neural network (Graph Convolutional Neural Network), the target matrix is the feature matrix; that is, the first layer processes the original feature matrix. However, to complete information propagation, for each subsequent layer of the Graph Convolutional Neural Network, the target matrix is determined based on the output matrix of the previous layer; that is, each subsequent layer processes the output of the previous layer.
[0083] As mentioned above, to improve the network's memory and backpropagation efficiency, the concept of a reversible neural network can be introduced into the graph convolutional neural network, making the graph convolutional neural network provided in this application reversible during forward and backward propagation. As an optional implementation, for each layer of the graph convolutional neural network after the first layer, the following operations are performed: based on the reversible adjustment factor and the first output matrix of the other networks preceding the current layer, the target matrix corresponding to the current layer is determined, so that the operations performed by the current layer are reversible; the first output matrix includes at least the output matrix of the previous layer.
[0084] The core of reversible neural networks is the use of reversible transformations to construct network layers. These transformations map the input data to the output space during forward propagation, and accurately reconstruct the input from the output during backward propagation, without information loss. Due to their reversible nature, reversible neural networks do not need to store a large number of intermediate activation values for backpropagation calculations during training, unlike traditional neural networks. This allows them to significantly reduce memory usage and improve training efficiency when dealing with large-scale data or deep networks.
[0085] This embodiment incorporates a reversible adjustment factor to achieve the reversibility of the graph convolutional neural network, resulting in higher memory and backpropagation efficiency. It enables complete data recovery during both forward and backward propagation without the need for additional storage of intermediate results, thus exhibiting high memory efficiency. Furthermore, the absence of the need to save intermediate gradient information during backpropagation reduces computational and storage burdens, further improving backpropagation efficiency.
[0086] S1052: Based on the set function of the current layer network, process the input matrix of the current layer network to obtain the output matrix of the current layer network; the output matrix of the last layer network of the graph convolutional neural network is the relation matrix.
[0087] Each layer of a graph convolutional neural network has a corresponding set function. In each layer, the input matrix is processed based on the corresponding set function to complete the information propagation of the current layer.
[0088] This application processes the adjacency matrix and feature matrix sequentially through each layer of a graph convolutional neural network, thereby efficiently obtaining the final relation matrix. Furthermore, the output of each layer of the graph convolutional neural network is used to determine the input of the next layer. Through multi-layer information propagation and transformation, nodes can capture information from further distances in the graph.
[0089] To avoid altering the magnitude of features during aggregation, this application provides a specific implementation scheme. Before determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network, the degree matrices of multiple features can be obtained. Correspondingly, determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network includes: processing the adjacency matrix and the target matrix based on the degree matrix; and then determining the input matrix of the current layer network based on the processed adjacency matrix and the target matrix.
[0090] A degree matrix can concisely describe the connectivity of each node in a network. By examining the diagonal elements of the degree matrix, the degree of each node can be directly determined, allowing for the analysis of the network's local and global structural characteristics.
[0091] Updating node features by summing neighbor nodes and the node itself has problems. This aggregation method may cause changes in the magnitude of features, which will increase with the number of iterations. Therefore, this application adopts an averaging method, that is, adding a degree matrix during the calculation to ensure that the magnitude of features does not change during the aggregation process.
[0092] To better obtain the final expression of the neural network, this application can determine the weight matrix corresponding to each layer of the graph convolutional neural network before determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network; then, when determining the input matrix, the input matrix of the current layer network can be determined based on the weight matrix, adjacency matrix and target matrix.
[0093] Self-attention mechanisms enable models to automatically learn long-term dependencies in text or images. To better obtain the final expression of the neural network, this application uses an attention mechanism to model the importance of each layer of embedding, that is, to process the input of each layer of the network based on the weight matrix to obtain more accurate output results.
[0094] To ensure that each node retains its uniqueness when updating features, this application proposes to incorporate an identity matrix when constructing the adjacency matrix. Specifically, Figure 3A flowchart illustrating S104 according to an embodiment of this application is shown. Figure 3 As shown, S104 may include S1041 and S1042.
[0095] S1041: Construct the original adjacency matrix based on the association relationships between multiple features.
[0096] The original adjacency matrix is directly determined based on the relationships between multiple features. For a graph G = (V, E) with n nodes, where V is the set of nodes and E is the set of edges, its adjacency matrix A is an n×n matrix.
[0097] S1042: Add the original adjacency matrix and the identity matrix to obtain the adjacency matrix.
[0098] An identity matrix is a special type of square matrix. It is a square matrix where all elements along its main diagonal (from the top left to the bottom right corner) are 1, and all other elements are 0. The order of the identity matrix is the same as the original adjacency matrix; adding the two together yields the adjacency matrix.
[0099] To avoid different features resulting in the same feature vector after adjacency matrix operations, when calculating new node features, in addition to considering the information of neighboring nodes, the information of the node itself also needs to be taken into account. Therefore, this application adds an identity matrix to the adjacency matrix to ensure that each node retains its uniqueness when updating features.
[0100] To achieve full-chain data mapping, it is necessary to draw the entire path of an event from start to finish, thereby enabling better data tracing. Therefore, in this embodiment, before extracting multiple features from the operation log data and the relationships between these features, multiple feature matching fields for matching features are obtained; then, a data parsing template library is constructed based on these multiple feature matching fields.
[0101] Correspondingly, Figure 4 A flowchart illustrating S102 according to an embodiment of this application is shown. Figure 4 As shown, S102 may include S1021 and S1022.
[0102] S1021: Based on multiple feature matching fields in the data parsing template library, multiple target fields are matched in the operation log data.
[0103] In practical applications, regular expression matching can be used. This involves matching multiple target fields in the operation log data based on multiple feature matching fields from the data parsing template library. This includes: constructing a regular expression based on the feature matching fields; and then matching multiple target fields in the operation log data based on the regular expression.
[0104] Regular expression matching can accurately locate and extract specific information from text. It can adapt to diverse text matching scenarios. Furthermore, when processing large-scale data, regular expressions are highly efficient, capable of scanning and matching large amounts of data in a short time to quickly find content that meets the criteria.
[0105] S1022: Based on multiple target fields, determine multiple features respectively; and based on the position of the target fields in the operation log data, determine the correlation between multiple features.
[0106] In practical applications, it can be assumed that there is a correlation between multiple features in the same segment of operation log data.
[0107] This application embodiment constructs a data parsing template library by matching feature fields, and then, based on this data parsing template library, it can efficiently and accurately match the target fields in the operation log data.
[0108] To better illustrate the solution of this application, a feasible embodiment is provided herein.
[0109] First, the data is structured.
[0110] Specifically, various methods, including FTP / SFTP, SYSLOG, TCP, ODBC / JDBC, SOAP, and FlumeAPI, can be used to collect operation log data generated by the system, network, and applications. The raw data format is analyzed, and data parsing templates are compiled to form a data parsing template library (as shown in Table 1). Further, the collected operation log data can be parsed based on this template library. The parsing method can be regular expression matching, such as parsing the system's operation logs into structured data information as shown in Table 2, revealing the relationships between multiple features.
[0111] Table 1 Data Parsing Template Library
[0112]
[0113]
[0114] Table 2 Structured Data Information
[0115]
[0116]
[0117] Next, the structured data shown in Table 2 is used to construct a graph, which is then used as the input to the graph convolutional neural network.
[0118] Furthermore, the graph data is represented using a graph convolutional neural network to obtain connection information, identify the relationships between node events, and obtain a relationship matrix.
[0119] This application specifically employs a reversible histogram convolutional neural network, and the inter-layer propagation rule for each layer is as shown in formula (1):
[0120]
[0121] Among them, H (x+1) It is the output matrix of the current layer network, where x represents the layer number of the current layer network, and H... (x) For the target matrix, H is the input layer. (x) =X, which is the characteristic matrix. W (x) Let be the trainable weight matrix, and σ(·) be the set function used by this layer of the network. The adjacency matrix is the original adjacency matrix plus an identity matrix. It is a degree matrix.
[0122] right To explain further, To avoid different nodes obtaining the same feature vector after adjacency matrix operations, when calculating new node features, in addition to considering the information of neighboring nodes, it is also necessary to take into account the information of the node itself. This is reflected in the adjacency matrix by adding an identity matrix to ensure that each node retains its uniqueness when updating its features.
[0123] Updating node features by summing neighbor and self nodes presents a problem: this aggregation method can lead to changes in the magnitude of features, which amplifies with increasing iterations. Therefore, a more suitable approach is to use averaging, which in graph structures is equivalent to left-multiplying the node's degree matrix. This ensures that the magnitude of the features remains unchanged during aggregation. The degree matrix is expressed as follows:
[0124]
[0125] A ij These are the adjacency matrix elements of the graph.
[0126] To further optimize the inter-layer propagation rules of the reversible histogram convolutional neural network, and to accelerate the convergence speed of the neural network and alleviate the gradient vanishing problem in deep layers, the idea of a reversible neural network is introduced (see the following formula). This allows the neural network to learn information more effectively, avoids the gradient vanishing problem, and reduces the over-smoothing phenomenon in deep networks. This design can improve training speed and enhance model performance while maintaining model complexity, and also enables deeper networks to be trained and optimized smoothly, thereby improving the overall model performance and generalization ability. The node representation of the current layer is calculated using the following formula:
[0127] H (l+1) =F l+1 (H l H l-m+2 )+μH l-m+1 (3)
[0128] H l-m+1 =μ -1 [H (l+1) -F l+1 (H l H l-m+2 (4)
[0129] Among them, F l+1 Represents any nonlinear operation of the standard residual function; in this embodiment, F l+1 =αH (0) +βH (l) +γH (l-m+2) μ represents a reversible operation, where l represents the number of layers in the current network and m represents the order of recursion, dividing every m features into a group.
[0130] In practical applications, H can be used (l+1) Substituting H into the above formula (x) .
[0131] Further based on node representation, to better obtain the final expression of the neural network, an attention mechanism is adopted to model the importance of each layer's embedding, using W... z The weighting coefficient represents the importance of each layer, W z The softmax function is used to normalize the weights of the embeddings from layer 0 to layer z, which contain the weights of the embeddings from layer z.
[0132] W z =softmax(ReLU(H) (0) ×H (z) (5)
[0133] Further based on the attention mechanism, its forward propagation formula is:
[0134]
[0135] in, W (0) W is the weight matrix from the input layer to the hidden layer. 1 This is the weight matrix from the hidden layer to the output layer;
[0136] ReLU and softmax are activation functions. ReLU takes the maximum value between the input and 0 and outputs the result, which has a one-sided inhibition effect on the output. This means that when the input is negative, the ReLU output is 0, and when the input is positive, the output is the input value itself. This not only makes the neurons sparsely activated, that is, inactive neurons output 0 and active neurons output non-zero values, but also helps to solve the gradient vanishing problem and promotes stable training of the network.
[0137] The ReLU activation function can be used to solve nonlinear problems and helps neural networks converge quickly. The ReLU expression is:
[0138] ReLU(input)=max(0,input) (7)
[0139] The output layer uses softmax as the activation function, and the output of each element after normalization is:
[0140]
[0141] The softmax function maps the output to a value in the interval (0,1), and the sum of the outputs is 1. Its function is to convert the output into a probability distribution, which can convert the output of the last layer of the neural network into probability predictions for each category, helping to determine the final classification result.
[0142] Furthermore, for the classification problem to be solved in this embodiment, cross-entropy error can be used as the loss function:
[0143]
[0144] Where M is the number of categories, y ic p is an indicator variable (taking values of 0 or 1). It takes a value of 1 if the class is the same as that of sample i, and a value of 0 otherwise. ic Let be the predicted probability that observed sample i belongs to category c.
[0145] During training, gradient descent is used to adjust the weight matrix W. (0) and W 1 Furthermore, by introducing randomness into the training process through dropout, the model is prevented from overfitting, and its generalization ability is improved, enabling the model to better adapt to unseen data and thus improve its performance on unknown data.
[0146] In this embodiment, to verify the model's effectiveness, the feature matrix X and adjacency matrix A of the input sample image are fed into the trained network. Using the forward propagation formula of the invertible histogram convolutional neural network and the trained weight matrix W, the network output is calculated. The output is then normalized using the softmax activation function. The label corresponding to the output with the highest probability is selected as the predicted label, and compared with the true label to verify the model's prediction accuracy. When the model's accuracy reaches the expected level, it can be deployed in production. This process ensures the model's robustness and reliability, providing a reliable foundation for practical applications.
[0147] As can be seen, this application collects data generated by systems, networks, applications, etc., parses the data using a subject parsing template to obtain structured data, then constructs graph data, classifies the graph data based on the reversible histogram convolutional neural network in this embodiment, and draws the entire data link, drawing the path of the entire process of an event from start to finish, thereby tracing the source of the data.
[0148] Based on the data tracing method provided in the above embodiments, this application provides a corresponding data tracing device. Figure 5 This is a schematic diagram of the structure of a data traceability device provided in another embodiment of this application. For example... Figure 5 As shown, the device includes:
[0149] Module 501 is used to acquire operation log data;
[0150] Extraction module 502 is used to extract multiple features from operation log data and the relationships between these features; features include events and event attributes;
[0151] Module 503 is used to construct a feature matrix based on multiple features, and to construct an adjacency matrix based on the association relationships between multiple features;
[0152] The operation module 504 is used to perform corresponding operations on the adjacency matrix and feature matrix in turn by the set functions corresponding to each layer of the graph convolutional neural network to obtain a relation matrix that represents multiple features and the relationship between multiple features.
[0153] The tracing module 505 is used to trace the data source of the first target feature based on the relationship matrix to obtain the second target feature that is related to the first target feature.
[0154] This application constructs graph data and transforms it into a corresponding relation matrix based on a graph convolutional neural network. Relying on the strong expressive power of the graph convolutional neural network, the integrity of the data can be guaranteed, thereby improving the accuracy of tracing. Simultaneously, this application constructs graph data by converting features and relationships into nodes and edges, which can accurately identify all features, further improving the accuracy of the tracing results.
[0155] As some optional embodiments, the arithmetic module 504 is specifically used for:
[0156] For each layer of a graph convolutional neural network, perform the following operations:
[0157] Based on the adjacency matrix and the target matrix corresponding to the current layer, the input matrix of the current layer is determined; for the first layer of the graph convolutional neural network, the target matrix is the feature matrix; for each subsequent layer of the graph convolutional neural network, the target matrix is determined based on the output matrix of the previous layer.
[0158] Based on the set function of the current layer network, the input matrix of the current layer network is processed to obtain the output matrix of the current layer network; the output matrix of the last layer network of the graph convolutional neural network is the relation matrix.
[0159] As some optional embodiments, the arithmetic module 504 is specifically used for:
[0160] For each layer of the graph convolutional neural network after the first layer, perform the following:
[0161] Based on the reversible adjustment factor and the first output matrix of other networks preceding the current layer, the target matrix corresponding to the current layer is determined so that the operation performed by the current layer is reversible; the first output matrix includes at least the output matrix of the previous layer.
[0162] As some optional embodiments, the acquisition module 501 is further configured to: acquire the degree matrix of multiple features before determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network;
[0163] Correspondingly, the arithmetic module 504 is specifically used for:
[0164] Process the adjacency matrix and the target matrix based on the degree matrix;
[0165] Based on the processed adjacency matrix and target matrix, the input matrix of the current layer network is determined.
[0166] As some optional embodiments, the arithmetic module 504 is specifically used for:
[0167] Determine the weight matrix for each layer of the graph convolutional neural network;
[0168] The input matrix of the current layer network is determined based on the weight matrix, adjacency matrix, and target matrix.
[0169] As some optional embodiments, the first building module 503 is specifically used for:
[0170] Based on the relationships between multiple features, construct the original adjacency matrix;
[0171] Add the original adjacency matrix and the identity matrix to obtain the adjacency matrix.
[0172] As some optional embodiments, the acquisition module 501 is further configured to: acquire multiple feature matching fields for matching features before extracting multiple features in the operation log data and the correlation between the multiple features;
[0173] Data tracing also includes: a third building module, which constructs a data parsing template library based on multiple feature matching fields;
[0174] Correspondingly, extraction module 502 is specifically used for:
[0175] Based on multiple feature matching fields in the data parsing template library, multiple target fields are matched in the operation log data;
[0176] Multiple features are determined based on multiple target fields; and the relationships between multiple features are determined based on the position of the target fields in the operation log data.
[0177] As some optional embodiments, the extraction module 502 is specifically used for:
[0178] Construct regular expressions based on feature matching fields;
[0179] Based on regular expressions, multiple target fields are matched in the operation log data.
[0180] The data traceability device provided in this application corresponds to the data traceability method provided in the above embodiments. The two have the same embodiments and beneficial effects, which will not be repeated here.
[0181] Figure 6 A schematic diagram of the data traceability device provided in an embodiment of this application is shown. Figure 6 As shown, the data traceability device may include a processor 601 and a memory 602 storing computer program instructions.
[0182] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0183] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.
[0184] Memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 602 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.
[0185] The processor 601 implements any of the data tracing methods described in the above embodiments by reading and executing computer program instructions stored in the memory 602.
[0186] In one example, the data traceability device may further include a communication interface 603 and a bus 604. For example, Figure 3 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 604 and complete communication with each other.
[0187] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.
[0188] Bus 604 includes hardware, software, or both, that couples components of a data tracing device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VESA Local Bus, VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.
[0189] The data traceability device can execute the data traceability method in the embodiments of this application, thereby realizing the data traceability method described above.
[0190] Furthermore, in conjunction with the data tracing methods described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the data tracing methods described in the above embodiments.
[0191] This application also provides a computer program product, including a computer program, which, when executed, implements any of the data tracing methods described in the above embodiments.
[0192] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0193] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0194] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0195] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.
[0196] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.
Claims
1. A data tracing method, characterized in that, include: Retrieve operation log data; Extract multiple features from the operation log data and the correlation relationships between these features; The features include events and event attributes; A feature matrix is constructed based on the multiple features, and an adjacency matrix is constructed based on the association relationships between the multiple features; Based on a graph convolutional neural network, the adjacency matrix and the feature matrix are processed to obtain a relation matrix that represents the multiple features and the relationships between the multiple features. Based on the relationship matrix, data tracing is performed on the first target feature to obtain a second target feature that is related to the first target feature.
2. The data tracing method according to claim 1, characterized in that, The graph convolutional neural network-based method processes the adjacency matrix and the feature matrix to obtain a relation matrix representing the multiple features and the relationships between them, including: For each layer of the graph convolutional neural network, perform the following operations: Obtain the target matrix corresponding to the current layer of the network; for the first layer of the graph convolutional neural network, the target matrix is the feature matrix; for each subsequent layer of the graph convolutional neural network, the target matrix is determined based on the output matrix of the previous layer. Based on the adjacency matrix and the target matrix corresponding to the current layer network, the input matrix of the current layer network is determined; Based on the set function of the current layer network, the input matrix of the current layer network is processed to obtain the output matrix of the current layer network; the output matrix of the last layer network of the graph convolutional neural network is the relation matrix.
3. The data tracing method according to claim 2, characterized in that, For each layer of the graph convolutional neural network after the first layer, obtaining the target matrix corresponding to the current layer includes: Based on the reversibility adjustment factor and the first output matrix of other layers preceding the current layer, the target matrix corresponding to the current layer is determined so that the operation performed by the current layer is reversible; the first output matrix includes at least the output matrix of the previous layer.
4. The data tracing method according to claim 2, characterized in that, Before determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network, the method further includes: Obtain the degree matrix of the multiple features; Correspondingly, based on the adjacency matrix and the target matrix corresponding to the current layer network, the input matrix of the current layer network is determined, including: The adjacency matrix and the target matrix are processed based on the degree matrix; Based on the processed adjacency matrix and target matrix, the input matrix of the current layer network is determined.
5. The data tracing method according to claim 2, characterized in that, Before determining the input matrix of the current layer network based on the adjacency matrix and the target matrix corresponding to the current layer network, the method further includes: Determine the weight matrix corresponding to each layer of the graph convolutional neural network; Correspondingly, based on the adjacency matrix and the target matrix corresponding to the current layer network, the input matrix of the current layer network is determined, including: The input matrix of the current layer network is determined based on the weight matrix, the adjacency matrix, and the target matrix.
6. The data tracing method according to any one of claims 1, characterized in that, Based on the association relationships between the multiple features, an adjacency matrix is constructed, including: Based on the relationships between these features, an original adjacency matrix is constructed; The adjacency matrix is obtained by adding the original adjacency matrix and the identity matrix.
7. The data tracing method according to any one of claims 1 to 6, characterized in that, Before extracting multiple features from the operation log data and the correlations between these features, the method further includes: Retrieve multiple feature matching fields used for feature matching; Based on the aforementioned multiple feature matching fields, a data parsing template library is constructed; Correspondingly, multiple features and the relationships between these features are extracted from the operation log data, including: Based on the multiple feature matching fields in the data parsing template library, multiple target fields are matched in the operation log data; Based on the multiple target fields, the multiple features are determined respectively; and based on the position of the target fields in the operation log data, the association relationship between the multiple features is determined.
8. The data tracing method according to claim 7, characterized in that, Based on the multiple feature matching fields in the data parsing template library, multiple target fields are matched in the operation log data, including: Based on the feature matching fields, construct a regular expression; Based on the regular expression, the multiple target fields are matched in the operation log data.
9. A data traceability device, characterized in that, The device includes: The acquisition module is used to acquire operation log data; The extraction module is used to extract multiple features from the operation log data and the relationships between these features; the features include events and event attributes. A construction module is used to construct a feature matrix based on the multiple features, and to construct an adjacency matrix based on the association relationships between the multiple features; The computation module is used to perform operations on the adjacency matrix and the feature matrix sequentially based on a graph convolutional neural network to obtain a relation matrix that represents the multiple features and the association relationships between the multiple features. The tracing module is used to trace the data of the first target feature based on the relationship matrix to obtain the second target feature that is related to the first target feature.
10. A data traceability device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the data tracing method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the data tracing method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the data tracing method as described in any one of claims 1 to 8.
