An Incremental Dynamic Data Extraction Method Based on Expert Models

Abstract

This invention relates to the field of network security technology and discloses an incremental dynamic data extraction method based on an expert model. The method includes: generating a local attack attribution subgraph; protecting privacy through homomorphic encryption and differential privacy; using a federated graph neural network to identify cross-organizational attack patterns; reconstructing cross-domain attack chains through Markov chain analysis; verifying the attack path based on zero-knowledge proofs; and outputting a threat level score. This invention resolves the contradiction between data privacy protection and collaborative analysis in cross-organizational APT attack detection, achieving accurate identification and reconstruction of cross-domain attack chains while protecting the sensitive data of each organization.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and more specifically, to an incremental dynamic data extraction method based on an expert model. Background Technology

[0002] In the cybersecurity protection systems of multinational corporations, various branches, supply chain partners, and cloud service providers form a complex business network. Advanced persistent threat (APT) attackers exploit this interconnected nature to move laterally between different organizations through long-term infiltration and multi-stage penetration.

[0003] In current technologies, organizations primarily rely on independent security monitoring systems for local threat detection, employing rule-based intrusion detection systems (IDS) and centralized log analysis methods to identify security incidents. Some enterprises are attempting to achieve cross-organizational collaborative analysis by directly sharing security log data.

[0004] However, the main technical challenges currently facing cross-organizational security collaboration include: First, APT attackers meticulously design attack chains, mimicking normal behavior within each organization, making it difficult for single-point detection to identify complete attack patterns; second, attack paths cross multiple organizational boundaries, forming complex multi-hop propagation networks, making it impossible for a single organization to obtain complete attack chain information; third, there are significant differences in security data formats, monitoring granularity, and expert rules among organizations, and direct data aggregation can expose system vulnerabilities and sensitive information; fourth, traditional centralized analysis methods violate data sovereignty and privacy regulations, hindering effective cross-organizational security collaboration. These technical problems make it difficult to detect and accurately trace cross-domain APT attacks in a timely manner, posing a serious security threat. Summary of the Invention

[0005] This invention provides an incremental dynamic data extraction method based on an expert model, which solves the technical problems in related technologies such as the contradiction between data privacy protection and collaborative analysis, and the inability to accurately identify and reconstruct cross-domain attack chains in the detection of APT attacks.

[0006] This invention provides an incremental dynamic data extraction method based on an expert model, comprising:

[0007] Acquire multi-source security log data, analyze entity relationships based on time-series characteristics, and generate local attack attribution sub-graphs reflecting attack behaviors within the organization;

[0008] Homomorphic encryption algorithm is used to encrypt the key path features and node attributes of the local attack tracing subgraph, and differential privacy is used to add noise to the graph structure to generate encrypted graph feature vectors that meet privacy protection requirements;

[0009] Each organization exchanges encrypted graph feature vectors and behavior transition probability matrices through a secure multi-party computation protocol, and uses a federated graph neural network to perform distributed learning on the exchanged encrypted data, outputting the results of attack jump point identification between organizations;

[0010] Markov chain analysis was performed on the results of inter-organizational attack jump point identification and the behavior transition probability matrix of each organization. Temporal correlation algorithm was used to correlate and match suspicious event fragments scattered in each organization to reconstruct the complete cross-domain attack chain.

[0011] The reconstructed cross-domain attack chain is verified based on the zero-knowledge proof protocol. A comprehensive threat level score is calculated based on the asset value, attack technology complexity, and time urgency involved in the attack chain, and cross-domain security incident warning information is output.

[0012] The federated graph neural network includes a local graph encoder, an encryption aggregator, and a global graph decoder. The local graph encoder aggregates and updates node features through graph convolutional layers. The encryption aggregator aggregates encryption gradients computed locally by each organization. The global graph decoder maps the aggregated features to the probability distribution of attack jump points through a multilayer perceptron.

[0013] Furthermore, the analysis of entity relationships based on time-series features includes: sorting each security event according to its timestamp, calculating the time interval, event frequency, and periodicity of adjacent events to form a time-series feature vector; and applying standardization processing to the time interval, event frequency, and periodicity features respectively. The standardization processing involves subtracting the mean of the feature from all samples and then dividing by the standard deviation to eliminate the influence of dimensional differences on subsequent analysis.

[0014] The local attack attribution subgraph uses entities in the security event as nodes, including IP addresses, processes, files, and user accounts, and uses the behavioral relationships between entities as edges, including access, execution, and modification.

[0015] Furthermore, the encryption process for tracing the local attack source subgraph using homomorphic encryption algorithms includes: generating public and private keys, performing encryption operations on each element in the graph feature matrix, and outputting the ciphertext feature matrix;

[0016] The addition of noise to the graph structure via differential privacy includes adding noise following a Laplace distribution to each element of the adjacency matrix, wherein the scale parameter of the noise is calibrated according to the privacy budget, and the scale parameter of the Laplace distribution is equal to the query sensitivity divided by the privacy budget.

[0017] Before adding differential privacy noise, a local sensitivity analysis is performed on the graph structure. The importance of each node is measured by calculating its PageRank value. Core nodes with PageRank values ​​greater than the threshold are given a smaller noise scale parameter, while ordinary nodes with PageRank values ​​less than or equal to the threshold are given a standard noise scale parameter.

[0018] Furthermore, the secure multi-party computation protocol splits the input of each organization into multiple shares through secret sharing. It adopts the Shamir secret sharing scheme. For the secret value held by an organization, a polynomial is randomly selected over a finite field to calculate the share and distribute it to each participating organization. When the secret is reconstructed, the secret value is restored through the Lagrange interpolation formula.

[0019] The behavior transition probability matrix represents the transition probability between different security event types, and the matrix dimension is the square of the number of security event types.

[0020] Furthermore, in the local graph encoder of the federated graph neural network, the node feature update of the graph convolutional layer is achieved by aggregating the features of neighboring nodes and multiplying them with the trainable weight matrix, and the pooling layer aggregates the node features into a graph-level feature vector through the concatenation operation of global average pooling and global max pooling.

[0021] The federated graph neural network adopts a federated learning training mode. Each organization performs forward and backward propagation locally to calculate local gradients, uploads the encrypted gradients to the central server for aggregation, and the central server distributes the aggregated gradients to each organization for model parameter updates.

[0022] The loss function employs a combination of transentropy loss, graph regularization term, and temporal consistency loss, where temporal consistency loss is used to constrain the continuity of the attack chain in the time dimension.

[0023] Furthermore, the Markov chain analysis constructs a global state space and calculates a global transition probability matrix, where each element represents the probability of transitioning from an event type in one organization to an event type in another organization.

[0024] The time-series correlation algorithm determines the correlation between event segments by calculating the time correlation coefficient between standardized event segments. When the correlation coefficient is greater than the threshold and the time window meets the conditions, the segments are connected into a complete attack event sequence.

[0025] Furthermore, the zero-knowledge proof protocol employs a non-interactive proof system, where the prover generates a proof based on public statements and private witnesses, and the verifier judges the validity of the proof through a verification algorithm, ensuring that the authenticity of the attack fragment is proven without exposing the original data;

[0026] The comprehensive threat level score is obtained by weighted summation after normalizing the asset value, attack technology complexity, and time urgency. The normalization process maps each factor value to the range of 0 to 1, and the sum of the weight coefficients is 1.

[0027] Furthermore, a dynamic weight adjustment strategy is adopted. Based on the latest threat intelligence and historical early warning feedback, the weight coefficients of each scoring factor are updated in real time through a feedback-based reward function. The reward function is equal to the early warning accuracy multiplied by the first balance coefficient minus the false alarm rate multiplied by the second balance coefficient.

[0028] The weight update process satisfies the constraint condition that the sum of the updated weights is 1 and each weight is non-negative. The constraint is satisfied by truncating negative values ​​and normalizing them.

[0029] Furthermore, an attention layer is introduced during the training of the federated graph neural network. This attention layer calculates attention weights through a dot product attention mechanism based on the business relevance and historical attack correlation between different organizations, and dynamically adjusts the weight coefficients of feature aggregation.

[0030] The business relevance is quantified by the frequency of data interaction between organizations and the proportion of business traffic, while the historical attack relevance is quantified by the number of common attack events between organizations in the historical threat intelligence database.

[0031] This invention provides an incremental dynamic data extraction system based on an expert model, comprising:

[0032] The data acquisition module is used to obtain multi-source security log data from various organizations;

[0033] The graph generation module is used to generate local attack tracing subgraphs based on temporal features;

[0034] The privacy protection module is used to perform homomorphic encryption and differential privacy processing;

[0035] The federated learning module is used to implement distributed training of federated graph neural networks;

[0036] The link reconstruction module is used to reconstruct cross-domain attack links using a time-series correlation algorithm.

[0037] The early warning output module is used to calculate the threat level and output a security warning.

[0038] The beneficial effects of this invention are as follows:

[0039] This invention overcomes the security and compliance obstacles of cross-organizational data sharing by employing federated graph neural networks and privacy protection methods, and solves the technical problem of the contradiction between data privacy protection and collaborative analysis in cross-domain APT attack detection, achieving the following technical effects:

[0040] First, the dual protection of homomorphic encryption and differential privacy ensures the security of sensitive data in various organizations, eliminates the risk of system vulnerability exposure caused by direct data sharing, meets data sovereignty and privacy regulations, and enables cross-organizational secure collaboration to be achieved under compliance.

[0041] Second, federated graph neural networks, through distributed learning, effectively identify cross-organizational attack patterns while protecting data privacy. This breaks through the limitations of a single organizational perspective, accurately identifies attack transition points between organizations, and significantly improves the detection accuracy of cross-domain attacks.

[0042] Third, the combination of Markov chain analysis and temporal correlation algorithm accurately reconstructs attack fragments scattered across various organizations into complete attack chains, solving the technical challenges of attack paths crossing multiple organizational boundaries and single-point detection being unable to identify complete attack patterns, thus providing technical support for the complete tracing of APT attacks.

[0043] Fourth, the zero-knowledge proof protocol ensures the credibility of cross-organizational collaborative analysis results. Organizations can verify the authenticity of attack fragments without exposing the original data, providing a reliable technical foundation for multi-party collaborative defense.

[0044] Fifth, the comprehensive threat level scoring mechanism combines multiple dimensions such as asset value, attack technology complexity, and time urgency, and optimizes the scoring model in real time through a dynamic weight adjustment strategy. This improves the timeliness and accuracy of early warnings, reduces the false alarm rate, and provides accurate decision-making basis for rapid response to cross-domain security incidents. Attached Figure Description

[0045] Figure 1 This is a flowchart of an incremental dynamic data extraction method based on an expert model according to the present invention;

[0046] Figure 2 This is a mixed plot of the attack link timing evolution of the present invention;

[0047] Figure 3 This is a bar chart of attack tracing subgraph node analysis of the present invention.

[0048] Figure 4 This is a line graph of the federated graph neural network training process of the present invention;

[0049] Figure 5 This is a bar chart of the threat level scoring factor analysis of the present invention;

[0050] Figure 6 This is a heatmap of the MITREATT&CK attack technique of this invention. Detailed Implementation

[0051] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0052] At least one embodiment of the present invention discloses an incremental dynamic data extraction method based on an expert model, such as... Figure 1 As shown, it includes the following steps:

[0053] Step 1: Obtain multi-source security log data and generate a local attack attribution sub-graph;

[0054] Security data from multiple sources, including network traffic logs, system operation logs, and application logs, is acquired locally from each organization, while external threat intelligence data is also obtained. Entity relationship analysis is performed on the acquired data based on time-series characteristics. Entities (IP addresses, processes, files, user accounts, etc.) in security events are treated as nodes, and behavioral relationships between entities (access, execution, modification, etc.) are treated as edges, generating a local attack attribution subgraph reflecting internal attack behavior within the organization.

[0055] It should be noted that the extraction of time-series features refers to sorting each security event according to its timestamp, calculating the time interval, event frequency, and periodicity of adjacent events, and forming a time-series feature vector. Since the time interval, event frequency, and periodicity have different dimensions and numerical ranges, these features are standardized using Z-scores before forming the time-series feature vector to eliminate the impact of dimensional differences on subsequent analysis.

[0056] Furthermore, the aforementioned Z-score standardization process is effective for features. The specific calculation formula is as follows ,in Features The mean of all samples is calculated as follows: , Features The standard deviation is calculated as follows: , For the sample size, These are the standardized feature values.

[0057] Step 2: Perform privacy protection processing on the local attack tracing subgraph to generate encrypted graph feature vectors;

[0058] Homomorphic encryption is used to encrypt the key path features and node attributes in the local attack tracing subgraph generated in step 1. The key path features include graph structure features such as path length, node degree, and edge weight. Since path length, node degree, and edge weight have different dimensions and numerical ranges, these graph structure features are normalized using mean normalization based on range before encryption, mapping the feature values ​​to... Interval. Apply Laplacian noise to the differential privacy-oriented graph structure; the noise's scale parameter is determined based on the privacy budget. Perform calibration to generate a product that meets the requirements. - Privacy-preserving graph feature vectors of differential privacy.

[0059] Furthermore, the aforementioned mean normalization process based on range is less effective for features. The specific calculation formula is as follows ,in Features The minimum value among all samples, Features The maximum value, For the sample size, The normalized eigenvalues ​​and .

[0060] The input to the aforementioned homomorphic encryption algorithm is a dimension of plaintext image feature matrix ,in For feature dimension, This represents the number of nodes. The algorithm first generates a public key. and private key Then for each element in the feature matrix Perform encryption operation ,in The row index represents the feature dimension index ( ), For column indexes, represent node indexes ( ), output dimension is Ciphertext feature matrix .

[0061] The input to the aforementioned differential privacy is the graph adjacency matrix. Privacy budget parameters By adding noise that follows a Laplace distribution to each element of the adjacency matrix The adjacency matrix after adding noise is obtained. Output graph-structured data that meets privacy protection requirements.

[0062] It should be noted that the addition of Laplace noise follows the formula ,in , For query sensitivity.

[0063] Furthermore, the aforementioned - Parameters in differential privacy This represents the probability of privacy breach failure, and its value ranges from [value missing]. In this embodiment of the application, it is set as follows: ,in This refers to the number of nodes. The aforementioned query sensitivity... Query sensitivity is defined as the maximum impact of a change in a single graph node or edge on the query results. For edge count queries of a graph adjacency matrix, it is the query sensitivity factor. For node degree queries, query sensitivity .

[0064] In this embodiment of the application, in order to balance the strength of privacy protection and data availability, a local sensitivity analysis is first performed on the graph structure before adding differential privacy noise. An adaptive noise addition strategy is adopted for nodes and edges of different importance to maintain high data accuracy for core attack paths.

[0065] Furthermore, the aforementioned local sensitivity analysis measures the importance of each node by calculating its PageRank value in the graph, using the following formula: ,in Set the damping coefficient to , Indicates pointing to a node The set of incoming edges, Represents a node The set of outgoing edges. The aforementioned adaptive noise addition strategy adjusts the noise scale according to node importance, specifically for nodes with PageRank values ​​greater than a threshold. The core node, with noise scale parameters set to For PageRank values ​​less than or equal to For ordinary nodes, the noise scale parameter is set to ,in In this embodiment, it is set to the median of the PageRank values ​​of all nodes.

[0066] Step 3: Exchange encrypted data through a secure multi-party computation protocol to train the federated graph neural network model;

[0067] Each organization exchanges the encrypted graph feature vectors and behavior transition probability matrices generated in step 2 through a secure multi-party computation protocol. The behavior transition probability matrix represents the transition probability between different security event types. Without exposing the original data, a federated graph neural network is used to perform distributed learning on the exchanged encrypted data. Through multiple rounds of iterative training, a cross-organizational attack pattern recognition model is obtained, outputting the attack jump point identification results between organizations.

[0068] The input to the aforementioned secure multi-party computation protocol is the set of encrypted data shares held by each organization. and the function to be calculated .

[0069] The protocol splits the inputs of each organization into multiple shares through secret sharing. Participants perform addition and multiplication operations in an encrypted domain to calculate the encrypted output of the function. Finally, the plaintext result is obtained through joint decryption. While outputting the calculation results, it ensures that the original inputs of each participant are not known to other parties.

[0070] Furthermore, the aforementioned secret sharing adopts the Shamir secret sharing scheme.

[0071] For organizations Secret value held In a finite field Random selection polynomial of degree ,in For large prime numbers, Set the reconstruction threshold to , For the first The first polynomial of the organization Each random coefficient ( ).organize calculate Individual shares And the first Individual shares Distributed to the organization During secret reconstruction, any An organization uses the Lagrange interpolation formula Restore the secret value, among which For the set of organizational indexes participating in the reconstruction and .

[0072] It should be noted that the aforementioned federated graph neural network comprises three units: a local graph encoder, a cryptographic aggregator, and a global graph decoder. The local graph encoder includes graph convolutional layers and pooling layers, where the graph convolutional layers aggregate and update node features. Layer node feature updates follow the formula ,in To add the adjacency matrix after adding self-loops, Its degree matrix, For trainable weight matrix, The activation function is used; the pooling layer is activated through... The node features are aggregated into a graph-level feature vector, where Indicates the first Subgraphs of the organization, as mentioned above The function uses a concatenation operation of global average pooling and global max pooling.

[0073] The specific calculation is as follows: ,in This represents a vector concatenation operation. This represents the number of nodes in the subgraph. The cryptographic aggregator aggregates the cryptographic gradients computed locally by each organization, resulting in the aggregated global gradient. ,in To the number of participating organizations, This indicates a homomorphic encryption operation.

[0074] The global graph decoder maps aggregated features to the probability distribution of attack jump points using a multilayer perceptron, and the calculation formula is as follows: ,in For trainable weights, This is a bias term.

[0075] Furthermore, the aforementioned degree matrix It is a diagonal matrix, and its diagonal elements Represents a node The degree in the adjacency matrix after adding self-loops, where For node index ( ), The summation variable represents the index of the adjacent node ( The aforementioned activation function Using the ReLU function, defined as The aforementioned number of convolutional layers. In this application embodiment, it is set to That is, the local graph encoder contains 3 graph convolutional layers.

[0076] The input layer of the aforementioned federated graph neural network receives two types of data: one type is data with dimension 1. Encryption node feature matrix ,in For node feature dimensions, For the first The number of nodes in an organization; another type is the dimension of Behavior transition probability matrix ,in This represents the number of security event types. The output layer output dimension is... Inter-organizational attack jump probability matrix ,in Indicates from the organization Jump to organization The probability of attack.

[0077] The aforementioned federated graph neural network employs a federated learning training model. Each organization performs forward and backward propagation locally to calculate its local gradients, then uploads the encrypted gradients to a central server for aggregation. The central server then distributes the aggregated gradients to each organization for model parameter updates. The optimization strategy uses the Adam optimizer, with a learning rate set to... The momentum parameter is set to and .

[0078] The loss function uses a combination of cross-entropy loss, graph regularization term, and temporal consistency loss, and the calculation formula is as follows: ,in For sample index ( ), For category index ( ), For the first The first sample The true label of the class, The corresponding predicted probability, For the sample size, The regularization coefficient is . Denotes the Frobenius norm. The weighting coefficient for time consistency loss. This is the time consistency loss term.

[0079] Furthermore, the aforementioned regularization coefficient Used to prevent model overfitting, its value range is: In this embodiment of the application, it is set as follows: To balance the model's fitting ability and generalization performance.

[0080] The aforementioned time consistency loss item Used to constrain the continuity of the attack chain in the time dimension, specifically calculated as follows: ,in The number of attack links. For link index ( ), For the first The number of time steps in each attack chain. index for time steps ( ), For the first Each link at time step The hidden state vector, The time interval between adjacent time steps. For the first The time evolution velocity vector of each link This represents the L2 norm. The aforementioned time consistency loss weighting coefficients... The range of values ​​is In this embodiment of the application, it is set as follows: To balance the constraints of time continuity and classification accuracy.

[0081] In this embodiment of the application, in order to improve the accuracy of identifying cross-organizational attack patterns, an attention layer is introduced during the training process of the federated graph neural network. This attention layer is based on the business relevance and historical attack correlation between different organizations.

[0082] By calculating attention weights The weight coefficients of feature aggregation are dynamically adjusted, where The similarity scoring function is calculated using a dot product attention mechanism. ,in The attention weight matrix is ​​a learnable matrix. For the hidden layer feature dimension, and The first The and the first The feature vector of an organization.

[0083] Furthermore, the aforementioned business relevance is quantified through the frequency of data interaction between organizations and the proportion of business traffic. The specific calculation formula is as follows: ,in Indicates organization With organization Number of data interactions within the statistical period This indicates the total volume of the corresponding business traffic. and The weighting coefficients are and satisfy the following conditions: In this embodiment of the application, it is set as follows: , The aforementioned statistical period is determined based on the frequency of business data updates, and in this embodiment, it is set to 30 days, meaning that it counts the data interactions between organizations within the most recent 30 days. The aforementioned historical attack correlation is quantified by the number of common attack events between organizations in the historical threat intelligence database, calculated using the following formula: ,in and Representing organizations and organization The set of attack types suffered during a historical period. The cardinality of the set is represented by the aforementioned historical period, which is set to the past 180 days in this embodiment of the application.

[0084] Step 4: Analyze the global behavior transfer probability and reconstruct the cross-domain attack chain;

[0085] Markov chain analysis was performed on the inter-organizational attack jump point identification results and the behavior transition probability matrices of each organization obtained in step 3 to calculate the global behavior transition probability matrix across organizations. A temporal correlation algorithm was used to correlate and match suspicious event fragments scattered across various organizations. Based on time windows and behavioral similarity, the fragments were connected into a complete event sequence, reconstructing a complete cross-domain attack chain.

[0086] The input to the aforementioned Markov chain analysis is The set of behavioral transition probability matrices for each organization , where each matrix Indicates the first Internal organization The transition probabilities between different security event types, and the inter-organizational attack jump probability matrix output in step 3. .

[0087] The algorithm constructs a global state space. Calculate the global transition probability matrix , of which elements Indicates from the organization event types Transfer to organization event types The probability when The probability is when ,when The probability is when The output dimension is The global behavior transition probability matrix.

[0088] Furthermore, the aforementioned This represents the initial event type when an attacker enters a new organization, and its value in the behavior transition probability matrix. The index corresponding to the first row is used. The initial event type index is determined by analyzing the first intrusion event type in the historical attack logs of each organization. In this embodiment, the intrusion event type with the highest statistical frequency is used as the index. Status, if organization Historical data shows that network scanning events are the most common initial attack behavior. The index value corresponding to the network scan event type.

[0089] The input to the aforementioned time-series correlation algorithm is a set of suspicious event fragments reported by each organization. Each segment Includes timestamp sequence and event type sequence .

[0090] First, the event types are one-hot encoded into numerical vectors. Then, the timestamp sequence and the encoded event type sequence are Z-score standardized to eliminate dimensional differences. The algorithm calculates the temporal correlation coefficient between the standardized event segments. To determine the relevance of segments, among which and They represent the standardized first digits. Each timestamp and event type encoding, The number of events in the segment. When the correlation coefficient... Greater than the threshold And the time window satisfies At that time, the fragment and Connect and output the reconstructed cross-domain attack event sequence. .

[0091] Furthermore, the aforementioned time correlation coefficient threshold The range of values ​​is The closer the correlation coefficient is to 1, the higher the temporal similarity between event segments. In this embodiment, it is set to... To balance detection accuracy and recall. The aforementioned maximum time window. The value is set according to the typical latency period of an APT attack, and its range is [value range missing]. In this embodiment of the application, it is set as follows: That is, if the time interval between two suspicious event segments exceeds 24 hours, they are not considered as consecutive segments of the same attack chain.

[0092] Step 5: Verify the attack path and output a cross-domain security incident alert;

[0093] The authenticity and integrity of the cross-domain attack chain reconstructed in step 4 are verified using a zero-knowledge proof protocol. Each organization proves the authenticity of its provided attack fragments without exposing the original data. A comprehensive threat level score is calculated based on the asset value, attack technique complexity, and time urgency involved in the attack chain. Since asset value, attack technique complexity, and time urgency have different dimensions and numerical ranges, mean normalization based on the range is applied to these three factors before calculating the comprehensive score, mapping each factor value to... The range is then calculated, and a comprehensive threat level score is obtained through weighted summation. This results in the output of early warning information for cross-domain security incidents requiring coordinated response.

[0094] Furthermore, the aforementioned comprehensive threat level score is calculated through a weighted summation, using the following formula:

[0095] ,

[0096] in , and These represent the normalized asset value, the complexity of the attack technique, and the time urgency, respectively. , and The corresponding weight coefficients and satisfying And the range of values ​​for each weight coefficient is: In this embodiment of the application, it is set as follows: , , .

[0097] Furthermore, the aforementioned asset value Obtained through the asset configuration database involved in the attack chain, specifically calculated as follows: ,in This refers to the set of assets involved in the attack chain. For assets The preset value score is determined based on the asset's business importance, data sensitivity, and replacement cost, with a value range of [value range missing]. .

[0098] The aforementioned attack technique complexity Obtained through the technical difficulty level in the MITRE ATT&CK framework, specifically calculated as follows: ,in A set of attack techniques used in the attack chain. attack techniques The difficulty levels in the MITREATT&CK framework range from [value range missing]. The aforementioned time urgency It is calculated based on the attack progress rate and remaining defense time, specifically as follows: ,in This represents the number of attack phases that have been completed. This represents the total number of stages in the complete attack chain. The maximum expected time for the attack to complete is set to 72 hours. This is the estimated remaining defense time based on the current attack speed.

[0099] Furthermore, the aforementioned mean normalization process based on range affects the original factor values. The specific calculation formula is as follows ,in and These are the minimum and maximum values ​​of the original factor values ​​in historical statistical data, and the normalized values, respectively. Regarding asset value, Regarding the complexity of attack techniques, The minimum value is 1, and the maximum value is 5; regarding time urgency, When the original value exceeds 1, it is truncated to 1.

[0100] The aforementioned zero-knowledge proof protocol employs a non-interactive proof system, with inputs including public statements. Private witnesses held by the witnesses ,in Public information representing the attack chain (event time sequence, jump path structure). This represents each organization's private evidence (raw log data, internal monitoring records).

[0101] The prover through execution Generate proof ,prove It includes commitment and response values, satisfying the zero-knowledge property, meaning the verifier cannot obtain the value from the response. Derivation from Any information. Validators receive public statements. and proof As input, execute Output Boolean value ,when When the time indicates that the proof is valid, that is, the attack fragment is real, when This indicates that the proof is invalid.

[0102] In this embodiment of the application, in order to improve the timeliness and accuracy of early warnings, a dynamic weight adjustment strategy is adopted in the threat level score calculation. Based on the latest threat intelligence and historical early warning feedback, the weight is updated using a weight update formula. The weight coefficients of each rating factor are updated in real time. Indicates the number of iterations. For feedback-based reward functions,

[0103] The specific calculation is as follows: ,in Indicates the accuracy rate of early warnings. Indicates the false alarm rate. and For balance coefficient, is the learning rate for weights.

[0104] Furthermore, the aforementioned balance coefficient and Used to control the impact of accuracy and false alarm rate on weight updates, their value ranges are respectively and And satisfy In this embodiment of the application, it is set as follows: and Prioritize improving detection accuracy. The aforementioned weighted learning rate... The step size used to control weight updates, and its value range is: In this embodiment of the application, it is set as follows: This is to ensure the stability and convergence of the weight adjustment.

[0105] Furthermore, the aforementioned gradient It is calculated using numerical differentiation methods, specifically as follows:

[0106] ,

[0107] in Set the small perturbation amount to .

[0108] The aforementioned early warning accuracy The calculation is based on verification data from historical early warning results. Specifically, the calculation is as follows: ,in The number of true positive alerts represents the number of correctly identified attack events. The number of true negatives, i.e., the number of normal events that were correctly excluded. The number of false positives is the number of normal events that are falsely reported. This represents the number of false negatives, i.e., the number of attack events that were missed. The aforementioned false positive rate... Calculated as This represents the proportion of all actual normal events that are falsely flagged as attacks.

[0109] Furthermore, the aforementioned weight update process must satisfy the following constraints:

[0110] ,

[0111] Specifically, this is implemented by performing a weight update. Then, truncate the negative values ​​first. Then through normalization operation Ensure that the sum of the weights is 1, where This represents three rating factors.

[0112] A multinational manufacturing conglomerate consists of three main organizations: its headquarters (Organization A) is located in China, responsible for core business management and R&D data storage; its European branch (Organization B) is responsible for regional market operations; and its cloud service provider (Organization C) hosts the company's critical business systems. In October 2024, the conglomerate suffered a sophisticated APT attack. The attackers moved laterally between the three organizations through multi-stage penetration, attempting to steal core technical data. This example demonstrates how the method of this invention can be applied to detect and reconstruct this cross-domain attack chain.

[0113] Step 1 Implementation Example: Generate a local attack attribution subgraph;

[0114] Organization B's security monitoring system captured a series of suspicious security incidents between October 15 and October 17, 2024. The original security log data is shown in Table 1.

[0115] Table 1. Raw security log data of Organization B:

[0116]

[0117] Entity relationship analysis and temporal feature extraction were performed on the original data in Table 1. The time intervals between adjacent events were calculated to be 27 seconds, 46 seconds, 96 seconds, 105 seconds, 104457 seconds (approximately 29 hours), and 45803 seconds (approximately 12.7 hours). Z-score standardization was applied to the time intervals and event frequencies, with the mean time interval being... seconds, standard deviation The generated local attack attribution subgraph contains 9 nodes and 8 edges, and the node information is shown in Table 2.

[0118] Table 2. Node information of the local attack attribution subgraph for Organization B:

[0119]

[0120] Figure 3 The table shows the node information of the local attack tracing subgraph generated in step 1 (data in Table 2), which reflects the basic data of the local sensitivity analysis in step 2.

[0121] Figure 4 This demonstrates the training process of the federated graph neural network in step 3. It showcases the convergence of federated learning and the decreasing trend of each loss term.

[0122] Step 4 Implementation Example: Reconstructing the Cross-Domain Attack Chain;

[0123] After the privacy protection processing in step 2 and the federated learning training in step 3, the suspicious event fragments reported by each organization are shown in Table 3.

[0124] Table 3: Suspicious incident fragments from various organizations:

[0125]

[0126] A time-series correlation analysis was performed on the event segments to calculate the temporal correlation coefficients and time window intervals between the segments. The results are shown in Table 4.

[0127] Table 4. Results of temporal correlation analysis:

[0128]

[0129] Due to the time correlation coefficient between segments F1 and F2 and time interval F1, F2, and F3 satisfy the connection conditions; similarly, F2 and F3 also satisfy the connection conditions. Therefore, connecting F1, F2, and F3 in chronological order and reconstructing the complete cross-domain attack chain is shown in Table 5.

[0130] Table 5 Reconstructed Cross-Domain Attack Chain:

[0131]

[0132] Figure 2 It demonstrates the evolution of cross-domain attack chains over time, reflecting the temporal characteristics of cross-domain attacks and the jumping patterns between organizations.

[0133] Step 5 Implementation Example: Threat Level Score Calculation;

[0134] Based on the reconstructed attack chain, the raw data of each scoring factor were collected as shown in Table 6.

[0135] Table 6 Threat Level Scoring Factor Data:

[0136]

[0137] The original factor values ​​were normalized to mean based on the range, where the historical statistical range of asset values ​​was [missing information]. Calculated The range of attack technique complexity is: Calculated The original value of time urgency exceeds 1, so it is truncated. .

[0138] The comprehensive threat level scoring formula is applied, with the following weighting coefficients: , , The calculation results are shown in Table 7.

[0139] Table 7: Overall Threat Level Score Results

[0140]

[0141] Figure 5 The calculation process for the threat level score is shown (data in Tables 6 and 7).

[0142] Figure 6 It demonstrates the techniques used and their complexity at each stage of the reconstructed cross-domain attack chain.

[0143] Overall Threat Level Score This is classified as a high-risk event (score range 0.7-1.0). The system immediately outputs a cross-domain security incident warning and recommends that the three organizations work together to take emergency response measures: Organization B should immediately isolate the infected workstation and reset the domain controller credentials; Organization A should strengthen server access control and fix the exploited vulnerabilities; and Organization C should suspend suspicious API calls and review cloud storage access logs, while initiating a data breach impact assessment procedure.

[0144] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. An incremental dynamic data extraction method based on an expert model, characterized in that, The method comprises the following steps: obtaining multi-source security log data, analyzing entity relationships based on time sequence characteristics, and generating a local attack tracing subgraph reflecting internal attack behaviors of the organization; using a homomorphic encryption algorithm to encrypt key path characteristics and node attributes of the local attack tracing subgraph, and adding noise to the graph structure through differential privacy to generate an encrypted graph feature vector meeting privacy protection requirements; each organization exchanges the encrypted graph feature vector and the behavior transition probability matrix through a secure multi-party computation protocol, uses a federated graph neural network to perform distributed learning on the exchanged encrypted data, and outputs attack jump point identification results between organizations; performing Markov chain analysis on the attack jump point identification results between organizations and the behavior transition probability matrix of each organization, using a time sequence correlation algorithm to correlate and match suspicious event fragments scattered in each organization, and reconstructing a complete cross-domain attack link; verifying the reconstructed cross-domain attack link based on a zero-knowledge proof protocol, calculating a comprehensive threat level score according to asset value, attack technology complexity and time urgency involved in the attack link, and outputting cross-domain security event warning information. The federated graph neural network comprises a local graph encoder, an encrypted aggregator and a global graph decoder, the local graph encoder aggregates and updates node features through a graph convolution layer, the encrypted aggregator aggregates encrypted gradients calculated locally by each organization, and the global graph decoder maps aggregated features to an attack jump point probability distribution through a multi-layer perception machine.

2. The incremental dynamic data extraction method based on an expert model according to claim 1, wherein the analysis of entity relationships based on time sequence characteristics comprises: sorting each security event according to a time stamp, calculating the time interval, event frequency and periodicity characteristics of adjacent events, and forming a time sequence feature vector; and performing standardization processing on the time interval, event frequency and periodicity characteristics, wherein the standardization processing is to subtract the mean value of the feature in all samples from the feature value and then divide by the standard deviation, so as to eliminate the influence of dimensional difference on subsequent analysis; the local attack tracing subgraph takes entities in security events as nodes, and the entities include IP addresses, processes, files and user accounts, and takes behavior relationships between entities as edges, and the behavior relationships include access, execution and modification.

3. The incremental dynamic data extraction method based on an expert model according to claim 1, wherein the encryption processing of the local attack tracing subgraph using a homomorphic encryption algorithm comprises: generating a public key and a private key, performing an encryption operation on each element in the graph feature matrix, and outputting a ciphertext feature matrix; the addition of noise to the graph structure through differential privacy comprises: adding noise obeying a Laplace distribution to each element of the adjacency matrix, and the scale parameter of the noise is calibrated according to a privacy budget, wherein the scale parameter of the Laplace distribution is equal to the query sensitivity divided by the privacy budget. ​ ​ Before adding differential privacy noise, the local sensitivity of the graph structure is analyzed, the importance of each node is measured by calculating its PageRank value, and the core nodes with PageRank values greater than a threshold are given smaller noise scale parameters, while the ordinary nodes with PageRank values less than or equal to the threshold are given standard noise scale parameters.

4. The expert model-based incremental dynamic data extraction method of claim 1, wherein, The secure multi-party computation protocol splits the input of each organization into multiple shares through secret sharing, adopts a Shamir secret sharing scheme, randomly selects a polynomial on a finite field for a secret value held by an organization, calculates shares and distributes them to each participating organization, and restores the secret value through the Lagrange interpolation formula when reconstructing the secret. The behavior transition probability matrix represents the transition probabilities between different security event types, and the matrix dimension is the square of the number of security event types.

5. The expert model-based incremental dynamic data extraction method of claim 1, wherein, In the local graph encoder of the federated graph neural network, the node feature update of the graph convolution layer is achieved by aggregating neighbor node features and multiplying them with a trainable weight matrix, and the pooling layer aggregates node features into a graph-level feature vector through the concatenation operation of global average pooling and global maximum pooling; The federated graph neural network adopts a federated learning training mode, each organization performs forward propagation and backward propagation to calculate local gradients, uploads encrypted gradients to a central server for aggregation, and the central server distributes aggregated gradients to each organization for model parameter update; The loss function adopts a combination of cross-entropy loss, graph regularization term and time consistency loss, wherein the time consistency loss is used to constrain the continuity of attack links in the time dimension.

6. The expert model-based incremental dynamic data extraction method of claim 1, wherein, The Markov chain analysis calculates a global transition probability matrix by constructing a global state space, and the matrix elements represent the probability of transitioning from an event type of one organization to an event type of another organization; The time series correlation algorithm calculates the time correlation coefficient between standardized event segments to determine the correlation of the segments, and when the correlation coefficient is greater than a threshold and the time window meets the conditions, the segments are connected into a complete attack event sequence.

7. The expert model-based incremental dynamic data extraction method of claim 1, wherein, The zero-knowledge proof protocol adopts a non-interactive proof system, the prover generates a proof based on a public statement and a private witness, and the verifier judges the validity of the proof through a verification algorithm to ensure the authenticity of the attack segment without exposing the original data; The comprehensive threat level score is obtained by weighted summation after normalizing the asset value, attack technical complexity and time urgency, the normalization maps each factor value to the interval [0, 1], and the sum of the weight coefficients is 1.

8. The expert model-based incremental dynamic data extraction method of claim 7, wherein, The dynamic weight adjustment strategy is adopted, the weight coefficients of each scoring factor are updated in real time according to the latest threat intelligence and historical early warning feedback through a feedback-based reward function, and the reward function is equal to the early warning accuracy multiplied by a first balance coefficient minus the false positive rate multiplied by a second balance coefficient; The weight updating process satisfies the constraint condition that the sum of the updated weights is 1 and each weight is non-negative, and the constraint satisfaction is realized through truncation processing and normalization operation on the negative values.

9. The expert model-based incremental dynamic data extraction method according to any one of claims 1 to 8, characterized in that, An attention layer is introduced in the federal graph neural network training process, which calculates attention weights through dot product attention mechanism according to the business correlation degree and historical attack correlation between different organizations, and dynamically adjusts the weight coefficients of feature aggregation; The business correlation degree is quantitatively calculated by the data interaction frequency and business traffic proportion between organizations, and the historical attack correlation is quantitatively calculated by the number of common attacked events between organizations in the historical threat intelligence database.

10. An expert model-based incremental dynamic data extraction system for performing the expert model-based incremental dynamic data extraction method of any one of claims 1-9, wherein, Comprise: a data acquisition module for acquiring multi-source security log data from each organization; a graph generation module for generating a local attack tracing subgraph based on time sequence characteristics; a privacy protection module for performing homomorphic encryption and differential privacy processing; a federal learning module for realizing distributed training of the federal graph neural network; a link reconstruction module for reconstructing cross-domain attack links through time sequence correlation algorithm; an early warning output module for calculating threat level and outputting security early warning.

Images