Data security protection system based on reinforcement learning large model

Abstract

The application provides a data security protection system based on a reinforcement learning large model, relates to the technical field of data processing, and comprises the following steps: extracting corresponding attack behavior sequence features from a correlation history attack chain feature library; constructing a comprehensive security state representation body by fusing knowledge data entities, real-time environment baseline offset amounts and history attack chain features; generating a protection strategy instruction set by a pre-training reinforcement learning large model from the security state representation body; deploying the protection strategy instruction set in a zero trust architecture, collecting a strategy effective time delay, a security entropy gradient and a defense performance index to generate a feedback tensor; constructing a reward function based on the feedback tensor, updating a policy network parameter of the reinforcement learning model by using a proximal policy optimization algorithm to form a closed loop, and improving the intelligence, dynamics and adaptability of data security protection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data security protection system based on a large reinforcement learning model. Background Technology

[0002] With the deep integration of the digital economy and information technology, data has become a core production factor, and its security protection faces unprecedented challenges. On the one hand, the explosive growth in data volume and the diversification of data types make traditional security measures such as boundary protection and static rule matching inadequate for complex threat scenarios. On the other hand, the intelligence and stealth of attack methods place higher demands on the real-time performance, adaptability, and predictive capabilities of protection systems.

[0003] Traditional data security protection systems often rely on single-modal analysis, which suffers from problems such as incomplete feature extraction, weak threat correlation, and lagging policy updates, making it difficult to meet the needs of "proactive defense and precise protection" in dynamic environments. Existing technologies often rely on statically configured rule engines for policy generation, which, while capable of handling known threats, expose significant shortcomings in dynamic environments, failing to adjust permissions based on real-time risk assessments. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a data security protection system based on a large reinforcement learning model, which improves the intelligence, dynamism and adaptability of data security protection.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] Firstly, a data security protection system based on a large reinforcement learning model includes:

[0007] The acquisition module is used to acquire multi-source heterogeneous raw data streams, and uses protocol fingerprint parsing and adaptive feature encoding to generate a set of independent data units and extract the feature vectors of the data units;

[0008] The calculation module is used to generate topological space coordinate vectors using a topological space mapping algorithm with data unit sets as nodes, and to calculate threat propagation weights to generate topological subsets.

[0009] The fusion module is used to extract risk indicators based on topological subsets, calculate adjustment factors, and fuse the adjustment factors with the feature vectors of data units to generate a risk-enhanced feature matrix.

[0010] The extraction module is used to input the risk enhancement feature matrix into the data platform, map it to semantic nodes through the domain knowledge graph and inject topological risk identifiers to generate knowledge-based data entities, and start the security situation awareness device to generate real-time environmental baseline offset, and extract the corresponding attack behavior sequence features by associating with the historical attack chain feature library.

[0011] The execution module is used to integrate knowledge-based data entities, real-time environmental baseline offsets, and historical attack chain features to construct a comprehensive security status representation; and to generate a set of protection strategy instructions from the security status representation through a pre-trained reinforcement learning model.

[0012] The deployment module is used to deploy protection policy instruction sets in a zero-trust architecture, and collect policy activation latency, security entropy gradient and defense effectiveness indicators to generate feedback tensors.

[0013] The reinforcement module is used to construct a reward function based on the feedback tensor and to update the policy network parameters of the reinforcement learning model using a proximal policy optimization algorithm to form a closed loop.

[0014] Furthermore, the process involves acquiring multi-source heterogeneous raw data streams, using protocol fingerprint parsing and adaptive feature encoding to generate a set of independent data units, and extracting feature vectors from these data units, including:

[0015] The protocol signature is extracted from the multi-source heterogeneous raw data stream, and the signature is matched based on the pre-built protocol fingerprint library to generate a protocol identifier; the corresponding protocol parsing rules are loaded according to the protocol identifier, and the data stream is deconstructed in layers to generate a structured protocol metadata set.

[0016] The protocol metadata set is standardized to generate standardized data units; an adaptive feature encoder is constructed based on the spatiotemporal distribution characteristics of the standardized data units to obtain dimensionality-reduced feature vectors, statistical feature vectors, and fused feature vectors.

[0017] The dimensionality-reduced feature vector, statistical feature vector, and fused feature vector are aligned and concatenated to generate a data unit feature vector; all data unit feature vectors are then fused to generate an independent set of data units, and the data unit feature vectors are extracted.

[0018] Furthermore, using the data unit set as nodes, a topological space coordinate vector is generated through a topological space mapping algorithm, and threat propagation weights are calculated to generate a topological subset, including:

[0019] A node feature matrix is ​​constructed based on a set of independent data units; a multi-scale analysis of the similarity of feature dimensions and the correlation of time dimensions is performed on the node feature matrix to generate a node association strength matrix;

[0020] The node association strength matrix is ​​mapped to a low-dimensional space to generate a topological space coordinate vector through a manifold learning algorithm that preserves the local adjacency structure.

[0021] The original threat propagation weights are generated by analyzing the spatial geometric relationships between nodes based on topological coordinate vectors and integrating historical threat interaction records; attenuation compensation is applied to the original threat propagation weights to generate standardized threat propagation weights.

[0022] Based on standardized threat propagation weights, the final topological subset is generated by dividing the data into high-risk propagation clusters, medium-risk subsets, and independent subsets using dynamic connection thresholds.

[0023] Furthermore, risk indicators are extracted based on topological subsets to calculate adjustment factors. These adjustment factors are then fused with the feature vectors of data units to generate a risk-enhanced feature matrix, including:

[0024] Based on topological subsets, threat concentration indicators of high-risk propagation clusters, propagation potential indicators of medium-risk subsets, and isolation safety indicators of independent subsets are extracted to generate a basic risk indicator set;

[0025] Injecting a set of basic risk indicators into the severity coefficient of real-time threat intelligence and implementing time decay compensation of historical indicators to generate dynamically corrected risk indicators.

[0026] The S-shaped function transformation is applied to the dynamically corrected risk indicators to generate normalized risk values; the density index of high-risk propagation clusters and the diffusion rate index of medium-risk subsets are extracted based on topological subsets to generate environmental threat coefficients; the boundary values ​​of the adjustment interval are calculated based on the environmental threat coefficients; and the normalized risk values ​​are linearly mapped to the adjustment intervals to generate risk adjustment factors.

[0027] The feature vectors of data units are divided into network layer, transport layer and application layer feature channels according to the protocol stack level; the weighted features of the three layers are fused to generate a risk enhancement feature matrix.

[0028] Furthermore, the risk enhancement feature matrix is ​​input into the data platform, mapped to semantic nodes through the domain knowledge graph, and topological risk identifiers are injected to generate knowledge-based data entities. A security situation awareness device is then activated to generate real-time environmental baseline offsets, and corresponding attack behavior sequence features are extracted by associating them with the historical attack chain feature library, including:

[0029] The risk enhancement feature matrix is ​​input into the structure aligner of the data platform, matched with domain knowledge graph nodes to generate an initial semantic node set, and then similar to the domain knowledge graph nodes to generate an optimized semantic node set. Context consistency verification is performed on the initial semantic node set to generate an optimized semantic node set.

[0030] Based on standardized threat propagation weights, topological risk identifiers are injected into the optimized semantic node set to generate semantic nodes with risk identifiers; the semantic nodes with risk identifiers are encapsulated to generate knowledge-based data entities.

[0031] The security situation awareness device is activated based on knowledge-based data entities to generate real-time environmental baseline offsets; historical attack chain feature databases are retrieved based on the semantic node identifiers of knowledge-based data entities, and attack behavior sequence features are extracted.

[0032] Furthermore, a comprehensive security status representation is constructed by integrating knowledge-based data entities, real-time environmental baseline offsets, and historical attack chain features; this security status representation is then used to generate a set of protection strategy instructions through a pre-trained reinforcement learning model, including:

[0033] Topological semantic deconstruction is performed on knowledge-based data entities to extract entity relationship feature vectors; real-time environment baseline offset is scaled and standardized to generate environment offset features; temporal position encoding is performed on attack behavior sequence features to generate sequence feature vectors.

[0034] The entity relationship feature vector, environmental offset feature, and sequence feature vector are integrated along the feature fusion axis to construct a structured security state body. The structured security state body is input into a pre-trained reinforcement learning model for defense benefit evaluation. Candidate protection sequences are generated based on the defense benefit evaluation results. High-return protection sequences are selected by comparing benefit advantages. The high-return protection sequences are converted into a protection strategy instruction set.

[0035] Furthermore, a protection policy instruction set is deployed in the zero-trust architecture, and feedback tensors are generated by collecting policy activation latency, security entropy change gradient, and defense effectiveness indicators, including:

[0036] Deploy the protection policy instruction set to the policy execution node of the zero-trust architecture; monitor the policy execution node to generate a policy effective delay sequence; perform window standardization processing on the policy effective delay sequence to generate a standardized delay sequence;

[0037] Based on the standardized time delay sequence, the security entropy change rate of new data units is collected during the monitoring cycle. The security entropy change rate is then used to calculate the gradient to generate a security entropy change gradient matrix. During the monitoring cycle, defense effectiveness indicators are statistically analyzed to generate an effectiveness indicator vector. The effectiveness indicator vector is then transformed into a dimensionless form to generate a standardized effectiveness vector.

[0038] Align the standardized time delay sequence, the security entropy gradient matrix, and the standardized performance vector at the time origin; then concatenate the aligned data along the time dimension to generate a feedback tensor.

[0039] Furthermore, a reward function is constructed based on the feedback tensor, and the policy network parameters of the reinforcement learning model are updated using a proximal policy optimization algorithm to form a closed loop, including:

[0040] Based on the feedback tensor, a standardized time delay sequence is separated. Based on the preset time delay benchmark value, the timeliness of the standardized time delay sequence is evaluated to generate a timeliness quantification index. The timeliness quantification index is compared with the timeliness threshold to generate a timeliness reward component.

[0041] Based on the feedback tensor, the security entropy change gradient matrix is ​​separated. The security entropy change gradient matrix is ​​then used for threat diffusion analysis to generate a threat situation quantitative index. The threat situation quantitative index is compared with the threat suppression threshold to generate a threat suppression reward component.

[0042] Based on the time series feedback tensor, a standardized performance vector is separated. The standardized performance vector is then used to analyze the defense effect and generate a quantitative index of defense effect. The quantitative index of defense effect is compared with the defense performance threshold to generate a defense effect reward component.

[0043] The timeliness reward component, threat suppression reward component, and defense effect reward component are weighted and fused to generate a comprehensive reward value; based on the comprehensive reward value, the near-end policy optimization algorithm is used to calculate the adjustment amount of policy network parameters;

[0044] The application parameter adjustment amount updates the policy network parameters of the reinforcement learning large model, forming a closed loop.

[0045] In a second aspect, a computing device includes:

[0046] One or more processors;

[0047] A storage device for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the system.

[0048] Thirdly, a computer-readable storage medium storing a program that, when executed by a processor, implements the system.

[0049] The above-described solution of the present invention has at least the following beneficial effects:

[0050] By employing a topological space mapping algorithm to generate topological space coordinate vectors and calculating threat propagation weights, and combining risk indicator adjustment factors with data unit feature vectors to generate a risk enhancement feature matrix, the correlation between data units and potential risks can be accurately captured, improving the accuracy and comprehensiveness of risk assessment and enabling the timely detection of hidden security threats.

[0051] Furthermore, by integrating knowledge-based data entities, real-time environmental baseline offsets, and historical attack chain features to construct a comprehensive security status representation, and by generating a set of protection strategy instructions through a pre-trained reinforcement learning model, the protection strategies can fully combine real-time environmental conditions and historical attack experience, making them more aligned with actual security needs and improving the pertinence and effectiveness of protection measures. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of a data security protection system based on a large reinforcement learning model provided in an embodiment of the present invention.

[0053] Figure 2This is a schematic diagram of the process of a data security protection system based on a reinforcement learning large model provided by an embodiment of the present invention. The system extracts risk indicators based on topological subsets, calculates adjustment factors, and fuses the adjustment factors with the feature vectors of data units to generate a risk enhancement feature matrix. Detailed Implementation

[0054] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0055] like Figure 1 As shown, embodiments of the present invention propose a data security protection system based on a large reinforcement learning model, comprising:

[0056] The acquisition module is used to acquire multi-source heterogeneous raw data streams, and uses protocol fingerprint parsing and adaptive feature encoding to generate a set of independent data units and extract the feature vectors of the data units;

[0057] The calculation module is used to generate topological space coordinate vectors using a topological space mapping algorithm with data unit sets as nodes, and to calculate threat propagation weights to generate topological subsets.

[0058] The fusion module is used to extract risk indicators based on topological subsets, calculate adjustment factors, and fuse the adjustment factors with the feature vectors of data units to generate a risk-enhanced feature matrix.

[0059] The extraction module is used to input the risk enhancement feature matrix into the data platform, map it to semantic nodes through the domain knowledge graph and inject topological risk identifiers to generate knowledge-based data entities, and start the security situation awareness device to generate real-time environmental baseline offset, and extract the corresponding attack behavior sequence features by associating with the historical attack chain feature library.

[0060] The execution module is used to integrate knowledge-based data entities, real-time environmental baseline offsets, and historical attack chain features to construct a comprehensive security status representation; and to generate a set of protection strategy instructions from the security status representation through a pre-trained reinforcement learning model.

[0061] The deployment module is used to deploy protection policy instruction sets in a zero-trust architecture, and collect policy activation latency, security entropy gradient and defense effectiveness indicators to generate feedback tensors.

[0062] The reinforcement module is used to construct a reward function based on the feedback tensor and to update the policy network parameters of the reinforcement learning model using a proximal policy optimization algorithm to form a closed loop.

[0063] In this embodiment of the invention, a risk enhancement feature matrix is ​​generated by using a topological space mapping algorithm to generate topological space coordinate vectors and calculate threat propagation weights. This matrix is ​​then combined with risk indicator adjustment factors and data unit feature vectors to accurately capture the correlation between data units and potential risks, thereby improving the accuracy and comprehensiveness of risk assessment and enabling the timely detection of hidden security threats.

[0064] Furthermore, by integrating knowledge-based data entities, real-time environmental baseline offsets, and historical attack chain features to construct a comprehensive security status representation, and by generating a set of protection strategy instructions through a pre-trained reinforcement learning model, the protection strategies can fully combine real-time environmental conditions and historical attack experience, making them more aligned with actual security needs and improving the pertinence and effectiveness of protection measures.

[0065] In a preferred embodiment of the present invention, the process of acquiring multi-source heterogeneous raw data streams, generating an independent data unit set using protocol fingerprint parsing and adaptive feature encoding, and extracting feature vectors from the data units includes:

[0066] The protocol signature is extracted from the multi-source heterogeneous raw data stream, and the signature is matched based on the pre-built protocol fingerprint library to generate a protocol identifier; the corresponding protocol parsing rules are loaded according to the protocol identifier, and the data stream is deconstructed in layers to generate a structured protocol metadata set.

[0067] The protocol metadata set is standardized to generate standardized data units; an adaptive feature encoder is constructed based on the spatiotemporal distribution characteristics of the standardized data units to obtain dimensionality-reduced feature vectors, statistical feature vectors, and fused feature vectors.

[0068] The dimensionality-reduced feature vector, statistical feature vector, and fused feature vector are aligned and concatenated to generate a data unit feature vector; all data unit feature vectors are then fused to generate an independent set of data units, and the data unit feature vectors are extracted.

[0069] In this embodiment of the invention, key feature codes such as header identifiers and specific field sequences are extracted from multi-source heterogeneous raw data streams, such as network traffic, log files, and database operation records. The extracted feature codes are matched with a pre-set protocol fingerprint library, which contains feature templates of hundreds to thousands of protocols such as TCP, HTTP, and MQTT. The feature overlap is calculated, with a value ranging from 0 to 100%. When the overlap is ≥85%, the match is considered successful, and a unique protocol identifier, such as the string identifier HTTP / 1.1 or Modbus-TCP, is generated.

[0070] Based on the protocol identifier, corresponding parsing rules are loaded. For example, HTTP includes hierarchical definitions such as request line, header fields, and message body. The data stream is parsed layer by layer according to the physical layer (e.g., frame length), network layer (e.g., IP address, port number), and application layer (e.g., HTTP method, status code). Each layer extracts 3 to 15 core fields, such as the source IP, destination IP, and TTL value from the IP layer. This results in a structured protocol metadata set containing 10 to 40 fields. The protocol metadata set is then subjected to format standardization processing. Time fields, such as request time, are uniformly converted to UTC millisecond timestamps. Numeric fields, such as port numbers, retain their original range (0-65535) but are uniformly formatted as integers. Text fields, such as request URLs, are uniformly converted to UTF-8 encoding. Missing fields are filled with preset default values, ultimately generating a standardized set with consistent field format and value range. Data Unit; Adaptive Feature Encoding to Generate Sub-Vectors: An encoder is constructed based on the spatiotemporal characteristics of standardized data units (temporal characteristics such as generation interval of 0.001 seconds to 1 hour, spatial characteristics such as the distribution of the network segment to which the source IP belongs); Dimensionality Reduction Feature Vectors: 5 to 10 core distinguishing features are extracted, such as the source IP network segment, port number range, and request frequency, and the original field values ​​are mapped to the range of [-1, 1] (e.g., high-frequency requests are mapped to values ​​close to 1); Statistical Feature Vectors: The mean, variance, and peak value of nearly 100 similar data units are calculated, with the value range following the distribution of the original data, such as the mean in the range of 0.1 to 10 seconds and the variance in the range of 0.01 to 100; Fusion Feature Vectors: Combining spatiotemporal correlation, such as the overlap of protocol type with the first 5 data units and the consistency of IP location, 5 to 10 dimensional features are generated, with the value range of 0 to 1 (0 indicates no correlation, 1 indicates complete correlation).

[0071] The three types of sub-vectors—dimensionality reduction, statistics, and fusion—are aligned in dimension, such as being uniformly adjusted to 10 dimensions, with any insufficient dimensions filled with 0, and then concatenated to form a 30-dimensional data unit feature vector (the value range of each dimension inherits from the corresponding sub-vector, such as some dimensions [-1, 1], some [0, 1], and some the original range of statistics); the data units are grouped according to their logical correlation to generate an independent data unit set containing 50 to 1000 feature vectors, and the feature vector corresponding to each unit is extracted.

[0072] It achieves unified parsing and standardization of multi-source heterogeneous data, solving the compatibility problem of data with different protocols and formats; combined with adaptive feature encoding based on spatiotemporal characteristics, it can not only retain key distinguishing features (dimensionality reduction vectors), but also reflect the statistical regularity (statistical vectors) and correlation of data (fusion vectors), improving the comprehensiveness and accuracy of feature expression; the dimension of the feature vectors is controllable and the value range is clear, improving the system processing efficiency.

[0073] In a preferred embodiment of the present invention, using a set of data units as nodes, a topological space coordinate vector is generated through a topological space mapping algorithm, and threat propagation weights are calculated to generate a topological subset, including:

[0074] A node feature matrix is ​​constructed based on a set of independent data units; a multi-scale analysis of the similarity of feature dimensions and the correlation of time dimensions is performed on the node feature matrix to generate a node association strength matrix;

[0075] The node association strength matrix is ​​mapped to a low-dimensional space to generate a topological space coordinate vector through a manifold learning algorithm that preserves the local adjacency structure.

[0076] The original threat propagation weights are generated by analyzing the spatial geometric relationships between nodes based on topological coordinate vectors and integrating historical threat interaction records; attenuation compensation is applied to the original threat propagation weights to generate standardized threat propagation weights.

[0077] Based on standardized threat propagation weights, the final topological subset is generated by dividing the data into high-risk propagation clusters, medium-risk subsets, and independent subsets using dynamic connection thresholds.

[0078] In this embodiment of the invention, each data unit in the set of independent data units is taken as a node, with 50 to 1000 nodes. Each node corresponds to a 30-dimensional data unit feature vector. The value range of each dimension inherits the previous result. For example, some dimensions are [-1, 1], some are [0, 1], and some are the original range of statistics. A node feature matrix with a dimension of node number × 30 is constructed. The process of generating the node association strength matrix is ​​to calculate the cosine similarity of the feature vectors of any two nodes, with a value range of [-1, 1]. The closer it is to 1, the more similar the features are. The UTC millisecond-level timestamp of the data unit corresponding to the node is extracted, and the exponential decay coefficient of the time interval is calculated (the coefficient is ≥0.9 when the time interval is ≤10 seconds; the coefficient is 0.6 to 0.9 when the time interval is 10 to 60 seconds; and the coefficient is ≤0.1 when the time interval is more than 1 hour), with a value range of [0, 1].

[0079] Multi-scale fusion is a weighted average of the two indicators mentioned above (60% for feature similarity and 40% for time correlation) to obtain the correlation strength between nodes, with a value range of [0,1] (0 indicates no correlation and 1 indicates extremely strong correlation), forming a correlation strength matrix of node number × node number, with diagonal elements of 1 (the correlation strength between a node and itself is 1).

[0080] Topological spatial coordinate vectors are generated through manifold learning: based on the association strength matrix, a nearest neighbor threshold is set for each scenario (threshold 0.6 when the total number of nodes is ≤100; 0.65 when the total number of nodes is 101 to 500; 0.7 when the total number of nodes is >500). For each node, nearest neighbors with an association strength ≥ the threshold are selected (3 to 15 neighbors, if there are less than 3 neighbors, the top 3 neighbors with the highest strength are selected, and the lowest strength is not less than 0.5). High-dimensional conditional probability calculation: for node i and its nearest neighbor j, the association strength is used as the base value (0.5 to 1), and the influence of time difference is added (10% increase for ≤10 seconds; no change for 10 to 60 seconds; 20% decrease for >60 seconds), and normalized to [0.1, 0.9]. The probability of non-nearest neighbor nodes is set to 0.01 to 0.05 (linearly distributed according to the association strength). Initial coordinates are randomly generated in two-dimensional space, with the x-axis and y-axis both taking values ​​of [-8, -2] ∪ [2, 8], and the initial distance between any two nodes is ≥3.

[0081] Iterative optimization: Adjust coordinates in stages to make the low-dimensional probability approximate the high-dimensional probability: iterations 1 to 100 (learning rate 200 to 500), iterations 101 to 500 (learning rate 50 to 100); nearest neighbor nodes need to satisfy the following conditions: when the high-dimensional probability is ≥0.8, the low-dimensional distance is ≤1.5; when it is 0.6 to 0.8, the distance is 1.5 to 3; iterate until the KL divergence change is ≤0.001 for 20 consecutive iterations (final divergence 0.1 to 0.5); linearly scale the optimized coordinates. For x∈[-10, 10] and y∈[-10, 10], ensure that the average distance between nearest neighbors is ≤ 1 / 3 of that between non-nearest neighbors, and output the topological spatial coordinate vector; analyze the Euclidean distance between nodes (base value ≥ 0.8 when distance ≤ 1; base value ≤ 0.2 when distance ≥ 5), integrate historical threat interaction records (add 0.2 to 0.5 if there are records, add 0 to 0.1 if there are no records), and perform weighted summation (70% for spatial relationship and 30% for historical record), with a value range of [0, 1]. Standardize the threat propagation weight: apply time decay to the original weight (decay by 10% to 15% every 24 hours, decay coefficient 0.85 to 0.9 / day), and normalize to [0, 1] (0 indicates no propagation possibility, 1 indicates immediate high risk).

[0082] Based on standardized weights, nodes are divided using dynamic thresholds. High-risk propagation clusters have a threshold of 0.7 to 0.9 (reduced to 0.6 when the risk level is high). Nodes with weights ≥ the threshold form connected clusters (each cluster has 3 to 50 nodes). Medium-risk subsets have a threshold of 0.3 to 0.7. Nodes with weights in this range form subsets (each subset has 2 to 20 nodes). Independent subsets are created by dividing nodes with weights < 0.3 separately. The final result is a topological subset containing these three types of subsets.

[0083] By constructing a correlation strength matrix through multi-scale correlation analysis, the potential correlations between nodes are accurately captured, providing a reliable foundation for topology mapping. Low-dimensional topology mapping preserves local adjacency structures, making spatial relationships between nodes visible and computable, facilitating the analysis of spatial paths for threat propagation. Threat propagation weights integrate spatial geometric relationships and historical records, and are dynamically adjusted through attenuation compensation to improve the timeliness and accuracy of weight calculation. Topology subsets divided by dynamic thresholds can specifically focus on high-risk propagation clusters, improving the efficiency of threat propagation path identification.

[0084] like Figure 2 As shown, in another preferred embodiment of the present invention, risk indicators are extracted based on topological subsets to calculate adjustment factors, and the adjustment factors are fused with data unit feature vectors to generate a risk enhancement feature matrix, including:

[0085] Based on topological subsets, threat concentration indicators of high-risk propagation clusters, propagation potential indicators of medium-risk subsets, and isolation safety indicators of independent subsets are extracted to generate a basic risk indicator set;

[0086] Injecting a set of basic risk indicators into the severity coefficient of real-time threat intelligence and implementing time decay compensation of historical indicators to generate dynamically corrected risk indicators.

[0087] The S-shaped function transformation is applied to the dynamically corrected risk indicators to generate normalized risk values; the density index of high-risk propagation clusters and the diffusion rate index of medium-risk subsets are extracted based on topological subsets to generate environmental threat coefficients; the boundary values ​​of the adjustment interval are calculated based on the environmental threat coefficients; and the normalized risk values ​​are linearly mapped to the adjustment intervals to generate risk adjustment factors.

[0088] The feature vectors of data units are divided into network layer, transport layer and application layer feature channels according to the protocol stack level; the weighted features of the three layers are fused to generate a risk enhancement feature matrix.

[0089] In this embodiment of the invention, the threat concentration index (high-risk propagation cluster) is calculated by multiplying the number of nodes in each high-risk cluster by the average threat propagation weight (0.7 to 1) within the cluster, and then dividing by the average topological distance between nodes within the cluster. The value ranges from [0.5, 2.0], with higher values ​​indicating higher threat aggregation within the cluster. The propagation potential index (medium-risk subset) is calculated by summing the number of effective connections between nodes within the medium-risk subset (connections with a weight ≥ 0.3, 2 to 30) with the connection growth rate over the past 10 minutes (0 to 0.5 / minute). The value ranges from [0.2, 5.0] (higher values ​​indicate a higher potential for propagation). The higher the risk level, the better the isolation and the lower the risk. The isolation security index (independent subset) is the ratio of the minimum topological distance (≥5) between an independent node and all other nodes to the average threat propagation weight (<0.3), with a value range of [1.0, 10.0]. The higher the value, the better the isolation and the lower the risk. The three together constitute the basic risk index set. Injection severity coefficient: Based on real-time threat intelligence, such as intelligence levels divided into low, medium, high, and urgent, a severity coefficient of 0.8, 1.0, 1.2, and 1.5 is assigned respectively. This is multiplied by the basic risk index, such as high-risk cluster threat concentration × 1.2, medium-risk propagation potential × 1.0.

[0090] Time decay compensation: Historical indicators (records exceeding 1 hour) are decayed over time (5% to 10% decay per hour, decay coefficient 0.9 to 0.95 / hour). The final dynamic correction risk indicator values ​​are: threat concentration [0.4, 3.0], transmission potential [0.16, 7.5], and isolation security [0.8, 15.0]. Normalized risk value: The dynamic correction risk indicator is transformed by an S-shaped function (compressed to the [0, 1] interval), where the normalized value is ≥0.8 when the threat concentration is ≥2.0, ≥0.8 when the transmission potential is ≥5.0, and ≥0.8 when the isolation security is ≥10.0 (inverse indicator, the higher the value, the lower the risk).

[0091] The specific calculation process of the S-shaped function transformation is as follows: for dynamically corrected risk indicators (threat concentration [0.4, 3.0], transmission potential [0.16, 7.5], isolation security [0.8, 15.0]), the S-shaped function is used to map them to the interval [0, 1] in stages, as follows:

[0092] Threat Concentration Index (positive indicator, higher values ​​indicate higher risk): Input value ≤ 0.8 indicates low concentration: output value slowly increases from 0.1 to 0.3 as input increases (the curve is gentle, avoiding over-amplification of low risk); Input value 0.8 to 2.0 indicates medium concentration: output value rapidly increases from 0.3 to 0.8, the curve is steep, highlighting the differentiation of medium risk; Input value ≥ 2.0 indicates high concentration: output value slowly approaches 1.0 as input increases, the curve is gentle, maintaining a gradient in the high-risk range without over-compression. Transmission Potential Index (positive indicator, higher values ​​indicate higher risk): Input value ≤ 1.0 indicates low potential: output value increases from 0.1 to 0.2 (compressing the weight of low potential values); Input value 1.0 to 5.0 indicates medium to high potential: output value rapidly increases from 0.2 to 0.8 (focusing on distinguishing medium diffusion risk); Input value ≥ 5.0 indicates high potential: output value slowly approaches 1.0 from 0.8 (maintaining a gradient for high diffusion risk).

[0093] The isolation safety index is defined as follows: low isolation when the input value is ≤2.0, the output value increases from 0.8 to 1.0, indicating high risk and corresponding to high output; medium isolation when the input value is 2.0 to 10.0, the output value drops rapidly from 0.8 to 0.2, highlighting the impact of isolation changes on risk; and high isolation when the input value is ≥10.0, the output value slowly approaches 0.0 from 0.2, indicating low risk and corresponding to low output. After transformation, the final normalized risk value of all indicators strictly falls within the range of [0, 1], and maintains the relative magnitude relationship of the original indicators. For positive indicators, an increase in input leads to an increase in output, and vice versa for negative indicators.

[0094] High-risk cluster density index: the number of high-risk clusters (1 to 5) divided by the total number of nodes (50 to 1000), with a value range of [0.002, 0.1]; medium-risk diffusion rate index: the growth rate of the number of connections of medium-risk subsets (0 to 0.5 / minute) multiplied by the duration (1 to 30 minutes), with a value range of [0, 15]; the two are weighted and summed (density accounts for 40%, rate accounts for 60%) to obtain the environmental threat coefficient, with a value range of [0, 9.04].

[0095] The interval boundary adjustment is determined based on the environmental threat coefficient, where the interval is [0.8, 1.2] when the coefficient is ≤3; [0.6, 1.4] when the coefficient is 3 to 6; and [0.4, 1.6] when the coefficient is >6 (the higher the threat, the wider the interval). The normalized risk value is linearly mapped to the corresponding interval, such as a normalized value of 0 corresponding to the lower limit of the interval and 1 corresponding to the upper limit of the interval, generating a risk adjustment factor with a value range of [0.4, 1.6], where a value >1 indicates enhanced risk weight and <1 indicates reduced risk weight. Feature channel partitioning involves dividing the 30-dimensional data unit feature vector into network layer, transport layer, and application layer according to the protocol stack. The feature vector of each layer is weighted and multiplied by the corresponding risk adjustment factor (network layer × threat concentration factor, transport layer × propagation potential factor, application layer × isolation security factor), and then concatenated to form a 30-dimensional enhanced feature vector. The enhanced feature vectors of all nodes are combined to form a risk enhancement feature matrix of node number × 30.

[0096] Basic risk indicators are extracted hierarchically by combining topological subset characteristics to achieve targeted risk quantification and avoid the one-sidedness of a single indicator; dynamic correction introduces real-time intelligence and time decay, so that risk indicators are dynamically updated with environmental changes, improving timeliness and accuracy; risk adjustment factors are dynamically adjusted in range through environmental threat coefficients to achieve adaptive calibration of risk weights and enhance adaptability to complex threat environments; features and risk factors are fused hierarchically according to protocol stack, retaining the original physical meaning of features while injecting risk attributes, making the feature matrix more in line with security analysis needs.

[0097] In a preferred embodiment of the present invention, the risk enhancement feature matrix is ​​input into the data platform, mapped to semantic nodes through a domain knowledge graph and injected with topological risk identifiers to generate knowledge-based data entities. A security situation awareness device is then activated to generate a real-time environmental baseline offset, and corresponding attack behavior sequence features are extracted by associating with a historical attack chain feature library, including:

[0098] The risk enhancement feature matrix is ​​input into the structure aligner of the data platform, matched with domain knowledge graph nodes to generate an initial semantic node set, and then similar to the domain knowledge graph nodes to generate an optimized semantic node set. Context consistency verification is performed on the initial semantic node set to generate an optimized semantic node set.

[0099] Based on standardized threat propagation weights, topological risk identifiers are injected into the optimized semantic node set to generate semantic nodes with risk identifiers; the semantic nodes with risk identifiers are encapsulated to generate knowledge-based data entities.

[0100] The security situation awareness device is activated based on knowledge-based data entities to generate real-time environmental baseline offsets; historical attack chain feature databases are retrieved based on the semantic node identifiers of knowledge-based data entities, and attack behavior sequence features are extracted.

[0101] In this embodiment of the invention, the risk enhancement feature matrix, with a dimension of node number × 30, is input into the structure aligner of the data platform. The aligner first extracts the core attributes of each feature vector in the matrix, such as protocol type, IP network segment, risk weight, etc., 10 to 15 key features. The domain knowledge graph contains thousands of nodes, including network attacks, protocol specifications, asset attributes, etc. Each node contains 20 to 50 attributes. Dimensional matching is performed on the node attributes, such as the HTTP protocol in the features and the protocol type attribute of the application layer protocol node in the corresponding graph. The similarity between the matched features and node attributes is calculated (based on attribute value overlap and semantic relevance, with a value range of 0 to 1). A similarity threshold ≥ 0.7 is set to determine successful matching. Each data unit node generates 1 to 3 candidate semantic nodes, which are integrated to form an initial semantic node set.

[0102] Perform context consistency checks on the initial set of semantic nodes:

[0103] Verify the logical association between nodes. For example, TCP protocol nodes should not be strongly associated with UDP port attributes. If the association strength is ≥0.6, they are considered inconsistent. Verify the rationality of attribute values. For example, when the source IP network segment is 192.168.xx, the asset type should not be marked as a public server. Nodes with logical conflicts are included in the list to be corrected. For inconsistent nodes, correct them by supplementing feature matching (adding 3 to 5 auxiliary attribute verifications) or replacing them with nodes with the second highest similarity. If the similarity is ≥0.6, remove nodes that cannot be corrected, accounting for ≤5%. Finally, generate an optimized semantic node set (node ​​integrity ≥95%).

[0104] Based on standardized threat propagation weights, with values ​​ranging from 0 to 1, topological risk identifiers are injected into the optimized semantic node set: a weight ≥ 0.7 indicates high topological risk; a weight between 0.3 and 0.7 indicates medium topological risk; and a weight < 0.3 indicates low topological risk. Semantic nodes with risk identifiers are encapsulated to generate knowledge-based data entities: each entity contains structured information such as node ID, core semantic attributes (e.g., HTTP abnormal requests, topological risk identifiers), a list of associated node IDs (nodes with association strength ≥ 0.5), and a risk enhancement feature summary. The entity format is uniformly set to an extensible key-value pair format.

[0105] The security situation awareness is activated based on knowledge-based data entities: using knowledge-based data entities from the past 30 days as samples, a normal state baseline is calculated (such as 10 to 15 indicators, including average hourly request volume, percentage of each protocol type, and distribution ratio of topology risk identifiers); the difference between the current entity set and the baseline is compared. If the request volume fluctuates by more than ±30%, the percentage of high topology risk identifiers increases by ≥15%, or the percentage of abnormal protocols (such as protocols not appearing in the baseline) increases by ≥5%, the degree of deviation is quantified, with a value range of 0 to 1, where 0 indicates no deviation and 1 indicates a serious deviation from the baseline. Typically, a deviation of ≥0.3 triggers key attention.

[0106] Based on the semantic node identifiers of knowledge-based data entities, such as SQL injection attempts, port scanning, and abnormal logins, a historical attack chain feature database is retrieved, containing thousands of attack chain records. Each record includes an attack step sequence, involved semantic identifiers, time intervals, and tool characteristics. Historical attack chains with an overlap of ≥0.8 with the current semantic identifier are filtered out. For example, if the current entity contains port scanning and weak password attempts, attack chains with the same identifiers in the history are matched. Core behavioral sequence features are extracted from the matched attack chains, including the order of attack steps (such as information gathering, vulnerability exploitation, and data theft), the time interval pattern of each step (such as the interval between scanning and penetration is usually 1 to 5 minutes), and the topological node association method involved (such as propagation from edge nodes to core nodes). Each sequence contains 5 to 10 key features, forming an attack behavior sequence feature set associated with the current entity.

[0107] Knowledge-based data entities transform low-level features into understandable semantic information through semantic mapping and risk identification, improving data interpretability; context consistency verification reduces semantic matching errors and ensures the accuracy of knowledge-based entities; real-time environmental baseline offset dynamically reflects the difference between the current environment and the normal state, which can quickly identify potential anomalies and improve the timeliness of situational awareness.

[0108] In a preferred embodiment of the present invention, a comprehensive security status representation is constructed by integrating knowledge-based data entities, real-time environmental baseline offsets, and historical attack chain features; the security status representation is then used to generate a set of protection strategy instructions through a pre-trained reinforcement learning model, including:

[0109] Topological semantic deconstruction is performed on knowledge-based data entities to extract entity relationship feature vectors; real-time environment baseline offset is scaled and standardized to generate environment offset features; temporal position encoding is performed on attack behavior sequence features to generate sequence feature vectors.

[0110] The entity relationship feature vector, environmental offset feature, and sequence feature vector are integrated along the feature fusion axis to construct a structured security state body. The structured security state body is input into a pre-trained reinforcement learning model for defense benefit evaluation. Candidate protection sequences are generated based on the defense benefit evaluation results. High-return protection sequences are selected by comparing benefit advantages. The high-return protection sequences are converted into a protection strategy instruction set.

[0111] In this embodiment of the invention, topological semantic deconstruction is performed on knowledge-based data entities:

[0112] The process involves analyzing the types of relationships between entities, such as data transmission, protocol dependencies, and attack attribution (5 to 8 types in total), quantifying relationship features based on relationship strength (0 to 1, inherited topology propagation weight) and relationship stability (stable for ≥5 minutes, assigned a value of 0.8; otherwise, 0.3 to 0.7); extracting the distance and risk combination features between each entity and its associated entities (e.g., a distance of 2 hops corresponds to a high risk of 0.9, and a distance of 1 hop corresponds to a medium risk of 0.6); and integrating these features to generate an entity relationship feature vector with 15 to 20 dimensions, where each dimension has a value range of... 0 to 1 (higher values ​​indicate a more critical relationship); scale the real-time environmental baseline offset, where offset ≤ 0.3 is mapped to [-0.3, 0], indicating a slight offset, close to normal; offset 0.3 to 0.7 is mapped to (0, 0.7], indicating a moderate offset, requiring attention; offset > 0.7 is mapped to (0.7, 1], indicating a severe offset, requiring urgent handling; generate a 1D environmental offset feature with a value range of [-0.3, 1], preserving the offset direction, where negative values ​​indicate below the baseline and positive values ​​indicate above the baseline.

[0113] The temporal position encoding process for attack behavior sequence features is as follows: assign positional weights according to the attack steps, such as information gathering, vulnerability exploitation, and lateral movement (0.1 for step 1, 0.2 for step 2, ..., n×0.1 for step n, with a cumulative weight ≤ 1); multiply the feature values ​​of each step, such as time interval patterns and feature matching degree, by the positional weight to form temporally weighted features; concatenate them into a 10 to 15-dimensional sequence feature vector, with each dimension taking values ​​from 0 to 1, and higher values ​​indicating more obvious features at that temporal position.

[0114] The three types of features are integrated along the feature fusion axis (with a weight ratio of 6:1:3 for entity relationship, environment offset, and sequence features). The entity relationship feature vector is 15 to 20 dimensions, the environment offset feature is 1 dimension, and the sequence feature vector is 10 to 15 dimensions. These are then concatenated into a 30 to 36-dimensional structured vector. The vector is then range-calibrated (mapped to [-0.5, 1.5] to preserve the relative differences between the features) to form a comprehensive security status representation.

[0115] Construction of the pre-trained reinforcement learning large model: An Actor-Critic dual-network architecture is adopted. The Actor network (policy network) takes a 30 to 36-dimensional security state representation as input and outputs a probability distribution of protective actions, including 10 to 15 basic actions such as blocking, alarming, traffic restriction, and vulnerability scanning. The Critic network (value network) takes the same input as the Actor and outputs the expected reward value of the state (quantifying the defense effect). The input layer receives the structured security state and extracts features through 3 to 4 fully connected layers (64 to 128 neurons per layer). The output layer uses softmax activation (Actor) or linear activation (Critic) to ensure that the output conforms to the action space distribution (0 to 1 probability) and the reward range (-100 to 100, negative for loss, positive for gain).

[0116] The model training process uses historical security event data and simulated attack and defense data. Each sample contains a triplet of security state, protective action, and actual benefit (the benefit is calculated based on the risk reduction rate and strategy cost, such as +50 for blocking high-risk attacks and -30 for mistakenly blocking normal traffic).

[0117] Pre-training phase: The Proximal Policy Optimization (PPO) algorithm is used, with the initial learning rate set to 0.0003 to 0.001. Iterations are performed for 500 to 1000 epochs, with 1000-2000 samples sampled per epoch. The network parameters are updated by minimizing policy loss (ensuring that the difference between the new and old policies is controllable) and value loss (fitting the actual returns) until the validation set return fluctuation is ≤5%. Fine-tuning phase: Real-time environmental data (security status and policy effects over the past 7 days) are introduced, and the learning rate is adjusted to 0.0001 to 0.0003. Iterations are performed for 100 to 200 epochs to adapt the model to the current environmental characteristics (such as the attack preferences of specific network segments).

[0118] Model implementation process: Deployed on the edge node of the data platform, using distributed inference (supporting 100 to 500 concurrent state inputs), with a single inference latency of ≤100ms; receiving structured security state bodies in real time, outputting action probability distributions through model inference, and generating a candidate action list by sorting by probability.

[0119] Calculate the expected benefit for candidate actions, combining the Critic network output value with real-time environmental costs such as execution time and resource consumption. The benefit range is -50 to 80. Sort the benefits from highest to lowest, generating 3 to 5 protection sequences, each containing 2 to 4 consecutive actions, such as alarms, traffic restrictions, and vulnerability scanning. The sequence length is dynamically adjusted according to the urgency of the threat. Calculate the benefit advantage between sequences, i.e., the difference between the highest and second-highest benefit. When the advantage is ≥10, directly select the highest benefit sequence. Otherwise, further verify the synergistic effect of the sequences, such as whether there are conflicts between actions, and finally select one high-return sequence. Convert the sequence actions into machine-executable instructions, such as blocking IP: 192.168.1.100, port 8080, duration 30 minutes, triggering level 2 alarms, and notifying the security administrator. The instruction format conforms to the API specification of the zero-trust architecture, including instruction ID, execution object, parameters, and priority.

[0120] The comprehensive security status representation integrates entity relationships, environmental offsets, and attack sequence characteristics to achieve a multi-dimensional and structured description of the security situation, avoiding the one-sidedness of a single feature. The pre-trained reinforcement learning model, through historical data and real-time fine-tuning, has the ability to make adaptive decisions in complex threat scenarios, and the generated protection strategies take into account both benefits and costs. The benefit evaluation and synergy effect verification of the protection sequence ensure the high return and feasibility of the strategy and reduce ineffective operations, thereby improving the real-time performance and accuracy of data security protection.

[0121] In a preferred embodiment of the present invention, a protection policy instruction set is deployed in a zero-trust architecture, and a feedback tensor is generated by collecting policy activation latency, security entropy gradient, and defense effectiveness indicators, including:

[0122] Deploy the protection policy instruction set to the policy execution node of the zero-trust architecture; monitor the policy execution node to generate a policy effective delay sequence; perform window standardization processing on the policy effective delay sequence to generate a standardized delay sequence;

[0123] Based on the standardized time delay sequence, the security entropy change rate of new data units is collected during the monitoring cycle. The security entropy change rate is then used to calculate the gradient to generate a security entropy change gradient matrix. During the monitoring cycle, defense effectiveness indicators are statistically analyzed to generate an effectiveness indicator vector. The effectiveness indicator vector is then transformed into a dimensionless form to generate a standardized effectiveness vector.

[0124] Align the standardized time delay sequence, the security entropy gradient matrix, and the standardized performance vector at the time origin; then concatenate the aligned data along the time dimension to generate a feedback tensor.

[0125] In this embodiment of the invention, the protection policy instruction set is deployed to the policy execution nodes of the zero-trust architecture, such as edge firewalls, terminal access proxies, API gateways, etc., with the number of nodes ranging from 3 to 10, which can be dynamically expanded according to the network size.

[0126] The time interval from the moment the instruction is issued to the time interval from the execution node to the return of the effective completion signal is measured in milliseconds (ms). A time delay sequence is generated by collecting data, with sampling every 100ms for 1 to 5 minutes. The sequence length is 60 to 300 data points. The normal time delay range is 50 to 500ms, with priority level 1 instructions ≤ 200ms and priority level 5 instructions ≤ 500ms. Abnormal time delays may exceed 1000ms.

[0127] Window normalization uses a sliding window with a window size of 5 to 10 data points to smooth the time delay sequence, remove extreme values ​​that exceed the mean ± 3 times the standard deviation (retaining more than 95% of the valid data), and then linearly map the sequence to the interval [0, 1]. A time delay of 50ms corresponds to 0, 500ms corresponds to 1, and anything exceeding 500ms is treated as 1, thus generating a normalized time delay sequence.

[0128] Calculate the security entropy change gradient matrix: synchronized with the policy activation delay monitoring, with a cycle of 1 to 5 minutes (matching the policy's impact range, 1 minute for high-risk policies, and 5 minutes for medium and low-risk policies); within the monitoring cycle, collect the security entropy value of a new data unit every 10 seconds, and calculate the rate of change between two adjacent entropy values, where the entropy change rate = (current entropy value - previous entropy value) / time interval, unit: entropy value / second, value range [-0.5, 0.3], negative indicates entropy decrease, the system is more orderly; positive indicates entropy increase, the risk increases.

[0129] Generate a safety entropy change gradient matrix: Calculate the gradient of the entropy change rate, take the average slope of every 3 consecutive entropy change rates to reflect the trend of change, and form a gradient matrix. The dimension is the number of monitoring cycle steps × 1, and the gradient value range is [-0.1, 0.1]. The larger the absolute value of the negative gradient, the more obvious the entropy reduction trend and the better the strategy effect.

[0130] During the monitoring period, 3 to 5 core performance indicators are calculated: threat blocking rate (number of successfully blocked threats / total number of detected threats, ranging from 0 to 100%); false alarm rate (number of normal behaviors mistakenly identified as threats / total number of detected behaviors, ranging from 0 to 5%); resource utilization rate (CPU / memory resources used for policy execution, ranging from 5% to 30%); and business impact (the percentage increase in latency for normal business operations due to the policy, ranging from 0 to 20%).

[0131] Dimensionless transformation: The indicators are standardized. Positive indicators, such as blocking rate, are mapped to [0, 1], and negative indicators, such as false alarm rate and resource utilization rate, are mapped to [0, 1], forming an effectiveness indicator vector (dimensions 3 to 5), with values ​​ranging from 0 to 1.

[0132] Using the strategy deployment time as the time origin (t=0), the standardized time delay sequence (timestamp t=0 to t=end of the cycle), the security entropy change gradient matrix, and the standardized performance vector are aligned according to the timestamp to ensure that the data is synchronized in the time dimension.

[0133] Data is stitched and aligned along the time dimension, i.e. the first dimension. The feedback tensor dimension is the number of monitoring cycle steps × (time delay sequence dimension + gradient matrix dimension + performance vector dimension). For example, in a 30-step cycle, the time delay is 1 dimension + the gradient is 1 dimension + the performance is 4 dimensions, and the tensor is 30×6. All element values ​​are uniformly mapped to [-1, 1]. The time delay and performance positive indicators are 0 to 1, and the entropy change gradient and performance negative indicators are -1 to 0.

[0134] Feedback tensor fusion comprehensively reflects the execution quality of the strategy across three dimensions: strategy activation latency, security entropy gradient, and defense effectiveness, avoiding the limitations of a single indicator. Standardized processing allows for the direct fusion of indicators of different magnitudes and types, providing a unified scale of feedback signals for reinforcement learning models and improving the accuracy of model parameter updates. Adapting to dynamic deployment scenarios in zero-trust architectures, feedback data can be directly used for closed-loop optimization, driving the protection strategy to continuously adapt to the network environment and business needs, and enhancing its self-evolution capability.

[0135] In a preferred embodiment of the present invention, a reward function is constructed based on a feedback tensor, and the policy network parameters of the reinforcement learning model are updated using a proximal policy optimization algorithm to form a closed loop, including:

[0136] Based on the feedback tensor, a standardized time delay sequence is separated. Based on the preset time delay benchmark value, the timeliness of the standardized time delay sequence is evaluated to generate a timeliness quantification index. The timeliness quantification index is compared with the timeliness threshold to generate a timeliness reward component.

[0137] Based on the feedback tensor, the security entropy change gradient matrix is ​​separated. The security entropy change gradient matrix is ​​then used for threat diffusion analysis to generate a threat situation quantitative index. The threat situation quantitative index is compared with the threat suppression threshold to generate a threat suppression reward component.

[0138] Based on the time series feedback tensor, a standardized performance vector is separated. The standardized performance vector is then used to analyze the defense effect and generate a quantitative index of defense effect. The quantitative index of defense effect is compared with the defense performance threshold to generate a defense effect reward component.

[0139] The timeliness reward component, threat suppression reward component, and defense effect reward component are weighted and fused to generate a comprehensive reward value; based on the comprehensive reward value, the near-end policy optimization algorithm is used to calculate the adjustment amount of policy network parameters;

[0140] The application parameter adjustment amount updates the policy network parameters of the reinforcement learning large model, forming a closed loop.

[0141] In this embodiment of the invention, a standardized time delay sequence is extracted from the feedback tensor, with values ​​ranging from 0 to 1, where 0 corresponds to an optimal time delay of 50ms and 1 corresponds to a worst time delay of more than 500ms; a preset time delay benchmark value is dynamically adjusted according to the strategy priority: 0.3 for level 1 strategy, 0.5 for levels 2 to 3, and 0.7 for levels 4 to 5. The timeliness quantification index is calculated as 1 - (mean of standardized time delay sequence ÷ benchmark value), with a value range of [-2, 1] (positive when mean ≤ benchmark value, negative otherwise).

[0142] Set a timeliness threshold of 0 (i.e., the quantitative indicator ≥ 0 is considered to meet the standard), and generate a reward component: when the standard is met, take the quantitative indicator × 10 (range 0 to 10), and when the standard is not met, take the quantitative indicator × 5 (range -10 to 0), with the overall value being [-10, 10].

[0143] Extract the security entropy change gradient matrix from the feedback tensor (values ​​range from -0.1 to 0.1, negative gradients indicate entropy reduction and threat convergence); calculate the threat situation quantification index = proportion of negative gradients in the gradient matrix × (1 - absolute value of average gradient), with a value range of 0 to 1 (the higher the proportion of negative gradients and the larger the absolute value, the closer the index is to 1).

[0144] Set a threat suppression threshold of 0.5 (i.e., an index ≥ 0.5 is considered effective suppression), and generate a reward component: when the target is met, take (index - 0.5) × 20 (range 0 to 10), and when the target is not met, take (index - 0.5) × 10 (range -5 to 0), with an overall value of [-5, 10]. Extract a standardized effectiveness vector from the feedback tensor and calculate the defense effectiveness quantification index = (mean of positive index × 0.7) - (mean of negative index × 0.3), with a value range of -0.3 to 0.7.

[0145] Set a defense effectiveness threshold of 0.3 (i.e., an indicator ≥ 0.3 indicates good effectiveness), and generate reward components: when the target is met, take indicator × 30 (range 9 to 21); when the target is not met, take (indicator - 0.3) × 20 (range -6 to 0), with an overall value of [-6, 21]. The three reward components are weighted and integrated (timeliness weight 20%, threat suppression weight 40%, defense effectiveness weight 40%), and the comprehensive reward value = (timeliness component × 0.2) + (threat suppression component × 0.4) + (defense effectiveness component × 0.4), with a value range of [-6.4, 16.8] (negative values ​​indicate poor strategy effectiveness, and positive values ​​indicate good effectiveness).

[0146] The advantage function (the difference between the current reward and the historical average reward, ranging from -20 to -20) is calculated based on the comprehensive reward value to measure the room for policy improvement. The probability ratio between the new and old policies is limited to the range of [0.8, 1.2] (to avoid parameter abrupt changes). The policy network parameters are adjusted according to the sign of the advantage function: when there is a positive advantage, the probability of high-reward actions is increased, and when there is a negative advantage, the probability of low-reward actions is weakened. The parameter adjustment amount is updated gradually according to the learning rate (dynamically set from 0.0001 to 0.0005, taking the smaller value when the reward fluctuates greatly), and the single adjustment range is ≤5%.

[0147] The adjusted parameters are applied to the policy network of the large reinforcement learning model, and the new policy takes effect in the next round of security decision-making, repeating the decision-making, execution, feedback, and update process.

[0148] The reward function integrates three dimensions of indicators: timeliness, threat suppression, and defense effectiveness, to comprehensively quantify the merits of the strategy and avoid decision-making bias caused by a single indicator. Each reward component achieves differentiated rewards and punishments through threshold comparison, which strengthens the model's learning priority for key objectives (such as threat suppression). The near-end policy optimization algorithm ensures the stability of model updates by limiting the range of parameter adjustments, and avoids policy oscillations caused by feedback noise.

[0149] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the system as described above. All implementations in the above system embodiments are applicable to this embodiment and can achieve the same technical effects.

[0150] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the system as described above. All implementations in the above system embodiments are applicable to this embodiment and can achieve the same technical effects.

[0151] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data security protection system based on a reinforcement learning large model, characterized in that, The method comprises the following steps: A collection module is used to acquire multi-source heterogeneous original data streams, protocol fingerprint analysis and adaptive feature coding are used to generate independent data unit set and extract data unit feature vector; A calculation module is used to take the data unit set as a node, generate a topological space coordinate vector through a topological space mapping algorithm, calculate a threat propagation weight, and generate a topological subset; A fusion module is used to extract a risk index adjustment factor based on the topological subset, fuse the adjustment factor with the data unit feature vector to generate a risk enhanced feature matrix; An extraction module is used to input the risk enhanced feature matrix into a data platform, map to a semantic node through a domain knowledge graph, inject a topological risk identifier, generate a knowledge data entity, and start a security situation awareness device to generate a real-time environment baseline offset, and extract corresponding attack behavior sequence features from a historical attack chain feature library; An execution module is used to fuse the knowledge data entity, real-time environment baseline offset and historical attack chain features to construct a comprehensive security state representation; the security state representation is used to generate a protection strategy instruction set through a pre-trained reinforcement learning large model; A deployment module is used to deploy the protection strategy instruction set in a zero trust architecture, collect the policy effective time delay, security entropy gradient and defense efficiency index to generate a feedback tensor; A reinforcement module is used to construct a reward function based on the feedback tensor, update the policy network parameters of the reinforcement learning model using a proximal policy optimization algorithm to form a closed loop.

2. The data security protection system based on the reinforcement learning large model according to claim 1, wherein, The method comprises the following steps: Protocol feature codes are extracted from the multi-source heterogeneous original data streams, and feature code matching is performed based on a preconfigured protocol fingerprint library to generate a protocol identifier; corresponding protocol parsing rules are loaded according to the protocol identifier, layered deconstruction is performed on the data stream, and a structured protocol metadata set is generated; Standardized conversion is performed on the protocol metadata set to generate standardized data units; an adaptive feature encoder is constructed based on the spatiotemporal distribution characteristics of the standardized data units to obtain a dimension reduction feature vector, a statistical feature vector and a fusion feature vector; The dimension reduction feature vector, the statistical feature vector and the fusion feature vector are dimensionally aligned and spliced to generate a data unit feature vector; all data unit feature vectors are fused to generate an independent data unit set and extract data unit feature vectors.

3. The data security protection system based on the reinforcement learning large model according to claim 2, characterized in that, The method comprises the following steps: A node feature matrix is constructed based on the independent data unit set; a node correlation strength matrix is generated by performing multi-scale analysis on the feature dimension similarity and time dimension correlation of the node feature matrix; The node correlation strength matrix is mapped to a low-dimensional space through a manifold learning algorithm that preserves local adjacency structure to generate a topological space coordinate vector; The spatial geometric relationship between nodes is analyzed based on the topological space coordinate vector, and the original threat propagation weight is generated by fusing historical threat interaction records; the original threat propagation weight is compensated by attenuation to generate a standardized threat propagation weight; The final topology subset is generated by dividing high-risk propagation clusters, medium-risk subsets and independent subsets through a dynamic connection threshold according to a standardized threat propagation weight.

4. The data security protection system based on the reinforcement learning large model according to claim 3, characterized in that, Based on the topology subset, an adjustment factor is calculated by extracting a risk index, and the adjustment factor is fused with a data unit feature vector to generate a risk-enhanced feature matrix, including: Based on the topology subset, a basic risk index set is generated by extracting a threat concentration index of the high-risk propagation cluster, a propagation potential index of the medium-risk subset and an isolation security index of the independent subset; The basic risk index set is injected into a severity coefficient of real-time threat intelligence, and a time decay compensation of historical indexes is implemented to generate a dynamically revised risk index; An S-shaped function transformation is performed on the dynamically revised risk index to generate a normalized risk value; based on the topology subset, an environment threat coefficient is generated by extracting a high-risk propagation cluster density index and a medium-risk subset diffusion rate index; the adjustment interval boundary value is calculated according to the environment threat coefficient; and the risk adjustment factor is generated by linearly mapping the normalized risk value to the adjustment interval. The data unit feature vector is divided into network layer, transport layer and application layer feature channels according to the protocol stack level; and a risk-enhanced feature matrix is generated by fusing the weighted features of the three layers.

5. The data security protection system based on the reinforcement learning large model according to claim 4, characterized in that, The risk-enhanced feature matrix is input into a data hub, mapped to a semantic node through a domain knowledge graph, and injected with a topology risk identifier to generate a knowledge-based data entity, and a security situation awareness device is started to generate a real-time environment baseline offset, and the corresponding attack behavior sequence features are extracted by correlating the historical attack chain feature library, including: The risk-enhanced feature matrix is input into the structure aligner of the data hub, matches the domain knowledge graph nodes to generate an initial semantic node set, and the similarity matching is performed with the domain knowledge graph nodes to generate an initial semantic node set; the context consistency verification is performed on the initial semantic node set to generate an optimized semantic node set; Based on the standardized threat propagation weight, the optimized semantic node set is injected with a topology risk identifier to generate a semantic node with a risk identifier; and the semantic node with the risk identifier is encapsulated to generate a knowledge-based data entity; Based on the knowledge-based data entity, a security situation awareness device is started to generate a real-time environment baseline offset; and the attack behavior sequence features are extracted by searching the historical attack chain feature library according to the semantic node identifier of the knowledge-based data entity.

6. The data security protection system based on the reinforcement learning large model according to claim 5, characterized in that, The knowledge-based data entity, real-time environment baseline offset and historical attack chain features are fused to construct a comprehensive security state representation; and the security state representation is input into a pre-trained reinforcement learning model to generate a protection strategy instruction set, including: The knowledge-based data entity is topologically semantically deconstructed to extract an entity relationship feature vector; the real-time environment baseline offset is standardized by scaling to generate an environment offset feature; and the attack behavior sequence features are time-series position coded to generate a sequence feature vector; The entity relationship feature vector, environment offset feature and sequence feature vector are integrated along the feature fusion axis to construct a structured security state body; the structured security state body is input into a pre-trained reinforcement learning model for defense benefit evaluation; a candidate protection sequence is generated based on the defense benefit evaluation result; a high-return protection sequence is selected by comparing the benefits; and the high-return protection sequence is converted into a protection strategy instruction set.

7. The data security protection system based on the reinforcement learning large model according to claim 6, characterized in that, The zero-trust architecture deploys a protection policy instruction set, collects a policy effective time delay, a security entropy gradient, and a defense performance index to generate a feedback tensor, including: The protection policy instruction set is deployed to a policy execution node of the zero-trust architecture; the policy execution node is monitored to generate a policy effective time delay sequence; the policy effective time delay sequence is subjected to window standardization processing to generate a standardized time delay sequence; Based on the monitoring period of the standardized time delay sequence, the security entropy change rate of a new data unit is collected, the security entropy change rate is subjected to gradient calculation to generate a security entropy change gradient matrix, and the defense performance index is counted within the monitoring period to generate a performance index vector; the performance index vector is subjected to dimensionless conversion to generate a standardized performance vector; The standardized time delay sequence, the security entropy change gradient matrix, and the standardized performance vector are aligned according to the time origin; the aligned data is spliced along the time dimension to generate a feedback tensor.

8. The data security protection system based on the reinforcement learning large model according to claim 7, characterized in that, Based on the feedback tensor, a reward function is constructed, and a near-end policy optimization algorithm is used to update the policy network parameters of the reinforcement learning model to form a closed loop, including: The standardized time delay sequence is separated based on the feedback tensor, the standardized time delay sequence is subjected to timeliness evaluation based on a preset time delay reference value, and a timeliness quantitative index is generated; the timeliness quantitative index is compared with a timeliness threshold value to generate a timeliness reward component; The security entropy change gradient matrix is separated based on the feedback tensor, the security entropy change gradient matrix is subjected to threat diffusion analysis to generate a threat situation quantitative index; the threat situation quantitative index is compared with a threat suppression threshold value to generate a threat suppression reward component; The standardized performance vector is separated based on the feedback tensor, the standardized performance vector is subjected to defense effect analysis to generate a defense effect quantitative index; the defense effect quantitative index is compared with a defense performance threshold value to generate a defense effect reward component; The timeliness reward component, the threat suppression reward component, and the defense effect reward component are subjected to weighted fusion to generate a comprehensive reward value; the comprehensive reward value is used to calculate a policy network parameter adjustment amount based on the near-end policy optimization algorithm; The policy network parameters of the reinforcement learning model are updated by using the parameter adjustment amount to form a closed loop.

Images