Privacy computing security protection method based on reinforcement learning

Abstract

The invention discloses a privacy computing security protection method based on reinforcement learning. The method comprises the following steps: S1, establishing a causal relationship model based on privacy computing; s2, collecting operation data in a privacy calculation process, and generating a protection action vector; s3, an anti-fact action is generated under the same exogenous condition, a task efficiency difference value is calculated, and an anti-fact return value is obtained; s4, constructing a comprehensive evaluation index according to the anti-fact return value, the privacy budget consumption and the calculation delay; s5, calculating the change rate of the privacy budget consumption relative to each protection action component, and generating a privacy sensitivity index; s6, generating a privacy shielding matrix, and performing weight reduction or blocking on the high-sensitivity component to obtain a security action vector; and S7, on the basis of the comprehensive evaluation index and the security action vector, generating a privacy calculation security protection strategy. According to the method, the security protection level, the self-adaptive capability and the stability of the privacy computing task are effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of reinforcement learning and privacy computing technology, and in particular to a security protection method for privacy computing based on reinforcement learning. Background Technology

[0002] With the rapid development of data sharing, distributed computing, and artificial intelligence technologies, privacy computing has gradually become an important technical path to ensure data availability and privacy security. Existing privacy computing mainly relies on mechanisms such as differential privacy, homomorphic encryption, and federated learning to add noise, encrypt, or fragment sensitive data in a distributed environment to achieve a "usable but invisible" data utilization model. However, in complex task execution environments, existing privacy computing protection technologies generally suffer from difficulties in coordinating multi-dimensional constraints: the dynamic allocation and consumption monitoring of the privacy budget lack adaptive mechanisms, and protection strategies often rely on fixed thresholds and manual rules. When task load, data sensitivity, or communication latency changes, the system struggles to adjust the protection strength in a timely manner, leading to excessive consumption of the privacy budget or insufficient protection.

[0003] Furthermore, existing privacy-preserving computing security control frameworks generally lack modeling of the causal relationship between protective actions and task performance. Most solutions evaluate the effectiveness of protection only through empirical parameters or heuristic strategies, failing to quantify the real impact of protective actions on task performance or automatically extract patterns of privacy risk changes from data. This static control approach struggles to achieve fine-grained adjustment of protection strategies when facing complex privacy-preserving computing tasks. Especially in multi-node heterogeneous computing environments, there is a non-linear coupling relationship between computational resource load, network bandwidth, and privacy budget consumption. Traditional methods struggle to simultaneously address privacy security and computational timeliness, resulting in a decline in overall task performance.

[0004] Furthermore, existing machine learning-based privacy protection strategies mostly focus on rule generation or noise parameter optimization, lacking the self-feedback and dynamic optimization mechanisms of reinforcement learning. Because they do not utilize counterfactual reasoning and policy gradient updates, the system cannot adaptively adjust protection action parameters based on historical execution results, leading to slow policy updates and poor convergence of the protection strategy. In addition, privacy sensitivity is typically measured using static feature extraction or manual annotation, lacking a quantitative calculation model based on real-time budget consumption changes. This results in privacy protection being either too broad or too narrow, further reducing the transferability and stability of the algorithm across different privacy scenarios.

[0005] Therefore, how to provide a security protection method for privacy-preserving computation based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a security protection method for privacy-preserving computation based on reinforcement learning. This invention establishes a causal structure of states, actions, performance, and privacy variables in privacy-preserving computation through the synergistic effect of causal relationship modeling, counterfactual reasoning, and reinforcement learning optimization. It constructs a comprehensive evaluation index by combining counterfactual rewards, privacy budgets, and computational latency, and generates a privacy sensitivity and masking matrix. This enables dynamic constraints on highly sensitive actions and adaptive policy updates, achieving a dynamic balance between privacy budgets, security protection, and task performance, thereby improving the security and stability of privacy-preserving computation.

[0007] A privacy-preserving computation security protection method based on reinforcement learning according to an embodiment of the present invention includes the following steps: S1. Establish a causal relationship model based on privacy computing, the model including state nodes, action nodes, task performance nodes, privacy impact variable nodes, and computing delay nodes; S2. Collect the operational data during the privacy computing process, construct a state vector, and generate a protection action vector based on the state vector; S3. Generate counterfactual actions under the same exogenous conditions using the causal relationship model, calculate the task effectiveness difference between the protective actions and the counterfactual actions, and use it as the counterfactual reward value. S4. Construct a comprehensive evaluation index based on counterfactual return value, privacy budget consumption, and computational latency; S5. Based on the causal relationship model, calculate the rate of change of privacy budget consumption relative to each protective action component, and generate a privacy sensitivity index; S6. Generate a privacy shielding matrix based on the privacy sensitivity index, reduce the weight or block protection actions with sensitivity exceeding a preset threshold, and obtain a corrected security action vector. S7. Based on the comprehensive evaluation indicators and the corrected security action vector, generate a privacy computing security protection strategy.

[0008] Optionally, step S1 includes: S11. Establish the node set of the causal relationship model and determine the naming convention. The node set includes state nodes, action nodes, task performance nodes, privacy impact variable nodes, and computation delay nodes. The naming convention uses node type plus time index for identification. S12. Define the set of fields for the state node, including remaining privacy budget, protocol type encoding, data sensitivity index, computing resource load percentage, and task stage identifier, and establish timestamps and source records for each field; S13. Define the set of fields for the action node, including noise injection strength, key rotation cycle, protocol switching target code, encryption level bit length and audit level code, and establish value validity verification rules for each field; S14. Define the field set of the task performance node, including validation set accuracy, recall and communication success rate, and configure a unified sampling period and recording precision. S15. Define the set of fields for the privacy impact variable node, including the privacy budget consumption of the current batch, the cumulative privacy budget consumption, and the remaining privacy budget, and configure the batch number and numerical precision for each field; S16. Define the field set of the calculation delay node, including average delay, 95th percentile delay, number of ciphertext operator calls, average ciphertext operator time consumption and bandwidth utilization, and configure a unified statistical time window for delay-related fields. S17. Establish a directed acyclic graph structure and determine the topological order. The topological order is as follows: state node, action node, task performance node, privacy impact variable node, and computation delay node. Allowed directed edges include state node to action node, state node to task performance node, state node to privacy impact variable node, state node to computation delay node, action node to task performance node, action node to privacy impact variable node, and action node to computation delay node. Record the positive weight identifier and version number for each directed edge. S18. Generate an exogenous condition identifier. The exogenous condition identifier consists of the network jitter level, the hardware operating temperature range, the input data distribution number, and the random seed number, and is stored as a unique identifier.

[0009] Optionally, step S2 includes: S21. During the privacy computing process, collect operational data, including the CPU utilization rate, memory usage rate, network bandwidth usage rate, task execution stage identifier, current privacy budget remaining amount, and input data batch number of the processing node. Perform time synchronization and standardization processing on the operational data to generate a state vector. S22. Input the state vector into the reinforcement learning policy network. The policy network includes an input layer, three fully connected hidden layers and an output layer. The input layer receives the state vector, the output layer outputs the protection action component, and the hidden layer adopts the modified linear unit activation function and sets a batch normalization layer. S23. An action constraint mapping module is set in the output layer of the policy network. The mapping module uses a linear constraint matrix to constrain the value range of the protection action component. The constraint range is determined by the historical action statistics interval and the privacy budget threshold. S24. The constrained protection action components are sequentially assembled into a protection action vector, which includes a noise injection strength adjustment coefficient, an encryption algorithm switching control code, a key rotation trigger signal, a protocol level adjustment parameter, and an audit strategy selection identifier. S25. During the privacy computation execution phase, parameters are adjusted and strategies are switched based on the protection action vector, and the running status data before and after execution is recorded. S26. Generate a state transition sequence based on the recorded changes in the running state.

[0010] Optionally, step S3 includes: S31. Obtain the protection action vector and determine the exogenous condition identifier based on the time index corresponding to the protection action vector; S32. Under the condition that the exogenous condition remains unchanged, construct a set of counterfactual actions that are different from the protection action vector by perturbing some components of the protection action vector based on the causal relationship model. S33. Input the protective action vector and the counterfactual action set into the causal relationship model respectively, calculate the task effectiveness value corresponding to each action, and obtain the task effectiveness of protective actions and the task effectiveness of counterfactual actions. S34. Calculate the difference between the effectiveness of protective actions and the effectiveness of counterfactual actions, generate counterfactual reward values, and establish an index relationship between counterfactual reward values ​​and corresponding protective actions. S35. Synchronize the counterfactual reward value with the state transition sequence in time and store it to form the sample data required for strategy generation.

[0011] Optionally, step S4 includes: S41. Based on the counterfactual return value, privacy budget consumption and computation delay, establish corresponding data record sets respectively, and perform time synchronization and normalization processing on the three types of data; S42. Set privacy budget constraint coefficient and computation delay constraint coefficient. The privacy budget constraint coefficient reflects the degree to which privacy budget consumption inhibits task performance, and the computation delay constraint coefficient reflects the degree to which computation delay affects task timeliness. S43. Perform a weighted operation on the counterfactual return values ​​at each time point to obtain the constraint correction effectiveness index sequence; S44. The modified performance index at each time point is combined with the corresponding privacy budget constraint component and computation delay constraint component to form a three-dimensional comprehensive evaluation index vector in a fixed order. The first component of the comprehensive evaluation index vector is the comprehensive performance value after double constraint correction, the second component is the degree of privacy budget penalty, and the third component is the degree of computation delay penalty. S45. Perform boundary verification and truncation operations on the comprehensive evaluation index vector, and record the corresponding time index and action index.

[0012] Optionally, step S5 includes: S51. Match the protection action vectors and corresponding privacy budget consumption under each time index to form a protection action-budget consumption pairing set; S52. Perform differential calculation between each component in the protection action vector and the privacy budget consumption in the corresponding time period to obtain the response relationship between the change of each protection action component and the change of privacy budget consumption. S53. Perform local moving average processing on the response relationship of each component, and calculate the unit rate of change of the protective action component based on the smoothed difference result, so as to reflect the sensitivity of the corresponding component to the privacy budget. S54. Normalize the sensitivity of each protective action component to generate a privacy sensitivity index. The privacy sensitivity index consists of multiple components, each corresponding to the privacy sensitivity of the protective action component.

[0013] Optionally, step S6 includes: S61. Obtain the privacy sensitivity index under each time index, and compare each component value with the preset sensitivity threshold to form a set of sensitivity identifiers; S62. In the sensitivity identifier set, the protective action component with sensitivity exceeding a preset threshold is marked as a high-sensitivity component, and the protective action component with sensitivity less than the preset threshold is marked as a low-sensitivity component. S63. Based on the distribution of high-sensitivity components and low-sensitivity components, a privacy shielding matrix is ​​constructed. The privacy shielding matrix is ​​a diagonal matrix structure, and each diagonal element corresponds to the weight coefficient of a protection action component. S64. In the privacy shielding matrix, the weight coefficients corresponding to the high-sensitivity components are adjusted by a proportional coefficient α, where α is a preset constant with a value greater than zero and less than one, and the weight coefficients corresponding to the low-sensitivity components are kept at 1. S65. Perform matrix multiplication on the protection action vector and the privacy shielding matrix to obtain the corrected security action vector.

[0014] Optionally, step S7 includes: S71. By indexing the time, the comprehensive evaluation indicators are mapped one-to-one with the corrected safety action vectors to generate a strategy input alignment sequence. S72. Establish a strategy generation rule table, which is a set of rules that maps each component of the comprehensive evaluation index to the protection action control slot, including control slot number, value range, boundary processing method and exogenous condition identifier fields. S73. Based on the strategy input alignment sequence and the strategy generation rule table, construct a strategy candidate set. The strategy candidate set consists of multiple candidate strategy units. Each candidate strategy unit includes an action template number, a control slot value, an expiration date label, and an exogenous condition identifier. S74. Perform threshold pruning and conflict resolution on the policy candidate set. Threshold pruning is performed based on the value range of the privacy budget penalty component and the computation delay penalty component in the comprehensive evaluation index. Conflict resolution is performed based on the control slot priority and version number order to make retention decisions, thus obtaining the pruned candidate set. S75. Assemble the trimmed candidate set to generate a strategy configuration file. The strategy configuration file includes a strategy version number, an action template number list, a control slot value table, an exogenous condition identifier, a validity period window, and a priority queue. S76. Perform a consistency check on the policy configuration file. The consistency check includes control slot boundary check, exogenous condition consistency check and version continuity check. After the check passes, write it to the policy storage area and record the effective time window and the ineffective time window. S77. Determine the policy activation order using the priority queue in the policy configuration file, and load the policy configuration file within the activation time window to form a privacy computing security protection policy.

[0015] The beneficial effects of this invention are: This invention addresses the issues of rigid privacy budget allocation, limited task performance, and fluctuating computational latency in existing privacy computing protection methods by synergistically integrating causal relationship modeling, counterfactual reasoning, and reinforcement learning strategy optimization. It establishes a causal relationship model comprising state nodes, action nodes, task performance nodes, privacy-influencing variable nodes, and computational latency nodes, achieving a structured expression of the causal dependency between privacy protection actions and system performance. During the execution of privacy computing tasks, this invention generates state vectors by collecting system states, outputs protection action vectors via a reinforcement learning strategy network, and generates counterfactual actions under the same exogenous conditions using the causal model. The difference in task performance is calculated to form a counterfactual reward value. A comprehensive evaluation index is constructed using the counterfactual reward value, privacy budget consumption, and computational latency, achieving unified quantification of privacy and timeliness constraints during the model training phase. In the privacy sensitivity assessment stage, this invention generates a privacy sensitivity index by calculating the rate of change of privacy budget consumption relative to each protection action component. This index is then used to construct a privacy shielding matrix, dynamically reducing or blocking high-sensitivity actions to form a corrected safe action vector, ensuring that privacy constraints are not violated during policy updates. Based on the strategy, a rule table is generated for threshold pruning, conflict resolution, and assembly, and these are applied according to priority and time windows. Through the synergistic effect of the above mechanisms, this invention achieves three-dimensional synergistic optimization of privacy budget consumption, computational latency, and task performance, effectively improving the security protection level, adaptability, and stability of privacy computing tasks. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0017] Figure 1 This is a schematic diagram of the overall process of a privacy computing security protection method based on reinforcement learning proposed in this invention; Figure 2 This is a schematic diagram of the causal relationship model in this invention; Figure 3 This is a schematic diagram of the collaborative update mechanism between the reinforcement learning policy network and the privacy shielding matrix in this invention; Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figure 1-3 A method for protecting privacy-preserving computation based on reinforcement learning includes the following steps: S1. Establish a causal relationship model based on privacy computing, the model including state nodes, action nodes, task performance nodes, privacy impact variable nodes, and computing delay nodes; S2. Collect the operational data during the privacy computing process, construct a state vector, and generate a protection action vector based on the state vector; S3. Generate counterfactual actions under the same exogenous conditions using the causal relationship model, calculate the task effectiveness difference between the protective actions and the counterfactual actions, and use it as the counterfactual reward value. S4. Construct a comprehensive evaluation index based on counterfactual return value, privacy budget consumption, and computational latency; S5. Based on the causal relationship model, calculate the rate of change of privacy budget consumption relative to each protective action component, and generate a privacy sensitivity index; S6. Generate a privacy shielding matrix based on the privacy sensitivity index, reduce the weight or block protection actions with sensitivity exceeding a preset threshold, and obtain a corrected security action vector. S7. Based on the comprehensive evaluation indicators and the corrected security action vector, generate a privacy computing security protection strategy.

[0020] This invention provides a privacy-preserving computation security protection method based on reinforcement learning. By combining causal modeling, counterfactual reasoning, and reinforcement learning, it achieves a dynamic balance between privacy budget and task performance. During operation, the system first establishes a causal relationship model for privacy computation and collects state information to form a state vector. After generating protective actions through a policy network, counterfactual actions are constructed under fixed external conditions, and the difference in task performance is calculated to form a counterfactual reward value. Combining privacy budget consumption and computational latency, a comprehensive evaluation index is calculated, and a privacy sensitivity index is generated accordingly. After downweighting high-sensitivity action components, a modified security action vector is generated and input into the reinforcement learning network to achieve iterative parameter updates. Through this closed-loop mechanism, the system can continuously optimize the protection strategy for privacy computation tasks, achieving a unified goal of privacy protection and performance improvement.

[0021] In this embodiment, step S1 includes: S11. Establish the node set of the causal relationship model and determine the naming convention. The node set includes state nodes, action nodes, task performance nodes, privacy impact variable nodes, and computation delay nodes. The naming convention uses node type plus time index for identification. S12. Define the set of fields for the state node, including remaining privacy budget, protocol type encoding, data sensitivity index, computing resource load percentage, and task stage identifier, and establish timestamps and source records for each field; S13. Define the set of fields for the action node, including noise injection strength, key rotation cycle, protocol switching target code, encryption level bit length and audit level code, and establish value validity verification rules for each field; S14. Define the field set of the task performance node, including validation set accuracy, recall and communication success rate, and configure a unified sampling period and recording precision. S15. Define the set of fields for the privacy impact variable node, including the privacy budget consumption of the current batch, the cumulative privacy budget consumption, and the remaining privacy budget, and configure the batch number and numerical precision for each field; S16. Define the field set of the calculation delay node, including average delay, 95th percentile delay, number of ciphertext operator calls, average ciphertext operator time consumption and bandwidth utilization, and configure a unified statistical time window for delay-related fields. S17. Establish a directed acyclic graph structure and determine the topological order. The topological order is as follows: state node, action node, task performance node, privacy impact variable node, and computation delay node. Allowed directed edges include state node to action node, state node to task performance node, state node to privacy impact variable node, state node to computation delay node, action node to task performance node, action node to privacy impact variable node, and action node to computation delay node. Record the positive weight identifier and version number for each directed edge. S18. Generate an exogenous condition identifier. The exogenous condition identifier consists of the network jitter level, the hardware operating temperature range, the input data distribution number, and the random seed number, and is stored as a unique identifier.

[0022] In establishing the causal relationship model, this invention first determines the node set and topological order based on the operational structure of the privacy-preserving computation system. All node fields originate from real-time data from the system monitoring module and task scheduler. Each node is time-aligned using time indexes and naming conventions, ensuring that the causal chain correctly reflects the sequence of state transitions and performance feedback during reinforcement learning. The weights of directed edges between nodes represent dependency strength, which can be calculated based on the correlation of historical task data. Exogenous condition identifiers serve as unified environmental variable constraints, maintaining consistency of the external environment during model inference and counterfactual generation phases, thereby obtaining comparable reward values. The establishment of the entire causal model provides a structured foundation for subsequent counterfactual evaluation and privacy sensitivity calculation, ensuring the traceability and interpretability of calculations for various privacy and performance indicators.

[0023] In this embodiment, step S2 includes: S21. During the privacy computing process, collect operational data, including the CPU utilization rate, memory usage rate, network bandwidth usage rate, task execution stage identifier, current privacy budget remaining amount, and input data batch number of the processing node. Perform time synchronization and standardization processing on the operational data to generate a state vector. S22. Input the state vector into the reinforcement learning policy network. The policy network includes an input layer, three fully connected hidden layers and an output layer. The input layer receives the state vector, the output layer outputs the protection action component, and the hidden layer adopts the modified linear unit activation function and sets a batch normalization layer. S23. An action constraint mapping module is set in the output layer of the policy network. The mapping module uses a linear constraint matrix to constrain the value range of the protection action component. The constraint range is determined by the historical action statistics interval and the privacy budget threshold. S24. The constrained protection action components are sequentially assembled into a protection action vector, which includes a noise injection strength adjustment coefficient, an encryption algorithm switching control code, a key rotation trigger signal, a protocol level adjustment parameter, and an audit strategy selection identifier. S25. During the privacy computation execution phase, parameters are adjusted and strategies are switched based on the protection action vector, and the running status data before and after execution is recorded. S26. Generate a state transition sequence based on the recorded changes in the running state.

[0024] In this invention, a state transition sequence is used to record the correspondence between states and actions during privacy computation protection, reflecting the dynamic impact of protection actions on the system's operational state. During the privacy computation execution phase, for each protection action execution cycle, the state vector before the action execution, the current protection action vector, and the state vector after the action execution are recorded, along with a time index and exogenous condition identifier, forming a single state transition record. These records are connected chronologically to form a state transition sequence. The state transition sequence reflects the changes in key variables such as privacy budget consumption, computational resource load, communication success rate, and task stage identifiers, describing the state evolution patterns caused by protection actions. The generated state transition sequence, after time synchronization, is stored along with the counterfactual reward value. Through this sequence, the reinforcement learning model can adjust policy parameters based on historical state change patterns, achieving a dynamic balance between privacy protection and task performance.

[0025] In this embodiment, step S3 includes: S31. Obtain the protection action vector and determine the exogenous condition identifier based on the time index corresponding to the protection action vector; S32. Under the condition that the exogenous condition remains unchanged, construct a set of counterfactual actions that are different from the protection action vector by perturbing some components of the protection action vector based on the causal relationship model. S33. Input the protective action vector and the counterfactual action set into the causal relationship model respectively, calculate the task effectiveness value corresponding to each action, and obtain the task effectiveness of protective actions and the task effectiveness of counterfactual actions. S34. Calculate the difference between the effectiveness of protective actions and the effectiveness of counterfactual actions, generate counterfactual reward values, and establish an index relationship between counterfactual reward values ​​and corresponding protective actions. S35. Synchronize the counterfactual reward value with the state transition sequence in time and store it to form the sample data required for strategy generation.

[0026] In this invention, the counterfactual action set is constructed based on the protective action vector. The protective action vector includes a noise injection strength adjustment coefficient, a key rotation cycle, a protocol switching control code, an encryption level bit depth, and an audit strategy selection identifier. The value range of each component is limited by the action constraint mapping module. First, perturbation rules are set according to the component type. Noise injection strength is a continuous variable, using a stepped approach; the key rotation cycle is an ordinal variable, using an adjacent step approach; the protocol switching control code, encryption level bit depth, and audit strategy selection identifier are enumerated variables, using a valid code replacement approach. Second, under the premise that the exogenous conditions remain unchanged, one or more components of the protective action vector are perturbed to generate candidate actions, and the legality is verified and the boundaries are truncated by the action constraint mapping module. For continuous variables, candidate actions are filtered according to a preset change radius, and candidates exceeding the threshold are eliminated. Finally, the verified candidate actions are numbered and recorded. The number includes the action type identifier, time index, and sequence number information, forming a counterfactual action set, providing input data for calculating the task performance difference.

[0027] In this embodiment, step S4 includes: S41. Based on the counterfactual return value, privacy budget consumption and computation delay, establish corresponding data record sets respectively, and perform time synchronization and normalization processing on the three types of data; S42. Set privacy budget constraint coefficient and computation delay constraint coefficient. The privacy budget constraint coefficient reflects the degree to which privacy budget consumption inhibits task performance, and the computation delay constraint coefficient reflects the degree to which computation delay affects task timeliness. S43. Perform a weighted operation on the counterfactual return values ​​at each time point to obtain the constraint correction effectiveness index sequence; S44. The modified performance index at each time point is combined with the corresponding privacy budget constraint component and computation delay constraint component to form a three-dimensional comprehensive evaluation index vector in a fixed order. The first component of the comprehensive evaluation index vector is the comprehensive performance value after double constraint correction, the second component is the degree of privacy budget penalty, and the third component is the degree of computation delay penalty. S45. Perform boundary verification and truncation operations on the comprehensive evaluation index vector, and record the corresponding time index and action index.

[0028] This invention constructs a comprehensive evaluation index by weighting counterfactual reward values, privacy budget consumption, and computational latency. This index quantifies the balance between privacy protection strategies and task performance, privacy loss, and computational timeliness. By introducing privacy budget constraint coefficients and computational latency constraint coefficients, the reward value is double-constrained, generating a three-dimensional comprehensive evaluation index vector. This allows for dynamic adjustment of privacy consumption and system latency during reinforcement learning. This index reflects the comprehensive impact of protection actions on performance and provides quantifiable feedback for model training, improving the adaptability and stability of privacy-preserving computational security strategies.

[0029] In this embodiment, step S5 includes: S51. Match the protection action vectors and corresponding privacy budget consumption under each time index to form a protection action-budget consumption pairing set; S52. Perform differential calculation between each component in the protection action vector and the privacy budget consumption in the corresponding time period to obtain the response relationship between the change of each protection action component and the change of privacy budget consumption. S53. Perform local moving average processing on the response relationship of each component, and calculate the unit rate of change of the protective action component based on the smoothed difference result, so as to reflect the sensitivity of the corresponding component to the privacy budget. S54. Normalize the sensitivity of each protective action component to generate a privacy sensitivity index. The privacy sensitivity index consists of multiple components, each corresponding to the privacy sensitivity of the protective action component.

[0030] In privacy-preserving computation tasks, the consumption of privacy budget by different protective actions is not linear. Therefore, this invention extracts the response characteristics of action changes to budget changes by matching action vectors with budget consumption records in the time dimension and calculating the difference between adjacent time periods. By applying a local moving average to the difference results, the impact of system noise and instantaneous fluctuations is reduced. Based on this, the unit rate of change of each component of the protective action is calculated to reflect its privacy budget sensitivity. The normalized privacy sensitivity index maintains the same dimension as the protective action components and can be used as input for the subsequent generation of the privacy shielding matrix, enabling the system to dynamically suppress or shield highly sensitive behaviors at the action level. This mechanism realizes the adaptive linkage between privacy budget and action decision-making, which is the key foundation of the privacy protection of this invention.

[0031] In this embodiment, step S6 includes: S61. Obtain the privacy sensitivity index under each time index, and compare each component value with the preset sensitivity threshold to form a set of sensitivity identifiers; S62. In the sensitivity identifier set, the protective action component with sensitivity exceeding a preset threshold is marked as a high-sensitivity component, and the protective action component with sensitivity less than the preset threshold is marked as a low-sensitivity component. S63. Based on the distribution of high-sensitivity components and low-sensitivity components, a privacy shielding matrix is ​​constructed. The privacy shielding matrix is ​​a diagonal matrix structure, and each diagonal element corresponds to the weight coefficient of a protection action component. S64. In the privacy shielding matrix, the weight coefficients corresponding to the high-sensitivity components are adjusted by a proportional coefficient α, where α is a preset constant with a value greater than zero and less than one, and the weight coefficients corresponding to the low-sensitivity components are kept at 1. S65. Perform matrix multiplication on the protection action vector and the privacy shielding matrix to obtain the corrected security action vector.

[0032] After generating the privacy sensitivity index, this invention classifies and labels each action component by setting a dynamic threshold, distinguishing between high-sensitivity and low-sensitivity components. A privacy shielding matrix is ​​constructed based on the labeling results. This matrix is ​​a diagonal structure with the same dimensions as the protected action vector, with each main diagonal element representing an action weight. The weights of high-sensitivity components are scaled using a proportional coefficient α, thereby suppressing the influence of high-risk components without altering the overall action structure. This matrix is ​​regenerated based on the latest sensitivity after each training iteration, achieving dynamic updates. The corrected secure action vector is used for privacy computation tasks, creating a closed-loop coupling between the privacy shielding mechanism and the policy optimization process. Through this design, this invention achieves structured privacy protection at the action level, significantly reducing the risk of excessive privacy budget consumption while maintaining system performance stability.

[0033] In this embodiment, step S7 includes: S71. By indexing the time, the comprehensive evaluation indicators are mapped one-to-one with the corrected safety action vectors to generate a strategy input alignment sequence. S72. Establish a strategy generation rule table, which is a set of rules that maps each component of the comprehensive evaluation index to the protection action control slot, including control slot number, value range, boundary processing method and exogenous condition identifier fields. The control slots are used to identify the positions of adjustable protection parameters, including noise injection intensity, key rotation cycle, protocol level parameters and audit strategy selection. Each slot is mapped to a corresponding parameter, and the number and value range are given in the strategy generation rule table. S73. Based on the strategy input alignment sequence and the strategy generation rule table, construct a strategy candidate set. The strategy candidate set consists of multiple candidate strategy units. Each candidate strategy unit includes an action template number, a control slot value, an expiration date label, and an exogenous condition identifier. S74. Perform threshold pruning and conflict resolution on the policy candidate set. Threshold pruning is performed based on the value range of the privacy budget penalty component and the computation delay penalty component in the comprehensive evaluation index. Conflict resolution is performed based on the control slot priority and version number order to make retention decisions, thus obtaining the pruned candidate set. S75. Assemble the trimmed candidate set to generate a strategy configuration file. The strategy configuration file includes a strategy version number, an action template number list, a control slot value table, an exogenous condition identifier, a validity period window, and a priority queue. S76. Perform a consistency check on the policy configuration file. The consistency check includes control slot boundary check, exogenous condition consistency check and version continuity check. After the check passes, write it to the policy storage area and record the effective time window and the ineffective time window. S77. Determine the policy activation order using the priority queue in the policy configuration file, and load the policy configuration file within the activation time window to form a privacy computing security protection policy.

[0034] In this invention, a rule table for policy generation is constructed based on the comprehensive evaluation index and security action vector. This rule table is trained using recorded data from historical privacy computing tasks and defines the mapping relationship between protection action control slots and index components. Each control slot represents an adjustable privacy protection parameter position, such as noise intensity, key rotation cycle, or protocol level parameter. After generating a candidate policy set, the system performs threshold pruning by using the privacy budget penalty component and computation delay penalty component in the comprehensive evaluation index to eliminate policy units with excessive risk, and resolves conflicts based on slot priority and version number. The final generated policy configuration file includes the policy version number, action template number, slot value table, and exogenous condition identifier, and is automatically loaded and executed within the effective time window to achieve dynamic protection control of privacy computing tasks.

[0035] Example 1: To verify the feasibility and effectiveness of this invention in practical privacy-preserving computation tasks, it was applied to a privacy-preserving computation scenario for a collaborative diagnostic model in medical institutions. This scenario involved four large hospitals jointly training a lung disease diagnostic model using differential privacy and a federated learning framework, without sharing original patient data. Due to significant differences in network environments, computing resources, and privacy budget allocation among the hospitals, traditional privacy-preserving computation methods commonly suffer from imbalanced privacy budget consumption, large fluctuations in computational latency, and decreased model performance. Especially under high loads, the budget of some nodes is exhausted prematurely, leading to the failure of global model aggregation.

[0036] This invention establishes an adaptive balance mechanism between privacy protection and task performance by introducing four core modules: a causal relationship model, a counterfactual reasoning mechanism, a reinforcement learning self-updating strategy, and a privacy masking matrix. First, before task execution, a causal relationship model is established based on historical task records, including state nodes, action nodes, privacy-impacting variable nodes, task performance nodes, and computational latency nodes. State nodes record CPU utilization, remaining privacy budget, network bandwidth, and data sensitivity index; action nodes represent noise injection intensity, key rotation cycle, and protocol switching targets; privacy-impacting variable nodes reflect the proportion of privacy budget consumption and remaining balance; task performance nodes describe model accuracy and recall; and computational latency nodes record average latency and the 95th percentile latency.

[0037] During training, the system collects the aforementioned node information in real time and generates state vectors. The reinforcement learning policy network outputs protective action vectors based on these state vectors, used to dynamically adjust differential privacy noise and encryption strategies. When the model detects that the privacy budget decrease rate exceeds 15% of the mean, the system triggers causal model inference, generating a counterfactual action that differs from the current action but is consistent with the exogenous conditions. The counterfactual reward value is calculated by comparing the corresponding task performance values ​​of the two actions, thus eliminating the interference of random environmental factors. The counterfactual reward value, along with the privacy budget consumption and computational latency, forms a three-dimensional comprehensive evaluation index, where the first component is the comprehensive performance value, the second component is the degree of privacy budget penalty, and the third component is the degree of computational latency penalty.

[0038] In the privacy sensitivity assessment stage, this invention calculates the response rate of changes in each protective action component to changes in privacy budget consumption, generating a privacy sensitivity index. The system finds that the noise intensity parameter has the highest correlation with changes in budget consumption, so the weight of this component is adjusted to 0.63 using a privacy masking matrix to reduce the impact of highly sensitive actions. The dynamic constraints of the masking matrix ensure that protective actions maintain a safe boundary in subsequent rounds, avoiding drastic fluctuations in the privacy budget. In each iteration, the reinforcement learning strategy uses the comprehensive evaluation index as input and updates parameters through policy gradients, gradually bringing the policy network towards a stable balance between privacy budget, security, and performance.

[0039] To verify the beneficial effects of this invention, in actual verification, the system operation cycle was set to 72 hours, and 100 rounds of model aggregation tasks were trained. In the experimental environment, the average privacy budget for each hospital was ε=2.5, and the communication latency between nodes was controlled between 200 and 260 milliseconds. Compared with the traditional differential privacy static control scheme, the reinforcement learning dynamic protection mechanism of this invention effectively improves model performance and budget utilization under the same budget conditions, and significantly reduces the risk of privacy leakage. The privacy leakage risk score is calculated by the risk control or compliance assessment module based on privacy budget consumption records, access logs, and task audit information, with a value range between 0 and 1. The smaller the value, the higher the privacy protection strength and the lower the leakage risk. The experimental results are shown in Table 1.

[0040] Table 1. Performance Comparison of the Causal Enhancement Privacy Protection Mechanism of the Invention with Traditional Solutions

[0041] As shown in Table 1, with the introduction of causal reasoning, the counterfactual reward fluctuation coefficient of this invention is reduced by 48.6%, indicating that the system can stably evaluate the difference in action performance; the number of convergence rounds of reinforcement learning is reduced by 39.7%, indicating a significant improvement in policy optimization speed; the privacy budget utilization rate is increased by 30.4%, and the proportion of highly sensitive actions is reduced by 63.2%, indicating that the privacy shielding matrix is ​​effective in controlling sensitive actions; the accuracy of the model validation set is improved to 91.8%, and the leakage risk score is reduced by 47.8%, achieving a dynamic balance between privacy protection and performance improvement.

[0042] Experimental results show that this invention provides accurate reward signals through causal models and counterfactual reasoning, achieves adaptive optimization of protective actions by combining reinforcement learning policy updates, and forms a dynamic privacy protection boundary at the action level through structural constraints of privacy sensitivity and the masking matrix. This collaborative mechanism creates a positive feedback loop between privacy budget allocation, policy optimization, and performance improvement, significantly outperforming traditional static control or single noise adjustment schemes, and demonstrating strong engineering feasibility and cross-scenario applicability.

[0043] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A privacy-preserving computation security protection method based on reinforcement learning, characterized in that, Includes the following steps: S1. Establish a causal relationship model based on privacy computing, wherein the causal relationship model includes state nodes, action nodes, task performance nodes, privacy impact variable nodes, and computing delay nodes; S2. Collect the running data during the privacy computing process, form a state vector, and generate a protection action vector based on the state vector; S3. Generate counterfactual actions under the same exogenous conditions using the causal relationship model, calculate the task effectiveness difference between the protective actions and the counterfactual actions, and use it as the counterfactual reward value. S4. Construct a comprehensive evaluation index based on counterfactual return value, privacy budget consumption, and computational latency; S5. Based on the causal relationship model, calculate the rate of change of privacy budget consumption relative to each protective action component, and generate a privacy sensitivity index. S6. Generate a privacy shielding matrix based on the privacy sensitivity index, reduce the weight or block protection actions with sensitivity exceeding a preset threshold, and obtain a corrected security action vector. S7. Based on the comprehensive evaluation indicators and the revised security action vector, generate a privacy computing security protection strategy.

2. The privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S1 includes: S11. Establish the node set of the causal relationship model and determine the naming convention. The node set includes state nodes, action nodes, task performance nodes, privacy impact variable nodes, and computation delay nodes. The naming convention uses node type plus time index for identification. S12. Define the set of fields for the state node, including remaining privacy budget, protocol type encoding, data sensitivity index, computing resource load percentage, and task stage identifier, and establish timestamps and source records for each field; S13. Define the set of fields for the action node, including noise injection strength, key rotation cycle, protocol switching target code, encryption level bit length and audit level code, and establish value validity verification rules for each field; S14. Define the field set of the task performance node, including validation set accuracy, recall and communication success rate, and configure a unified sampling period and recording precision. S15. Define the set of fields for the privacy impact variable node, including the privacy budget consumption of the current batch, the cumulative privacy budget consumption, and the remaining privacy budget, and configure the batch number and numerical precision for each field; S16. Define the field set of the calculation delay node, including average delay, 95th percentile delay, number of ciphertext operator calls, average ciphertext operator time consumption and bandwidth utilization, and configure a unified statistical time window for delay-related fields. S17. Establish a directed acyclic graph structure and determine the topological order. The topological order is as follows: state node, action node, task performance node, privacy impact variable node, and computation delay node. Allowed directed edges include state node to action node, state node to task performance node, state node to privacy impact variable node, state node to computation delay node, action node to task performance node, action node to privacy impact variable node, and action node to computation delay node. Record the positive weight identifier and version number for each directed edge. S18. Generate an exogenous condition identifier. The exogenous condition identifier consists of the network jitter level, the hardware operating temperature range, the input data distribution number, and the random seed number, and is stored as a unique identifier.

3. The privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S2 includes: S21. During the privacy computing process, collect operational data, including the CPU utilization rate, memory usage rate, network bandwidth usage rate, task execution stage identifier, current privacy budget remaining amount, and input data batch number of the processing node. Perform time synchronization and standardization processing on the operational data to generate a state vector. S22. Input the state vector into the reinforcement learning policy network. The policy network includes an input layer, three fully connected hidden layers and an output layer. The input layer receives the state vector, the output layer outputs the protection action component, and the hidden layer adopts the modified linear unit activation function and sets a batch normalization layer. S23. An action constraint mapping module is set in the output layer of the policy network. The mapping module uses a linear constraint matrix to constrain the value range of the protection action component. The constraint range is determined by the historical action statistics interval and the privacy budget threshold. S24. The constrained protection action components are sequentially assembled into a protection action vector, which includes a noise injection strength adjustment coefficient, an encryption algorithm switching control code, a key rotation trigger signal, a protocol level adjustment parameter, and an audit strategy selection identifier. S25. During the privacy computation execution phase, parameters are adjusted and strategies are switched based on the protection action vector, and the running status data before and after execution is recorded. S26. Generate a state transition sequence based on the recorded changes in the running state.

4. The privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S3 includes: S31. Obtain the protection action vector and determine the exogenous condition identifier based on the time index corresponding to the protection action vector; S32. Under the condition that the exogenous condition remains unchanged, construct a set of counterfactual actions that are different from the protection action vector by perturbing some components of the protection action vector based on the causal relationship model. S33. Input the protective action vector and the counterfactual action set into the causal relationship model respectively, calculate the task effectiveness value corresponding to each action, and obtain the task effectiveness of protective actions and the task effectiveness of counterfactual actions. S34. Calculate the difference between the effectiveness of protective actions and the effectiveness of counterfactual actions, generate counterfactual reward values, and establish an index relationship between counterfactual reward values ​​and corresponding protective actions. S35. Synchronize the counterfactual reward value with the state transition sequence in time and store it to form the sample data required for strategy generation.

5. A privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S4 includes: S41. Based on the counterfactual return value, privacy budget consumption and computation delay, establish corresponding data record sets respectively, and perform time synchronization and normalization processing on the three types of data; S42. Set privacy budget constraint coefficient and computation delay constraint coefficient. The privacy budget constraint coefficient reflects the degree to which privacy budget consumption inhibits task performance, and the computation delay constraint coefficient reflects the degree to which computation delay affects task timeliness. S43. Perform a weighted operation on the counterfactual return values ​​at each time point to obtain the constraint correction effectiveness index sequence; S44. The modified performance index at each time point is combined with the corresponding privacy budget constraint component and computation delay constraint component to form a three-dimensional comprehensive evaluation index vector in a fixed order. The first component of the comprehensive evaluation index vector is the comprehensive performance value after double constraint correction, the second component is the degree of privacy budget penalty, and the third component is the degree of computation delay penalty. S45. Perform boundary verification and truncation operations on the comprehensive evaluation index vector, and record the corresponding time index and action index.

6. The privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S5 includes: S51. Match the protection action vectors and corresponding privacy budget consumption under each time index to form a protection action-budget consumption pairing set; S52. Perform differential calculation between each component in the protection action vector and the privacy budget consumption in the corresponding time period to obtain the response relationship between the change of each protection action component and the change of privacy budget consumption. S53. Perform local moving average processing on the response relationship of each component, and calculate the unit rate of change of the protective action component based on the smoothed difference result, so as to reflect the sensitivity of the corresponding component to the privacy budget. S54. Normalize the sensitivity of each protective action component to generate a privacy sensitivity index. The privacy sensitivity index consists of multiple components, each corresponding to the privacy sensitivity of the protective action component.

7. A privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S6 includes: S61. Obtain the privacy sensitivity index under each time index, and compare each component value with the preset sensitivity threshold to form a set of sensitivity identifiers; S62. In the sensitivity identifier set, the protective action components whose sensitivity exceeds a preset threshold are marked as high-sensitivity components, and the protective action components whose sensitivity does not exceed the preset threshold are marked as low-sensitivity components. S63. Based on the distribution of high-sensitivity components and low-sensitivity components, a privacy shielding matrix is ​​constructed. The privacy shielding matrix is ​​a diagonal matrix structure, and each diagonal element corresponds to the weight coefficient of a protection action component. S64. In the privacy shielding matrix, the weight coefficients corresponding to the high-sensitivity components are adjusted by a proportional coefficient α, where α is a preset constant with a value greater than zero and less than one, and the weight coefficients corresponding to the low-sensitivity components are kept at 1. S65. Perform matrix multiplication on the protection action vector and the privacy shielding matrix to obtain the corrected security action vector.

8. A privacy-preserving computation security protection method based on reinforcement learning according to claim 1, characterized in that, Step S7 includes: S71. By indexing the time, the comprehensive evaluation indicators are mapped one-to-one with the corrected safety action vectors to generate a strategy input alignment sequence. S72. Establish a strategy generation rule table, which is a set of rules that maps each component of the comprehensive evaluation index to the protection action control slot, including control slot number, value range, boundary processing method and exogenous condition identifier fields. S73. Based on the strategy input alignment sequence and the strategy generation rule table, construct a strategy candidate set. The strategy candidate set consists of multiple candidate strategy units. Each candidate strategy unit includes an action template number, a control slot value, an expiration date label, and an exogenous condition identifier. S74. Perform threshold pruning and conflict resolution on the policy candidate set. Threshold pruning is performed based on the value range of the privacy budget penalty component and the computation delay penalty component in the comprehensive evaluation index. Conflict resolution is performed based on the control slot priority and version number order to make retention decisions, thus obtaining the pruned candidate set. S75. Assemble the trimmed candidate set to generate a strategy configuration file. The strategy configuration file includes a strategy version number, an action template number list, a control slot value table, an exogenous condition identifier, a validity period window, and a priority queue. S76. Perform a consistency check on the policy configuration file. The consistency check includes control slot boundary check, exogenous condition consistency check and version continuity check. After the check passes, write it to the policy storage area and record the effective time window and the ineffective time window. S77. Determine the policy activation order using the priority queue in the policy configuration file, and load the policy configuration file within the activation time window to form a privacy computing security protection policy.

Images