Deep reinforcement learning optimization algorithm based on adaptive encryption framework

Abstract

The invention provides a deep reinforcement learning optimization algorithm based on an adaptive encryption framework, and the algorithm comprises ten steps: data input preprocessing, data sensitivity evaluation, system resource state acquisition, state vector construction, strategy decision, encryption execution, calculation reward, model updating, result output, and loop optimization. The method aims at solving the problems that encrypted data processing efficiency is low and safety is difficult to consider. According to the method, intelligent optimization of the encryption strategy is realized through the data sensitivity evaluation module, the adaptive encryption strategy adjustment module, the deep reinforcement learning optimization module, the parallel processing module, the resource scheduling module and the like. On the premise that the data security is ensured, the processing efficiency of the system is remarkably improved, and the method is suitable for scenes such as cloud computing, big data processing and the Internet of Things which need to process a large amount of sensitive data.

Description

Technical Field

[0001] The present invention relates to the field of computer software encryption algorithms, and in particular to an optimization algorithm that integrates adaptive encryption technology and deep reinforcement learning, for improving the efficiency and security of encrypted data processing. Background Art

[0002] With the rapid development of information technology, data security and privacy protection have become the focus of attention in various industries. Traditional encryption algorithms, such as symmetric encryption (AES, DES) and asymmetric encryption (RSA, ECC), have played an important role in protecting data security. However, as the scale of data continues to expand, traditional encryption algorithms have problems with high computational overhead and low processing efficiency when processing large-scale data. In addition, deep reinforcement learning, as an important branch of artificial intelligence, has achieved remarkable results in the fields of games, robot control, etc., but its application in the field of encryption strategy optimization is still immature.

[0003] In the prior art, the selection of encryption strategies is usually static and cannot be dynamically adjusted according to data characteristics and system status. This makes it difficult to strike a balance between security and performance when processing encrypted data. For example, using a high-strength encryption algorithm can improve data security, but it will increase computing overhead and reduce system performance; using a low-strength encryption algorithm can improve processing efficiency, but may not meet security requirements. Therefore, there is an urgent need for an optimization algorithm that can dynamically adjust encryption strategies according to data sensitivity and system status. Summary of the invention

[0004] The present invention aims to provide an optimization algorithm based on an adaptive encryption framework and deep reinforcement learning to solve the problems in the prior art of low efficiency in encrypted data processing and inability to dynamically balance security and performance. By applying the deep reinforcement learning algorithm to the optimization of the adaptive encryption strategy, real-time perception of data sensitivity and system resources can be achieved, the encryption strategy can be dynamically adjusted, and the overall performance and security of the system can be improved.

[0005] To achieve the above object, the present invention proposes the following technical solutions:

[0006] A deep reinforcement learning optimization algorithm based on an adaptive encryption framework includes the following steps:

[0007] Step 1: Data input and preprocessing: receiving the data D to be processed, extracting its features, and obtaining the feature vector X;

[0008] Step 2: Data sensitivity assessment: Calculate the data sensitivity score S d =f sens (X);

[0009] Step 3: Obtain system resource status: Obtain system resource utilization in real time to form a system resource status vector S r ;

[0010] Step 4: Construct a state vector: Combine the data sensitivity score and the system resource status to form the current state vector S = [S d ,S r ];

[0011] Step 5: Policy decision: Using policy network π θ (a|s), select the optimal encryption strategy A=(E,θ) under the state vector S;

[0012] Step 6, perform encryption: use the selected encryption algorithm E and the configured encryption parameter θ to encrypt the data D to obtain the encrypted data D';

[0013] Step 7: Calculate the reward: Calculate the reward R based on the security assessment and performance consumption assessment;

[0014] Step 8, model update: use the sampled data (S, A, R, S') to update the policy network parameters θ and value network parameters φ;

[0015] Step 9, output result: output the encrypted data D' for subsequent use;

[0016] Step 10: Loop optimization: Continuously monitor new data input and system status, repeat steps 1 to 9, and continuously optimize the encryption strategy.

[0017] In step 1, the features include data type, content keywords, data source, and historical access records, generating a feature vector X = [x1, x2, ... x i ,…x n ], where x i Represents the value of the i-th feature.

[0018] In step 2, a machine learning algorithm is used to evaluate the sensitivity of the data and generate a sensitivity score S d ; Sensitivity score calculation formula: Among them, f i (x) is the value of the i-th feature, w i is the corresponding weight, and n is the number of features.

[0019] In step 3, the system resource utilization includes the CPU utilization U CPU , GPU utilization U GPU 、Memory usage U Memory 、Network bandwidth B Network ; System resource state vector S r =[UCPU ,U GPU ,U Memory ,B Network ].

[0020] In step 4, the state space includes the data sensitivity score S d and system resource state vector S r , S=[S d ,S r ]=[S d ,U CPU ,U GPU ,U Memory ,B Network ].

[0021] The step five specifically includes the following steps:

[0022] (1) Policy network input: Input the state vector S into the policy network;

[0023] (2) Policy network output: Select encryption algorithm E, configure encryption parameters θ, and pass policy network π θ (a|S), select an action based on the current state vector S.

[0024] In step 6, the encryption process is: D'=E θ (D), where E θ It indicates the encryption algorithm configured with parameter θ. When the amount of data is large, the data is sharded and encrypted in parallel using multi-threading or multi-process. After the encryption is completed, the encrypted data D' is generated.

[0025] The step seven specifically includes the following steps:

[0026] (1) Safety assessment: Calculate the safety score Q security , defined as: Among them, S a Score the security level of the selected encryption algorithm, S max The highest safety level score; the safety score is normalized to the interval [0,1];

[0027] (2) Performance consumption evaluation: record the execution time Te and resource usage Ru of the encryption process, and calculate the performance consumption score Q performance , defined as: Among them, ω1 and ω2 are weight coefficients, T max and R max The preset maximum execution time and maximum resource usage;

[0028] (3) Instant reward calculation: R = α·Q security -β·Qperformance , where α and β are weight coefficients for adjusting the impact of security and performance consumption.

[0029] The step eight specifically includes the following steps:

[0030] (1) Experience data collection: The current state S, action A, reward R, and next state S' form an experience sample (S, A, R, S');

[0031] (2) Value network update:

[0032] Calculate TD error: δ = R + γV φ (S')-V φ (S), where V φ is the value network, γ is the discount factor;

[0033] Update the value network parameter φ: Among them, η v is the learning rate;

[0034] (3) Policy network update:

[0035] Calculate the advantage function A(S,A): A(S,A) = δ;

[0036] Update the policy network parameters θ: Among them, η is the learning rate.

[0037] The step nine specifically includes the following steps:

[0038] (1) Data output: save the encrypted data D' to a specified location or transmit it to the target recipient;

[0039] (2) Record log: record the relevant information of this encryption task.

[0040] The step ten specifically includes the following steps:

[0041] (1) Detect new data: Check whether there is new data to be processed D new ; When there is new data, return to step 1 and continue processing;

[0042] (2) Update system status: Update system resource status Sr in real time;

[0043] (3) Continuous model learning: Through continuous interaction and learning, the model’s strategy network and value network are gradually optimized to improve decision-making quality.

[0044] The present invention realizes intelligent optimization of adaptive encryption strategy through deep reinforcement learning algorithm, which has the following advantages:

[0045] (1) Dynamically adjust encryption strategies: The system can dynamically select the optimal encryption algorithm and parameter configuration based on the sensitivity of the data and the real-time system resource status, balancing security and performance requirements.

[0046] (2) Improve system efficiency: Through parallel processing and resource scheduling, fully utilize computing resources such as multi-core CPUs and GPUs to accelerate the encryption process and improve data processing efficiency.

[0047] (3) Enhance data security: Ensure the security of data encryption while meeting performance requirements and adapt to security needs in different application scenarios.

[0048] (4) Self-learning and optimization: Using deep reinforcement learning algorithms, the system can continuously learn and optimize during operation, improving the accuracy and adaptability of strategic decisions.

[0049] (5) Strong scalability: The algorithm framework has good scalability and can add new encryption algorithms and strategies according to specific needs to adapt to a variety of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a logic flow chart of the encryption and decryption process of the optimization algorithm described in the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, the present invention includes ten steps, which are as follows:

[0053] Step 1: Data input and preprocessing

[0054] Purpose: Receive the data D to be processed, extract its features, and obtain the feature vector X.

[0055] The specific operations are as follows:

[0056] (1) Data input: The system receives raw data D, which can be of various types, such as text, images, audio, etc.

[0057] (2) Data preprocessing:

[0058] Cleaning: Remove noise, null values, and outliers.

[0059] Formatting: Convert data into a unified format for easy subsequent processing.

[0060] (3) Feature extraction:

[0061] Extract features related to data sensitivity, such as keywords, data types, data sources, historical access records, etc.

[0062] (4) Generate feature vector X = [x1, x2, ... x i ,…x n ], where x i Represents the value of the i-th feature.

[0063] Step 2: Data sensitivity assessment

[0064] Purpose: Calculate the sensitivity score S of the data d , providing a basis for strategic decision-making.

[0065] Specific operations:

[0066] Sensitivity assessment function: Use a pre-trained model or rule to calculate the sensitivity score of the data: Among them, w i is the weight of the i-th feature, reflecting the influence of this feature on the sensitivity.

[0067] Result: Get the sensitivity score S of the data d , the value range is usually between [0,1], and the larger the value, the more sensitive the data.

[0068] Step 3: Get system resource status

[0069] Purpose: To obtain system resource utilization in real time and form a system resource state vector S r .

[0070] Specific operations:

[0071] Monitoring parameters: CPU utilization U CPU , GPU utilization U GPU 、Memory usage U Memory 、Network bandwidth B Network .

[0072] System resource state vector: S r =[U CPU ,U GPU ,U Memory ,B Network ]

[0073] Step 4: Construct the state vector

[0074] Purpose: Combine the data sensitivity score and system resource status to form the current state vector S.

[0075] Specific operations:

[0076] State vector: S = [S d ,Sr ]=[S d ,U CPU ,U GPU ,U Memory ,B Network ]

[0077] Function: The state vector S will be used as the input of the deep reinforcement learning model to reflect the comprehensive information of the current environment.

[0078] Step 5: Strategic Decision

[0079] Purpose: Utilize the policy network π θ (a|S), select the optimal encryption strategy A=(E,θ) in state S.

[0080] Specific operations:

[0081] (1) Policy network input: Input the state vector S into the policy network.

[0082] (2) Policy network output:

[0083] Selection of encryption algorithm E, for example: AES, RSA, DES, etc.

[0084] Configuration of encryption parameters θ, such as key length, encryption mode, etc.

[0085] According to the data sensitivity score S d and system resource state vector S r (such as CPU utilization, memory usage), using the policy function φ(S d ,S r ) selects the optimal encryption algorithm E and parameter configuration θ. The policy function is defined as: (E, θ) = φ(S d ,S r ).

[0086] Action selection: A = π θ (a|S) Through the policy network, action A is selected according to the current state S.

[0087] Step 6: Execute encryption

[0088] Purpose: To encrypt data D using the selected encryption algorithm E and parameter configuration θ, and obtain encrypted data D'.

[0089] Specific operations:

[0090] (1) Encryption process: D' = E θ (D) Among them, E θ Represents the encryption algorithm configured using parameter θ.

[0091] (2) Parallel processing: If the amount of data is large, the data can be sharded. First, the data set to be encrypted can be divided into multiple subtasks, and then dynamically allocated to different computing resources (such as multi-core CPUs and GPUs) according to the system resource status. Then, multi-threading or multi-process technology can be used to achieve parallel processing of encryption tasks and speed up encryption. Real-time monitoring of system resource utilization can be carried out to adjust task allocation strategies to avoid resource overload or idleness and improve the overall efficiency of the system.

[0092] (3) Encryption completed: Encrypted data D' is generated.

[0093] Step 7: Calculate rewards

[0094] Purpose: Calculate the immediate reward R based on security evaluation and performance consumption for model learning and optimization.

[0095] Specific operations:

[0096] (1) Safety assessment:

[0097] Calculate the safety score Q security , based on the known security level of the encryption algorithm and key length, etc.

[0098] Defined as: Among them, S a Score the security level of the selected encryption algorithm, S max The highest security level score is normalized to the interval [0,1]. The security score can be normalized to the interval [0,1].

[0099] (2) Performance consumption evaluation:

[0100] Record the execution time Te during the encryption process.

[0101] Record resource usage Ru (such as CPU and GPU utilization).

[0102] Compute performance consumption score Q performance , which can be defined as:

[0103]

[0104] Among them, ω1 and ω2 are weights, T max and R max The preset maximum allowed value.

[0105] Instant reward calculation: R = α·Q security -β·Q performance

[0106] Among them, α and β are weight coefficients for adjusting the impact of security and performance consumption.

[0107] Step 8: Model Update

[0108] Purpose: Use the sampled data (S, A, R, S') to update the policy network parameters θ and the value network parameters φ.

[0109] Specific operations:

[0110] (1) Experience data collection: The current state S, action A, reward R, and next state S' form an experience sample (S, A, R, S').

[0111] (2) Value network update:

[0112] Calculate the TD error:

[0113] δ=R+γV φ (S′)-V φ (S)

[0114] Among them, V φ is the value network, and γ is the discount factor.

[0115] Update the value network parameter φ:

[0116]

[0117] Among them, η v is the learning rate.

[0118] (3) Policy network update:

[0119] Calculate the advantage function A(S,A):

[0120] A(S,A)=δ

[0121] Update the policy network parameters θ:

[0122]

[0123] Among them, η is the learning rate.

[0124] The deep reinforcement learning algorithm used in this embodiment is the PPO algorithm, which can ensure the stability and convergence of the model. The specific algorithm flow is as follows:

[0125] (1) Initialization

[0126] Randomly initialize the policy network encryption parameters θ and value network parameters φ; policy network π θ (a|s) Input state s, output action a probability; value network V φ (s) Input state s and output the value estimate of that state.

[0127] (2) Sampling data

[0128] In the current strategy π θ Next, collect a set of interaction data (s t ,a t ,r t ,s t+1 ), where r t For immediate rewards.

[0129] (3) Calculation of advantage function

[0130] Advantage function A t Calculation: A t =r t +γV φ (s t+1 )-V φ (s t ), where γ is the discount factor.

[0131] (4) Update strategy network parameters

[0132] Update θ using policy gradient:

[0133] Policy gradient loss function L PG (θ) is defined as: L PG (θ) = E t [logπ θ (a t |s t )A t ], where η is the learning rate.

[0134] (5) Update value network parameters

[0135] Update φ using the mean square error:

[0136] Value network loss function L V (φ) is defined as:

[0137] L V (φ)=E t [(V φ (s t )-(r t +γV φ (s t+1 ))) 2 ], where η v is the learning rate of the value network.

[0138] (6) Circuit training

[0139] Repeat steps (2) to (5) until the policy network and value network converge.

[0140] Step 9: Output results

[0141] Purpose: Output encrypted data D' for subsequent use.

[0142] Specific operations:

[0143] (1) Data output: The encrypted data D' is saved to a specified location or transmitted to the target recipient.

[0144] (2) Record log: record the relevant information of this encryption task, such as time, encryption strategy used, performance indicators, etc.

[0145] Step 10: Loop Optimization

[0146] Purpose: Continuously monitor new data input and system status, repeat steps 1 to 9, and continuously optimize encryption strategies.

[0147] Specific operations:

[0148] (1) Detect new data: Check whether there is new data to be processed D new , if there is new data, return to step 1 and process in a loop.

[0149] (2) Update system status: Update system resource status Sr in real time.

[0150] (3) Continuous model learning: Through continuous interaction and learning, the model’s strategy network and value network are gradually optimized to improve decision-making quality.

[0151] The dynamics and adaptability of the overall process of the present invention are mainly manifested in the following aspects:

[0152] Real-time: The system can respond to changes in data and environment in real time and dynamically adjust encryption strategies to adapt to current security requirements and system performance.

[0153] Self-learning capability: Through deep reinforcement learning algorithms, the model can continuously learn during interaction, optimize strategies, and adapt to different data characteristics and system states.

[0154] Balance: Achieve a dynamic balance between security and performance to ensure optimal system performance while meeting security requirements.

Claims

1. A deep reinforcement learning optimization algorithm based on an adaptive encryption framework, characterized in that: The following steps are involved: Step 1: Data input and preprocessing: receiving the data D to be processed, extracting its features, and obtaining the feature vector X; Step 2: Data sensitivity assessment: Calculate the data sensitivity score S d =f sens (X); Step 3: Obtain system resource status: Obtain system resource utilization in real time to form a system resource status vector S r ; Step 4: Construct a state vector: Combine the data sensitivity score and the system resource status to form the current state vector S = [S d ,S r ]; Step 5: Policy decision: Using policy network π θ (a|s), select the optimal encryption strategy A=(E,θ) under the state vector S; Step 6, perform encryption: use the selected encryption algorithm E and the configured encryption parameter θ to encrypt the data D to obtain the encrypted data D'; Step 7: Calculate the reward: Calculate the reward R based on the security assessment and performance consumption assessment; Step 8, model update: Use the sampled data (S, A, R, S') to update the policy network encryption parameters θ and the value network parameters φ; Step 9, output result: output the encrypted data D' for subsequent use; Step 10, loop optimization: Continuously monitor new data input and system status. When new data is available, repeat steps 1 to 9 to continuously optimize the encryption strategy.

2. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1 is characterized in that: In step 2, a machine learning algorithm is used to evaluate the sensitivity of the data and generate a sensitivity score S d ; Sensitivity score calculation formula: Among them, f i (x) is the value of the i-th feature, w i is the corresponding weight, and n is the number of features.

3. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1 is characterized in that: In step 3, the system resource utilization includes the CPU utilization U CPU , GPU utilization U GPU 、Memory usage U Memory 、Network bandwidth B Network ; System resource state vector S r =[U CPU ,U GPU ,U Memory ,B Network ].

4. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1, characterized in that: In step 4, the state space includes the data sensitivity score S d and system resource state vector S r , S=[S d ,S r ]=[S d ,U CPU ,U GPU ,U Memory ,B Network ].

5. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1, characterized in that: The step five specifically includes the following steps: (1) Policy network input: Input the state vector S into the policy network; (2) Policy network output: Select encryption algorithm E, configure encryption parameters θ, and pass policy network π θ (a|S), and selects an action based on the current state vector S.

6. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1 is characterized in that: In step 6, the encryption process is: D'=E θ (D), where E θ It indicates the encryption algorithm configured with parameter θ. When the amount of data is large, the data is sharded and encrypted in parallel using multi-threading or multi-process. After the encryption is completed, the encrypted data D' is generated.

7. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1 is characterized in that: The step seven specifically includes the following steps: (1) Safety assessment: Calculate the safety score Q security , defined as: Among them, S a Score the security level of the selected encryption algorithm, S max The highest safety level score; the safety score is normalized to the interval [0,1]; (2) Performance consumption evaluation: record the execution time Te and resource usage Ru of the encryption process, and calculate the performance consumption score Q performance , defined as: Among them, ω1 and ω2 are weight coefficients, T max and R max The preset maximum execution time and maximum resource usage; (3) Instant reward calculation: R = α·Q security -β·Q performance , where α and β are weight coefficients for adjusting the impact of security and performance consumption.

8. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1, characterized in that: The step eight specifically includes the following steps: (1) Experience data collection: The current state S, action A, reward R, and next state S' form an experience sample (S, A, R, S'); (2) Value network update: Calculate TD error: δ = R + γV φ (S')-V φ (S), where V φ is the value network, γ is the discount factor; Update the value network parameter φ: Among them, η v is the learning rate; (3) Policy network update: Calculate the advantage function A(S,A): A(S,A) = δ; Update the policy network parameters θ: Among them, η is the learning rate.

9. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1, characterized in that: The step nine specifically includes the following steps: (1) Data output: save the encrypted data D' to a specified location or transmit it to the target recipient; (2) Record log: record the relevant information of this encryption task.

10. The deep reinforcement learning optimization algorithm based on the adaptive encryption framework according to claim 1 is characterized in that: The step ten specifically includes the following steps: (1) Detect new data: Check whether there is new data to be processed D new ; When there is new data, return to step 1 and continue processing; (2) Update system status: Update system resource status Sr in real time; (3) Continuous model learning: Through continuous interaction and learning, the model’s strategy network and value network are gradually optimized to improve decision-making quality.

Images