An AES encryption method for Internet of Vehicles based on multi-agent reinforcement learning

Abstract

This paper proposes an AES encryption method for the Internet of Vehicles (IoV) based on multi-agent reinforcement learning. By dynamically adjusting the encryption mode and key length for different message types, this method meets the security and latency requirements of diverse IoV services. Based on the current message type and vehicle location, the channel gain, encryption and decryption, transmission latency, and communication security level, the method constructs a system state. It then uses a multi-agent reinforcement learning algorithm to optimize the AES encryption mode and key length used by the vehicle to encrypt messages. This reduces encryption latency and the probability of information leakage due to eavesdropping or interception, improving communication security and ultimately enhancing the reliability and security of information transmission in the IoV.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to an AES encryption method for Internet of Vehicles based on multi-agent reinforcement learning. Background Art

[0002] The complex network topology structure poses a security risk to the channel. Sensitive information transmitted through insecure channels may be subject to active attacks such as interruption, interception and forgery, or passive attacks such as eavesdropping, which puts the privacy of both parties in communication at risk.

[0003] Cryptography is a method of protecting the confidentiality of information during communications. The sender encrypts plaintext information using a specific cryptographic algorithm before transmission. The receiver then uses the same algorithm to decrypt the ciphertext. Because it's the ciphertext, not the plaintext message itself, that's transmitted, cryptographic algorithms can effectively defend against potential attacks during communications.

[0004] Among various cryptographic algorithms, symmetric encryption methods are widely used due to their higher encryption and decryption efficiency and hardware compatibility. The Advanced Encryption Standard (AES), proposed by the National Institute of Standards and Technology in 1998, offers a better balance between security and computational complexity than other block ciphers. In low-latency and high-security communication scenarios (such as the Internet of Vehicles), the AES operating mode and key length need to be dynamically adjusted to meet different service types and requirements. Bojjagani et al. proposed a new system for communication between vehicles, roadside units, fog computing, and cloud servers. This system combines AES and elliptic curve algorithms to encrypt and decrypt information in various information exchange scenarios, enabling secure communication, mutual authentication, and key management between facilities. Raja et al. designed and implemented a traffic alert system for the Internet of Vehicles. AES is used in its secure beacon subsystem to encrypt beacon information, ensuring the security of vehicle communications and preventing unauthorized access and potential security threats. Chinese patent publication number CN113377547A proposes a method for intelligent offloading and security assurance of computing tasks in a 5G edge computing environment. This method considers the security of user tasks during offloading and employs AES encryption based on multi-dimensional biometrics for data transmitted between the mobile edge computing and the user. This method embeds AES in the IoV and improves communication security, but lacks a suitable deployment strategy for highly dynamic and complex application scenarios.

[0005] Reinforcement learning is a machine learning paradigm in which intelligent agents observe the real-time state of the system environment, dynamically adjust task strategies, and optimize their strategy selection based on environmental feedback. In multi-agent systems, the mutual influence of individual member behaviors must also be considered. By introducing communication and collaboration between agents, system performance can be optimized both individually and as a whole. Multi-agent reinforcement learning algorithms can be used to optimize cryptographic applications. Yuan et al. proposed a privacy-preserving multi-agent reinforcement learning strategy that, through the coordinated use of homomorphic encryption and differential privacy, reduces information noise and overhead during communication and improves learning efficiency. Abirami et al. proposed a cloud security strategy based on multi-agent reinforcement learning. This strategy utilizes cryptographic algorithms and transport layer security techniques to protect the system from side-channel attacks that attempt to obtain sensitive information by measuring execution time. These approaches combine multi-agent reinforcement learning algorithms with cryptographic techniques to improve system security, but they ignore the latency requirements of transmission.

[0006] This patent combines a multi-agent reinforcement learning algorithm with the AES cryptographic algorithm. Based on the current transmission information and the attacker's attack strength, it dynamically allocates the key length and working mode of the encryption and decryption process, reduces the delay and overhead of the transmission process, and increases the probability of the vehicle successfully defending against attacks, thereby protecting the safe and efficient communication between vehicles and between vehicles and road test units in the Internet of Vehicles scenario. Summary of the Invention

[0007] Purpose of the invention: The purpose of the present invention is to utilize a multi-agent reinforcement learning algorithm to analyze the message type and message length according to different transmission tasks at each moment, and to dynamically allocate the key length and working mode of the AES algorithm, taking into account the attack intensity of potential attackers in the network, under the premise of secure transmission of vehicle information in the Internet of Vehicles, so as to reduce the delay in communication, improve the vehicle's anti-attack capability and system security level, and thus realize the secure transmission of information.

[0008] The present invention comprises the following steps:

[0009] A vehicle network AES encryption method based on multi-agent reinforcement learning includes the following steps:

[0010] (1) Set up a vehicle network environment, where there are M vehicles in the vehicle network environment. For the mth vehicle, 1≤m≤M, deploy a deep neural network;

[0011] (2) The vehicle executes the task and determines the message type and length for executing the task;

[0012] (3) Dynamically determine the key length and operating mode based on local observations;

[0013] (4) Encrypt the message and upload it to the server;

[0014] (5) The server decrypts and sends feedback information;

[0015] (6) Estimate the number of attacks and security level, and calculate the communication reward;

[0016] (7) Construct and store experience sequences and update deep Q network parameters;

[0017] (8) Repeat steps 2 to 7 until the task is completed.

[0018] Preferably, in step (1), assuming that there are M vehicles, N road test units, and P attackers who can attack the vehicle or server in the current vehicle network environment, for the mth vehicle, 1≤m≤M, a deep neural network Q is deployed that can select an encryption strategy based on the current observation. m (·,·;θ), the parameters of the deep neural network are initialized as

[0019] Preferably, in step (2), in each time interval k, k≥1, the vehicle may perform different types of tasks and needs to send messages to the server and receive messages from the server to interact; for the mth vehicle, assuming that the task type performed in the kth time interval is The length of the message to be sent to perform this task is The unit is bytes; the attacker p determines the target vehicle to be attacked in the kth time interval and estimates the type of task performed by the target vehicle. Based on the estimated task type, the frequency of attack in the kth time interval is determined. and strength

[0020] Preferably, in step (3), local observation Consider the vehicle's current position The type of task executed in the kth time interval and the length of the ciphertext message to be sent Local encryption delay in the k-1th time interval Transmission delay Server-side decryption delay and the estimated number of attacks by the attacker in the k-1th time interval Right now

[0021]

[0022] Preferably, in step (4), based on local observations, the vehicle selects the action to be performed within the kth time interval Includes the key length used for encryption during the kth time interval and working mode Right now A is all possible combinations of key lengths and encryption modes. When selecting an action, there is a probability of e for random selection and a probability of 1-e for selecting the action with the greatest value, i.e. The vehicle encrypts the message according to the selected action and obtains the local encryption delay at that moment And upload the ciphertext to the server.

[0023] Preferably, in step (5): the server uses the corresponding decryption method to obtain the plaintext for the received ciphertext, and the delay is The possible feedback information is sent to the vehicle through the same encryption method. The overall delay during the transmission process is

[0024] Preferably, in step (6): the vehicle estimates the number of attacks launched by the attacker in the kth time interval based on the received information The security level of the communication process in the kth time interval is And calculate the overall delay in the kth time interval And estimate the probability of attack success based on the frequency and intensity of the attacker's attack Based on the above performance, calculate the reward for this round of communication Among them, ω1, ω2, ω3 are weight parameters corresponding to performance.

[0025] Preferably, in step (7): construct an experience sequence And store it in the experience replay buffer pool D m In the experience replay buffer pool D m Randomly sample Z experiences and construct a sample And use optimization algorithm to update the agent's deep Q network parameters

[0026] Compared with the prior art, the present invention has the following significant effects:

[0027] 1. This invention selects the key length and working mode for the AES algorithm by analyzing the vehicle location, task type, task completion efficiency and attacker's attack situation, thereby improving the vehicle's anti-attack capability and reducing the delay of the communication process and encryption and decryption process.

[0028] 2. The present invention effectively estimates the probability of success of an attacker's attack by comprehensively considering the attacker's attack frequency and attack intensity, and measures the security level of the environment by the number of attacks launched, thereby improving the security of the system.

[0029] 3. The present invention considers improving the overall collaborative ability of the system by using a multi-agent reinforcement learning algorithm. DETAILED DESCRIPTION

[0030] In order to more clearly understand the technical content of the present invention, the following embodiments are given to illustrate in detail.

[0031] A multi-agent reinforcement learning-based AES encryption method for connected vehicles includes the following steps:

[0032] Step 1: Assume that there are two vehicles, a drive test unit, and an attacker who may attack both the vehicle and the drive test unit in the current vehicle environment. That is, M = 2, N = 1, and P = 1.

[0033] Step 2: For vehicle m, initialize the vehicle's current position Local encryption delay Transmission delay Server-side decryption delay and the estimated number of times the attacker has attacked in the last round The types of information that a vehicle may send and the corresponding message lengths are: control signal messages (50KB), navigation messages (200KB), safety service messages such as vehicle self-inspection reports (450KB), and entertainment messages such as audio and video (600KB).

[0034] Step 3: Initialize the vehicle-side deep Q network Q m (·,·; θ), initialize the learning rate a=0.0001, discount factor g=0.9, exploration rate e=0.2, and neural network parameters

[0035] Step 4: Vehicle m determines the mission type The length of the message to be sent to perform this task is and build local observation

[0036] Step 5: Based on local observations The vehicle uses the output value of the deep Q network to select the key length of this round of the AES algorithm and working mode Right now AES supports three key lengths: 128 bits, 196 bits, and 256 bits. The operating modes include cipher block chaining mode, electronic codebook mode, cipher feedback mode, counter mode, and output feedback mode, i.e., |A| = 15.

[0037] Step 6: The vehicle encrypts the message to be transmitted based on the selected action and updates the encryption delay And upload it to the server. The server uses the corresponding decryption method to decrypt, and the delay is The same encryption method is used to send the feedback information to the vehicle. The overall delay during the transmission process is

[0038] Step 7: Calculate the overall latency and estimate the safety level and the attacker's attack success probability Rewards are calculated from this and construct an experience sequence Store it in the experience replay buffer pool D m middle.

[0039] Step 8: Extract Z samples from the buffer pool, traverse each sample, use the deep Q network algorithm to estimate the cumulative return, and use the Adam optimizer to update the Q network parameters.

[0040] Step 9: Repeat steps 4-8 until the vehicle completes all local tasks within this period or leaves the current network area.

[0041] The above embodiments are only preferred embodiments of the present invention and should not be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention. The above description is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A vehicle network AES encryption method based on multi-agent reinforcement learning, characterized in that: The following steps are involved: (1) Set up a vehicle network environment, where there are M vehicles in the vehicle network environment. For the mth vehicle, 1≤m≤M, deploy a deep neural network; (2) The vehicle performs the task and determines the message type and length for performing the task; in step (2), in each time interval k, k ≥ 1, the vehicle may perform different types of tasks and needs to send messages to the server and receive messages from the server to interact; For the mth vehicle, assume that the task type performed in the kth time interval is The length of the message to be sent to perform this task is The unit is bytes; the attacker p determines the target vehicle to be attacked in the kth time interval and estimates the type of task performed by the target vehicle. Based on the estimated task type, the frequency of attack in the kth time interval is determined. and strength (3) Dynamically determine the key length and operating mode based on local observations; (4) Encrypt the message and upload it to the server. In step (4), based on local observations, the vehicle selects the action to be performed within the kth time interval. Includes the key length used for encryption during the kth time interval and working mode Right now A is all possible combinations of key lengths and encryption modes. When selecting an action to execute, there is a probability of ε for random selection and a probability of 1-ε for selecting the action with the greatest value, i.e. The vehicle encrypts the message according to the selected action and obtains the local encryption delay at that moment And upload the ciphertext to the server; (5) The server decrypts and sends feedback information. In step (5): the server uses the corresponding decryption method to obtain the plaintext for the received ciphertext. The delay is The possible feedback information is sent to the vehicle through the same encryption method. The overall delay during the transmission process is (6) Estimate the number of attacks and security level, and calculate the communication reward. In step (6): Based on the received information, the vehicle estimates the number of attacks launched by the attacker in the kth time interval. The security level of the communication process in the kth time interval is And calculate the overall delay in the kth time interval And estimate the probability of attack success based on the frequency and intensity of the attacker's attack Based on the above performance, calculate the reward for this round of communication Among them, ω1, ω2, ω3 are weight parameters corresponding to performance; (7) Construct and store the experience sequence and update the deep Q network parameters. In step (7): construct the experience sequence And store it in the experience replay buffer pool D m In the experience replay buffer pool D m Randomly sample Z experiences and construct a sample And use optimization algorithm to update the agent's deep Q network parameters (8) Repeat steps (2) to (7) until the task is completed.

2. The AES encryption method for Internet of Vehicles based on multi-agent reinforcement learning according to claim 1, characterized in that: In step (1), assume that there are M vehicles, N road test units, and P attackers who can attack the vehicle or server in the current Internet of Vehicles environment. For the mth vehicle, 1≤m≤M, deploy a deep neural network Q that can select encryption strategies based on current observations m (·,·;θ), the parameters of the deep neural network are initialized as 3. The AES encryption method for Internet of Vehicles based on multi-agent reinforcement learning according to claim 1, characterized in that: In step (3), local observation Consider the vehicle's current position The type of task executed in the kth time interval and the length of the ciphertext message to be sent Local encryption delay in the k-1th time interval Transmission delay Server-side decryption delay and the estimated number of attacks by the attacker in the k-1th time interval Right now

Images