Safety protection method and device for intelligent networked automobile

Abstract

The invention provides a safety protection method and device for an intelligent networked automobile, and the method comprises the steps: detecting a vehicle control abnormal instruction in a communication process between a target vehicle and a cloud end based on a vehicle control behavior characterization model and an abnormal behavior classification learning model; and based on the malicious traffic identification model, identifying hidden malignant traffic in the process of communication between the target vehicle and the cloud, through a deep reinforcement learning confrontation defense mode, a decision control model is trained according to a vehicle control abnormal instruction and hidden malignant flow, and the decision control model is used for carrying out safety protection on the intelligent network connection vehicle. Therefore, high-precision and high-reliability detection of attacks such as vehicle control instruction tampering attacks and strongly hidden malicious traffic under the intelligent network connection vehicle-cloud cooperation scene is realized, the attack concealment and attack efficiency are effectively improved, the robustness and generalization of the model are enhanced, and the method is suitable for the intelligent network connection vehicle-cloud cooperation scene. And the communication safety, the data safety and the driving safety of the intelligent networked automobile ecological system are improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle networking technology, and in particular to a safety protection method and device for an intelligent networked vehicle. Background Art

[0002] Intelligent connected vehicle vehicle-cloud collaboration provides users with more accurate and personalized services by connecting vehicles to the cloud, and realizes remote control and management of vehicles. However, attacks on vehicle control instructions in vehicle-cloud collaboration and attacks such as malicious encrypted traffic have brought huge challenges to the security of the cloud of the Internet of Vehicles. First, attacks on vehicle control instructions mainly exist in the intelligent connected vehicle cloud platform, which may cause risks such as vehicle control instructions being tampered, missed, and wrongly sent; attacks on the transmission link of the vehicle-cloud collaboration process may cause vehicle control instructions to be intercepted, tampered, and deleted; attacks on the vehicle-cloud interaction interface may lead to risks such as unauthorized access, interface abuse, and instruction tampering. At present, there is a lack of implementation technology routes that can be applied to actual combat in domestic and foreign research on how to identify, detect, and determine vehicle control instruction tampering. Second, the research on highly concealed malicious traffic attacks on vehicle-cloud collaboration is still in its infancy at home and abroad. Malicious attackers may use various concealed means to disguise malicious traffic, such as using encryption, obfuscation and other technologies to steal personal privacy and corporate secrets, which will have a serious impact on vehicle safety. Third, in response to the robustness and generalization issues of the cloud-based intelligent decision-making model for Internet of Vehicles, domestic and foreign studies have proposed methods such as state space perturbation, reward function perturbation, and action space perturbation for adversarial attacks on deep reinforcement learning intelligent decision-making models. In terms of adversarial defense, adversarial training, game theory methods, robust learning, and adversarial detection methods are introduced to improve the robustness of the model. These adversarial attack methods have the problems of poor concealment and low attack efficiency, while adversarial defense methods have high training costs and low generalization, making it difficult to adapt to the interactive training of intelligent decision-making models in large-scale and complex environments. Summary of the invention

[0003] The present invention provides a security protection method and device for intelligent connected vehicles, which are used to solve the problems of poor concealment and low attack efficiency of the prior art counter-attack methods, high training cost and low generalization of the counter-defense methods, and the defects that it is difficult to adapt to the interactive training of intelligent decision-making models in large-scale and complex environments. The method realizes high-precision and high-reliability detection of attacks such as tampering attacks on vehicle control instructions and highly concealed malicious traffic in the vehicle-cloud collaboration scenario of intelligent connected vehicles, effectively improves the concealment and attack efficiency of the attacks, enhances the robustness and generalization of the model, and improves the communication security, data security and driving safety of the intelligent connected vehicle ecosystem.

[0004] The present invention provides a safety protection method for an intelligent network-connected vehicle, comprising the following steps: Based on the vehicle control behavior characterization model and the abnormal behavior classification learning model, abnormal vehicle control instructions during the communication between the target vehicle and the cloud are detected; and based on the malicious traffic identification model, hidden malicious traffic during the communication between the target vehicle and the cloud is identified; Through the adversarial defense method of deep reinforcement learning, a decision control model is trained according to the abnormal vehicle control instructions and the hidden malicious traffic; the trained decision model is used to provide safety protection for intelligent networked vehicles; Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; and the malicious traffic identification model is obtained based on the traffic interaction graph and graph neural network training.

[0005] According to a safety protection method for an intelligent networked vehicle provided by the present invention, the vehicle control behavior characterization model and the abnormal behavior classification learning model are constructed in the following manner: Through the protocol restoration process and reverse engineering process, the structured information of vehicle control instructions is obtained; Analyzing attack methods on the structured information to identify security vulnerabilities of the structured information; Based on the security vulnerabilities, a vehicle control behavior characterization model and an abnormal behavior classification learning model are constructed.

[0006] According to a security protection method for an intelligent connected vehicle provided by the present invention, the malicious traffic identification model is constructed in the following manner: Extract the statistical characteristics and behavioral characteristics of the traffic respectively; Based on the statistical features and the behavioral features, construct a traffic interaction graph; Performing graph reduction, flow clustering and flow fitting operations on the flow interaction graph to obtain a flow feature library; Based on the traffic feature library, the graph neural network is trained to obtain a malicious traffic identification model.

[0007] According to a safety protection method for an intelligent networked vehicle provided by the present invention, a decision control model is trained according to the abnormal vehicle control instructions and the hidden malicious traffic through a deep reinforcement learning adversarial defense method, specifically comprising: Determine the adversarial attack framework of deep reinforcement learning based on the time selection function, random noise and gradient perturbation, and determine the adversarial defense framework of deep reinforcement learning based on the adversarial sample generation method based on gradient descent and the adversarial loss function; Based on the adversarial attack framework and the adversarial defense framework, a decision control model is trained according to the abnormal vehicle control instructions and the hidden malicious traffic.

[0008] According to a safety protection method for an intelligent connected vehicle provided by the present invention, a deep reinforcement learning adversarial attack framework is determined according to a time selection function, random noise and gradient perturbation, specifically comprising: Calculate a time selection function based on the strategy distribution or Q-value distribution of the current state of the deep reinforcement learning agent model; the time selection function is used to determine the time to trigger the attack; Add random noise to the gradient perturbation to obtain adversarial perturbation; Based on the adversarial perturbation, a current state of the agent model is modified.

[0009] According to a safety protection method for an intelligent connected vehicle provided by the present invention, an adversarial defense framework for deep reinforcement learning is determined according to an adversarial sample generation method based on gradient descent and an adversarial loss function, specifically comprising: Generate adversarial samples based on gradient descent method; Based on the adversarial sample and the comprehensive adversarial loss function, training the deep reinforcement learning model; Among them, the comprehensive adversarial loss function includes strategy loss, value function loss, adversarial loss and strategy entropy, and the strategy entropy represents the degree of confusion of different actions output by the strategy corresponding to any state.

[0010] The present invention also provides a safety protection device for an intelligent network-connected vehicle, comprising the following modules: A detection module, for detecting abnormal vehicle control instructions during communication between the target vehicle and the cloud based on a vehicle control behavior characterization model and an abnormal behavior classification learning model; and for identifying hidden malicious traffic during communication between the target vehicle and the cloud based on a malicious traffic identification model; A training module, used for training a decision control model according to the abnormal vehicle control instructions and the hidden malicious traffic through a deep reinforcement learning adversarial defense method; the trained decision model is used for safety protection of intelligent networked vehicles; Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; and the malicious traffic identification model is obtained based on the traffic interaction graph and graph neural network training.

[0011] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a safety protection method for an intelligent connected vehicle as described above is implemented.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the safety protection method for an intelligent connected vehicle as described in any one of the above is implemented.

[0013] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned safety protection methods for intelligent connected vehicles.

[0014] The security protection method and device for intelligent connected vehicles provided by the present invention detect abnormal vehicle control instructions during the communication between the target vehicle and the cloud, identify hidden malicious traffic during the communication between the target vehicle and the cloud, and train a decision control model based on abnormal vehicle control instructions and hidden malicious traffic through the adversarial defense method of deep reinforcement learning, thereby achieving high-precision and high-reliability detection of attacks such as vehicle control instruction tampering attacks and highly hidden malicious traffic in the vehicle-cloud collaboration scenario of intelligent connected vehicles, effectively improving the concealment and efficiency of attacks, enhancing the robustness and generalization of the model, and improving the communication security, data security and driving safety of the intelligent connected vehicle ecosystem. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic flow chart of the safety protection method for an intelligent connected vehicle provided by the present invention.

[0017] Figure 2 An architectural diagram of the vehicle-cloud collaborative safety protection technology for the intelligent connected vehicle provided by the present invention.

[0018] Figure 3 This is a technical architecture diagram of the vehicle control instruction tampering detection technology based on protocol reversal provided by the present invention.

[0019] Figure 4 This is a diagram of the architecture of the intelligent detection technology for highly concealed malicious traffic provided by the present invention.

[0020] Figure 5 This is a diagram of the architecture of the intelligent decision-making model robustness enhancement technology provided by the present invention.

[0021] Figure 6 A schematic structural diagram of the safety protection device for an intelligent connected vehicle provided by the present invention.

[0022] Figure 7 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Figure 1 is a flow chart of the safety protection method for the intelligent networked vehicle provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 100: Detect abnormal vehicle control instructions during the communication between the target vehicle and the cloud based on the vehicle control behavior characterization model and the abnormal behavior classification learning model; and identify hidden malicious traffic during the communication between the target vehicle and the cloud based on the malicious traffic identification model.

[0025] Step 101: Through the adversarial defense method of deep reinforcement learning, a decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic; the trained decision model is used to provide security protection for intelligent connected vehicles.

[0026] Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; the malicious traffic identification model is based on the traffic interaction graph and graph neural network training.

[0027] Specifically, in the embodiment of the present invention, it is first necessary to construct a vehicle control behavior characterization model, an abnormal behavior classification learning model and a malicious traffic identification model.

[0028] Among them, the vehicle control behavior characterization model is used to extract the characteristics of vehicle control behavior. Its purpose is to quantify the vehicle's command behavior pattern in communication. It can extract the multi-dimensional behavior characteristics of the target vehicle through communication logs, vehicle sensor data (such as acceleration, braking, steering, etc.), collect time series features, and then model the features through machine learning (such as clustering, anomaly detection) or deep learning (such as LSTM, Transformer).

[0029] The abnormal behavior classification learning model can be established based on known abnormal behavior data (such as historical attack behaviors or specific samples). Normal instructions and known abnormal instructions can be collected, annotated and formed into training samples, and then a classifier can be built using traditional classification algorithms (such as support vector machines, random forests) or deep learning models (such as CNN, RNN). Finally, the abnormal behavior classification learning model is obtained after training.

[0030] In an embodiment of the present invention, the abnormal behavior characteristics of the target vehicle can be obtained through the vehicle control behavior characterization model to form an abnormal behavior data sequence, which is then input into the abnormal behavior classification learning model to determine the control behavior type of the target vehicle, thereby detecting abnormal vehicle control instructions during the communication between the target vehicle and the cloud.

[0031] In the process of modeling the malicious traffic identification model, the flow interaction graph can be used to extract the traffic behavior characteristics in the communication between the target vehicle and the cloud, and the graph neural network (GNN) can be applied to model the traffic and identify abnormal traffic patterns. The malicious traffic identification model can be trained through malicious traffic labels in a supervised manner or cluster analysis in an unsupervised manner to obtain the trained malicious traffic identification model.

[0032] In an embodiment of the present invention, based on the malicious traffic identification model, hidden malicious traffic in the communication process between the target vehicle and the cloud can be identified, and the hidden malicious traffic can be classified into known attack types (such as denial of service, data injection, etc.) or unknown attack types (obtained by clustering analysis using unsupervised methods).

[0033] After obtaining the abnormal vehicle control instructions and hidden malicious traffic during the communication between the target vehicle and the cloud, the abnormal vehicle control instructions and hidden malicious traffic can be used as input features to describe the potential attack behaviors and security threats in the communication between the vehicle and the cloud, and generate adversarial samples. Then, adversarial training is performed based on the adversarial samples combined with the loss function to train a decision control model.

[0034] This decision-making and control model can dynamically detect and respond to control anomalies of intelligent connected vehicles. Through the online learning capability of deep reinforcement learning, it can achieve real-time protection of vehicles in complex attack scenarios. Through adversarial training, the model's ability to identify and defend against unknown abnormal behaviors and attacks can be significantly improved, enabling the decision-making and control model to respond to attack behaviors in real time and maintain the safety and stability of vehicle operation.

[0035] The security protection method for intelligent connected vehicles provided by the present invention detects abnormal vehicle control instructions in the process of communication between the target vehicle and the cloud, identifies hidden malicious traffic in the process of communication between the target vehicle and the cloud, and trains a decision control model according to abnormal vehicle control instructions and hidden malicious traffic through a deep reinforcement learning adversarial defense method, thereby achieving high-precision and high-reliability detection of attacks such as vehicle control instruction tampering attacks and highly hidden malicious traffic in the vehicle-cloud collaboration scenario of intelligent connected vehicles, effectively improving the concealment and efficiency of attacks, enhancing the robustness and generalization of the model, and improving the communication security, data security and driving safety of the intelligent connected vehicle ecosystem.

[0036] According to a safety protection method for an intelligent networked vehicle provided by the present invention, a vehicle control behavior characterization model and an abnormal behavior classification learning model are constructed in the following manner: Through the protocol restoration process and reverse engineering process, the structured information of vehicle control instructions is obtained; Analyze attack methods on structured information and identify security vulnerabilities of structured information; Based on security vulnerabilities, a vehicle control behavior characterization model and an abnormal behavior classification learning model are constructed.

[0037] Specifically, the protocol restoration process is to extract and parse the structured information of control instructions from the vehicle communication traffic, and the reverse engineering process is to deeply analyze the interaction mechanism between the vehicle and the cloud, so as to clarify the logic of instruction generation and parsing.

[0038] In the process of protocol restoration, we can first study the implementation methods of data authenticity, integrity, confidentiality, anti-replay, behavior anti-repudiation and privacy protection in the vehicle control command data transmission process such as HTTP, MQTT, COAP, SOAP, etc., and implement the parsing and restoration of vehicle cloud commands such as real-time vehicle monitoring and vehicle remote control based on ACL (Access Control List) flow management, HyperScan and templated control protocols. In the process of reverse engineering, we can clarify the logic of command generation and parsing based on decompilation, debugging, dynamic analysis and static analysis.

[0039] Among them, ACL flow management is used to filter and manage data flows to achieve precise control of specific protocols, source addresses, destination addresses, ports, etc.; HyperScan is a high-performance, multi-mode regular expression matching engine that can quickly parse and identify specific protocol features in network traffic; templated control protocol deep restoration refers to the restoration and parsing of vehicle-cloud interaction instructions based on predefined protocol templates to recover the structured information of the instructions.

[0040] Decompilation refers to the use of decompilation tools to perform static analysis on the vehicle-cloud communication program or firmware to restore the logical flow of the control protocol; debugging refers to running the program with the help of a debugger to monitor its processing of the network communication protocol; static analysis refers to static analysis of the code of the vehicle-mounted communication firmware or application to extract the protocol format and interaction logic; dynamic analysis refers to simulating vehicle-cloud interaction in a test environment to capture and analyze dynamically generated instructions and responses.

[0041] After obtaining the structured information of the vehicle control command, the attack means analysis can be performed on the structured information to identify the security loopholes of the structured information. The attack means analysis process can be performed by protocol sniffing, protocol simulation and man-in-the-middle attack.

[0042] During the protocol sniffing process, tools can be used to capture data packets of the vehicle-cloud communication protocol and analyze their content and transmission logic; during the protocol simulation process, tools can be used to construct and send simulation data packets based on the restored protocol to verify the protocol behavior. During the man-in-the-middle attack process, an intermediate proxy can be set up to intercept and tamper with the vehicle-cloud communication data and evaluate potential security vulnerabilities.

[0043] After identifying the security vulnerability, the characteristics of the vehicle control behavior can be extracted based on the vehicle control instructions and vehicle control behaviors corresponding to the security vulnerability, and the sequence characteristics of the control behavior can be modeled using a time series model to obtain a vehicle control behavior characterization model, which is used to describe the vehicle control behavior and provide a benchmark for subsequent anomaly detection. A labeled data set can also be constructed based on the vehicle control instructions and vehicle control behaviors corresponding to the security vulnerability, and a classifier can be constructed using traditional classification algorithms (such as support vector machines, random forests) or deep learning models (such as CNN, RNN), and finally the training is completed to obtain an abnormal behavior classification learning model.

[0044] According to a security protection method for an intelligent connected vehicle provided by the present invention, a malicious traffic identification model is constructed in the following manner: Extract the statistical characteristics and behavioral characteristics of the traffic respectively; Construct traffic interaction graph based on statistical and behavioral characteristics; Perform graph reduction, flow clustering and flow fitting operations on the traffic interaction graph to obtain a traffic feature library; Based on the traffic feature library, the graph neural network is trained to obtain a malicious traffic identification model.

[0045] Specifically, in the process of building a malicious traffic identification model, we can first extract the statistical characteristics and behavioral characteristics of the traffic. Among them, the statistical characteristics are used to describe the overall characteristics of network traffic. Network traffic capture tools can be used to collect traffic packets, or statistical characteristics can be extracted based on the metadata of traffic logs or data packets; behavioral characteristics are used to describe the dynamic change pattern of traffic, mainly analyzing the time series and behavioral patterns of traffic. This can be done by dynamically capturing the traffic time series, statistically analyzing the behavioral patterns between traffic (such as response delay, traffic peak position), or using protocol analysis tools to parse behavioral characteristics.

[0046] Then, the traffic interaction graph can be constructed based on the obtained statistical and behavioral features. In the traffic interaction graph, each node corresponds to a network entity, and the edge represents the communication flow between two network entities, including statistical and behavioral features.

[0047] After constructing the traffic interaction graph, by analyzing the traffic interaction graph, key nodes, density subgraphs, isolated nodes, etc. of strong hidden malicious traffic can be identified for feature extraction. The constructed traffic interaction graph can be subjected to graph reduction, flow clustering, and flow fitting operations. The purpose of graph reduction is to simplify the traffic interaction graph, remove unimportant nodes and edges, and merge similar nodes; the purpose of flow clustering is to group together traffic with similar characteristics, statistically analyze their common behavior patterns such as packet size, duration, frequency, and perform clustering; the purpose of flow fitting is to evaluate the distribution and characteristics of traffic, generate standardized features from traffic patterns, and establish a traffic feature library, thereby improving the detection system's intelligent perception and real-time response capabilities to strong hidden malicious traffic.

[0048] After the traffic feature library is established, the graph neural network can be trained based on the traffic feature library. During the training process, the nodes, edges and weights of the traffic interaction graph can be input, and the labels (normal / malicious) of the traffic feature library can be used for supervised training. The trained malicious traffic identification model can classify the input traffic interaction graph and identify whether it contains malicious traffic.

[0049] According to a safety protection method for an intelligent networked vehicle provided by the present invention, a decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic through a deep reinforcement learning adversarial defense method, specifically including: Determine the adversarial attack framework of deep reinforcement learning based on the time selection function, random noise and gradient perturbation, and determine the adversarial defense framework of deep reinforcement learning based on the adversarial sample generation method based on gradient descent and the adversarial loss function; Based on the adversarial attack framework and adversarial defense framework, the decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic.

[0050] Specifically, in the embodiments of the present invention, improvements are made to the adversarial attack framework and adversarial defense framework of deep reinforcement learning.

[0051] Among them, based on the time selection function (TS Function), random noise and gradient perturbation, a deep reinforcement learning (DRL) adversarial attack framework is designed, which can effectively utilize the key attack moments and enhance the concealment and efficiency of the attack.

[0052] Among them, the time selection function can be used to determine the key attack moment, reduce the attack frequency, and improve the attack efficiency. By constructing adversarial samples through random noise and gradient perturbation, it can make it difficult for the agent to detect and effectively destroy the agent's strategy learning, increase the concealment and diversity of adversarial attacks, and bypass defense mechanisms. Modify the state of the environment at the selected time, interfere with the strategy or value estimation of the reinforcement learning model, destroy the learning ability or decision-making process of the reinforcement learning agent, and make its behavior deviate from the optimal strategy.

[0053] According to a safety protection method for an intelligent connected vehicle provided by the present invention, a deep reinforcement learning adversarial attack framework is determined according to a time selection function, random noise and gradient perturbation, specifically comprising: Based on the strategy distribution or Q-value distribution of the current state of the deep reinforcement learning agent model, the time selection function is calculated; the time selection function is used to determine the time to trigger the attack; Add random noise to the gradient perturbation to obtain adversarial perturbation; Based on the adversarial perturbation, the current state of the agent model is modified.

[0054] Specifically, the role of the time selection function is to find the most vulnerable time point of the agent in the reinforcement learning process, that is, the critical attack moment. The selection of the critical moment is based on the strategy distribution or Q value distribution of the reinforcement learning agent.

[0055] For deep reinforcement learning algorithms based on policy distribution, they mainly focus on policy Parameterize, calculate the policy gradient about the action, and then adjust the action along the gradient direction to gradually obtain the optimal policy. Its output Indicates that when the status is When, action Satisfy the parameters Therefore, the embodiment of the present invention selects the critical moment through the probability distribution of the action of the agent in a certain state. The time selection function can be expressed as: in represents the maximum value of the output action probability in a given state, represents the minimum value of the probability of outputting an action in a given state, represents the average probability of outputting an action in a given state, Score dynamic actions. (threshold), it is considered that the current moment It is a critical moment that triggers the attack.

[0056] For DRL algorithms based on value functions such as DQN, deep neural networks are mainly used to approximate the reward value function, and its Q value represents the action value distribution in this state. The design of the time selection function adopts a similar idea to the strategy-based algorithm, that is, the continuous action value is also converted into a probability distribution, so the Q value needs to be normalized by the softmax function first, and the time selection function can be expressed as: in It represents the process of normalizing the Q value through the softmax function. , and are the maximum, minimum and average values ​​of the normalized Q value. (threshold), it is considered that the current moment It is a critical moment that triggers the attack.

[0057] The time selection function can help select the most critical moment to ensure the number of attacks is as small as possible and achieve efficient attacks.

[0058] After obtaining the information of the attack time point, it is necessary to add the perturbation generated by the attack algorithm to the state of the environment at the attack time point. Gradient information is a key factor in controlling the learning effect of the model. Attackers can use this to generate adversarial perturbations by making small gradient perturbations to the input samples, so as to make the agent take wrong actions or produce unexpected results in the environment.

[0059] Therefore, the embodiment of the present invention proposes to add a small random perturbation to the input sample before injecting the gradient perturbation to increase the randomness and diversity of the perturbation, making the attack more effective and concealed, that is, in, is the generated adversarial example, is the original input sample, is the disturbance amplitude, is the gradient direction calculated by the attack algorithm, is a random noise variable.

[0060] Combining random noise with the disturbance generated by the existing adversarial attack algorithm can produce the final disturbance attack. However, whether this attack has an impact on the agent remains to be determined. This is because reinforcement learning has the characteristics of delayed rewards and goal orientation. The reinforcement learning agent can be run in the modified environment to observe its behavior changes and cumulative rewards, and optimize according to the attack effect (such as the decrease in the agent's reward). and .

[0061] In reinforcement learning, the ultimate goal of adversarial attack is to minimize the agent’s objective function (cumulative reward ),Right now: in is the set of perturbation attacks, Indicates time Rewards, is the discount factor.

[0062] In order to improve the robustness of the deep reinforcement learning model under adversarial attacks, the embodiment of the present invention can use the adversarial sample generation method based on gradient descent and the adversarial loss function to construct an adversarial defense framework. This framework is innovatively designed in two aspects: adversarial sample generation and loss optimization, balancing the standard performance and robustness of the model.

[0063] According to a safety protection method for an intelligent connected vehicle provided by the present invention, an adversarial defense framework for deep reinforcement learning is determined according to an adversarial sample generation method based on gradient descent and an adversarial loss function, specifically comprising: Generate adversarial samples based on gradient descent method; Training deep reinforcement learning models based on adversarial examples and comprehensive adversarial loss functions; Among them, the comprehensive adversarial loss function includes strategy loss, value function loss, adversarial loss and strategy entropy. The strategy entropy represents the degree of confusion of different actions output by the strategy corresponding to any state.

[0064] Specifically, in order to reduce the training cost, the embodiment of the present invention uses an adversarial sample generation method based on gradient descent. When constructing an adversarial sample generation algorithm, it is necessary to define an objective function to guide the process of generating adversarial samples, and the choice of the objective function is critical because it determines the goal that the attacker wants to achieve and the characteristics of the adversarial sample. Assuming the state Next select action The probability of ,state The action probability distribution under ,state The worst action under , then the objective function that maximizes the single attack benefit is: in , negative logarithmic function Mapping smaller probability values ​​to larger objective function values ​​is done to encourage the generation of adversarial samples in such a way that the probability distribution deviates from the expected distribution of the original samples.

[0065] in, Adversarial probability distribution ,for: In an embodiment of the present invention, a comprehensive adversarial loss function is also designed. The loss function includes strategy loss, value function loss, adversarial loss and strategy entropy. The strategy entropy represents the degree of confusion of different actions output by the strategy corresponding to any state.

[0066] Among them, policy loss and value function loss are common loss functions of the DRL model. The embodiment of the present invention optimizes the common loss function to obtain a comprehensive adversarial loss function, which consists of three parts: in represents the original loss function of the DRL algorithm (i.e., policy loss and value function loss), represents the anti-loss term designed in the embodiment of the present invention, It represents the policy entropy introduced by the embodiment of the present invention. is a hyperparameter that trades off between standard performance and robust performance, with a value between 0 and 1. The model is undergoing standardized training.

[0067] The adversarial loss term and policy entropy mechanism are introduced below.

[0068] Anti-loss of the embodiment of the present invention It is designed based on Neural Network Robustness Certification Bounds, which is a technique for neural network verification and robustness analysis. It derives the range of neural network output by maintaining the input and output boundaries in each layer of the neural network. For a given neural network, suppose that at the first Layer Input The output after ,The goal of the robust certification bound is to compute the layer-wise lower and upper bounds of the neural network, expressed as and , and for the input Any satisfaction The disturbance , both have: After introducing the adversarial loss, the robustness of the model is improved but some performance is sacrificed. This is because the weight of the original loss function of the algorithm is reduced, which will inevitably affect the standard performance of the model under noise-free conditions. Therefore, the embodiment of the present invention introduces the policy entropy , used to encourage the agent to explore the state and action space.

[0069] Entropy is a measure of the uncertainty or information content of a random variable. In information theory, entropy is used to measure the average amount of information or average uncertainty. For a discrete random variable, its entropy can be calculated by the following formula: in, is the set of values ​​of the random variable, is a random variable taking the value The higher the entropy, the greater the uncertainty of the random variable and the greater the amount of information. When the probability distribution of the random variable is more concentrated, the entropy will decrease, indicating a decrease in the amount of information and a decrease in uncertainty.

[0070] For reinforcement learning, at the beginning of training, the policy network is randomly initialized, so the probability distribution of the policy for all actions is equal, which means that the entropy of the policy is the largest. As training proceeds, the policy will gradually learn better action selection strategies, resulting in different selection probabilities for different actions, and the entropy of the policy will gradually decrease. This phenomenon makes the agent more inclined to explore the environment in the early stages of learning, and gradually tend to choose a certain action in the later stages. However, some reinforcement learning algorithms often face the problem of the entropy of the policy decreasing too quickly during training, causing the policy to converge prematurely, limiting the exploration of the environment. To solve this problem, an embodiment of the present invention uses the entropy of the policy as part of the loss function to maximize the entropy of the policy while learning to maximize the reward. The purpose of this is to maintain the diversity and uncertainty of the policy to promote the exploratory nature of the agent. In an embodiment of the present invention, the policy entropy represents the value of the policy for a certain state. ,Strategy The degree of confusion of the different actions that may be output, namely: in is the state space set, normal distribution .

[0071] Mix deep learning models on adversarial samples and normal samples and dynamically adjust the trade-off factor Evaluate the performance of the model under normal tasks and adversarial attacks to ensure that both standard performance and robustness meet expectations.

[0072] Through adversarial sample training, the model can effectively resist unknown attacks, thereby enhancing the robustness of the model; by designing a comprehensive adversarial loss function, the model can achieve a balance between robustness and standard performance; and the comprehensive adversarial loss function includes strategy entropy, which can encourage exploration and improve the model's adaptability to unseen environments.

[0073] The following further illustrates the safety protection method for the intelligent connected vehicle provided by the present invention through embodiments in specific application scenarios.

[0074] Figure 2 The architecture diagram of the vehicle-cloud collaborative safety protection technology for the intelligent connected vehicle provided by the present invention is as follows: Figure 2 As shown in the figure, the technical architecture consists of vehicle control instruction tampering detection technology based on protocol reversal, intelligent detection technology for strong concealed malicious traffic, and intelligent decision model robustness enhancement technology. It is applied to security protection of remote control, status monitoring, safety control, software update and other scenarios in intelligent connected vehicle vehicle-cloud collaborative communication. In order to solve the problems of illegal acquisition and tampering of vehicle control instructions during communication between vehicles and the cloud, low identification accuracy of strongly concealed malicious traffic, and limited decision-making efficiency, breakthroughs have been made in key technologies such as vehicle control instruction tampering technology based on protocol reversal, intelligent detection technology of strongly concealed malicious traffic based on traffic interaction graph, attack framework based on time selection function and attack trade-off strategy (TS-AO), and robust reinforcement learning defense framework based on adversarial loss optimization (ALO). This has achieved high-precision and high-reliability detection of vehicle control instruction tampering attacks, strongly concealed malicious traffic and other attacks in the vehicle-cloud collaboration scenario of intelligent connected vehicles, effectively improving the concealment and efficiency of attacks, enhancing the robustness and generalization of the model, and improving the communication security, data security and driving safety of the intelligent connected vehicle ecosystem.

[0075] (1) Vehicle control command tampering detection technology based on protocol reversal Figure 3 The vehicle control instruction tampering detection technology architecture diagram based on protocol reversal provided by the present invention is as follows: Figure 3As shown, this embodiment mainly studies the implementation methods of HTTP, MQTT, COAP, SOAP and other vehicle control command data transmission processes and data authenticity, integrity, confidentiality, anti-replay, behavior anti-repudiation and privacy protection. In-depth analysis of possible identity forgery, communication protocol attacks, denial of service attacks and software vulnerability exploits on the vehicle control cloud platform. A model-based abnormal instruction detection method is used to establish a classification learning model for existing abnormal behaviors and a vehicle control behavior characterization model, thereby realizing the discovery and early warning of vehicle control instruction tampering in cloud detection.

[0076] This embodiment studies the deep restoration technology of the control protocol based on ACL flow management, HyperScan and template.

[0077] Specifically, by analyzing the flow characteristics of the control protocol and the characteristics of related value fields, ACL flow management technology is used to extract control protocol data packets from massive data and effectively sort, reorganize and restore them. Through pattern matching and HyperScan technology, key information in the control protocol data stream is accurately matched and extracted to achieve the parsing and restoration of vehicle cloud commands such as real-time vehicle monitoring and vehicle remote control.

[0078] This embodiment adopts reverse engineering technology based on decompilation, debugging, dynamic analysis, static analysis and attack analysis technology based on protocol sniffing, protocol simulation, man-in-the-middle attack, etc., deeply analyzes the vehicle-cloud interaction mechanism, performs automatic deconstruction and reverse analysis of the APP through the APP sandbox, deeply analyzes the interaction mechanism, data encryption and encoding process in the vehicle control protocol, clarifies the encryption algorithm, encryption parameters, and encoding specifications, and the system simulates its data encryption or encoding process to ensure that the collected and submitted data is consistent with the normal application's own data, achieve the purpose of data collection and acquisition, identify potential security vulnerabilities or information leakage risks, and provide analysis means and basic basis for vehicle command tampering detection.

[0079] Finally, a model-based abnormal instruction detection method is adopted to establish a vehicle control behavior characterization model to form an abnormal behavior detection classification input data parameter sequence, improve the efficiency of abnormal behavior detection, and establish a classification learning model based on existing abnormal behaviors to solve the detection problems in multiple data packets and unknown abnormal behaviors.

[0080] (2) Intelligent detection technology for highly concealed malicious traffic based on traffic interaction graph Smart connected cars are facing new information security challenges. Rich connectivity also means that smart cars have more attack points and surfaces that can be exploited by car hackers. Malicious attackers may use various covert means to disguise malicious traffic, such as encryption, obfuscation and other technologies to evade detection. Compared with traditional malicious network traffic, strong covert malicious traffic uses advanced technical means such as encryption and covert channels to cleverly hide the behavior of traffic, aiming to evade general detection methods and make it more difficult to detect. The concealment of this traffic makes it difficult for traditional security protection systems to effectively identify and prevent its malicious activities. Therefore, there is an urgent need to improve the sensitivity and accuracy of the detection system to strong covert malicious traffic.

[0081] Figure 4 The intelligent detection technology architecture diagram of the strong hidden malicious traffic provided by the present invention is as follows: Figure 4 As shown in the figure, by deeply studying the characteristics and behavior patterns of strongly concealed malicious traffic, the problem of low distinction and difficulty in identification between benign communication behavior and strongly concealed malicious communication behavior at the level of traditional statistical features such as data packet size and frequency is analyzed, and a strongly concealed malicious traffic characteristic analysis method based on traffic interaction graph is proposed. By analyzing the interactive behavior of traffic, a traffic feature library is constructed by combining interaction graph technology, graph reduction technology, flow clustering technology and flow fitting technology, thereby improving the detection system's intelligent perception and real-time response capabilities to strongly concealed malicious traffic.

[0082] After data collection, basic communication information including source and destination IP, port, protocol type, packet size, timestamp, etc. of interaction data is used to construct an interaction graph. In the interaction graph, nodes represent network entities and edges represent the communication relationship between these entities. By analyzing the interaction graph, key nodes, density subgraphs, isolated nodes, etc. of highly concealed malicious traffic are identified for feature extraction. The interaction graph of highly concealed malicious traffic is simplified by graph reduction technology, unimportant nodes and edges are removed, and similar nodes are merged. Flow clustering technology is used to cluster malicious traffic with similar characteristics, and their common behavior patterns such as packet size, duration, frequency, etc. are statistically analyzed and clustered. At the same time, the distribution and characteristics of malicious traffic are evaluated by flow fitting methods, and a specific probability distribution model is output and reasonable predictions are made.

[0083] The focus is on studying efficient identification models and methods for highly concealed malicious traffic in the cloud network of intelligent connected vehicles, establishing a multi-feature, fine-grained, and highly timely knowledge base of highly concealed malicious traffic and its early communication characteristics, and proposing an accurate identification method for highly concealed malicious traffic based on traffic interaction graphs.

[0084] (3) Intelligent decision-making model robustness enhancement technology Figure 5 The intelligent decision model robustness enhancement technology architecture diagram provided by the present invention is as follows: Figure 5As shown, this embodiment comprehensively considers the impact of adversarial disturbances on the intelligent decision-making model of intelligent connected vehicles in large-scale and complex traffic scenarios, and explores reliable reinforcement learning algorithm adversarial attacks and defense solutions. On the one hand, this embodiment uses adversarial attacks against deep reinforcement learning algorithms as a starting point to solve the problems of poor concealment and low attack efficiency of traditional adversarial attacks. On the other hand, this embodiment uses adversarial defense against deep reinforcement learning algorithms as a starting point to solve the problems of high training costs of traditional adversarial defense methods, difficulty in achieving a balance between algorithm robustness and performance, and poor generalization.

[0085] 1. TS-AO adversarial attack framework In the process of reinforcement learning, frequent attacks will cause the agent to be alert, and ineffective attacks will reduce the efficiency of the attack, so the timing of the attack is very important. Therefore, this embodiment proposes an attack framework TS-AO based on a time selection function and an attack trade-off strategy.

[0086] First, for deep reinforcement learning algorithms based on policy gradients, they are mainly Parameterize, calculate the policy gradient about the action, and then adjust the action along the gradient direction to gradually obtain the optimal policy. Its output Indicates that when the status is When, action Satisfy the parameters Therefore, this embodiment selects the critical moment through the probability distribution of the action of the agent in a certain state, and its function can be expressed as: in represents the maximum value of the output action probability in a given state, represents the minimum value of the probability of outputting an action in a given state, It represents the average probability of outputting an action in a given state. Score dynamic action.

[0087] For DRL algorithms based on value functions such as DQN, deep neural networks are mainly used to approximate the reward value function, and its Q value represents the action value distribution in this state. The design of the time selection function This embodiment adopts a similar idea to the strategy-based algorithm, that is, the continuous action value is also converted into a probability distribution, so the Q value needs to be normalized by the softmax function first, and then the function can be expressed as: in It means that the Q value is normalized by the softmax function. The TS function can help us select the most critical moment, ensure the number of attacks is as small as possible, and achieve efficient attack.

[0088] After obtaining the information of the attack time point, it is necessary to add the disturbance generated by the attack algorithm to the environmental state at the attack time point. Gradient information is a key factor in controlling the learning effect of the model. Attackers can use this to generate adversarial disturbances by making small gradient perturbations to the input samples, so as to make the agent take wrong actions or produce unexpected results in the environment. Therefore, this embodiment proposes to add a small random perturbation to the input sample before injecting the gradient perturbation to increase the randomness and diversity of the perturbation, making the attack more effective and hidden, that is: in, is the generated adversarial example, is the original input sample, is the disturbance amplitude, is the gradient direction calculated by the attack algorithm, is a random noise variable.

[0089] Combining random noise with the disturbance generated by the existing adversarial attack algorithm can produce the final disturbance attack, but whether this attack has an impact on the agent remains to be determined. This is because reinforcement learning has the characteristics of delayed rewards and goal orientation. In reinforcement learning, the ultimate goal of adversarial attack is to minimize the objective function of the agent, that is: in is the set of disturbance attacks.

[0090] 2. ALO Countermeasures Defense Framework At present, most defense algorithms are developed for traditional deep learning tasks, and few defense algorithms are designed for deep RL agents. Therefore, this embodiment brings the defense methods for data and models in deep learning into reinforcement learning, and proposes a new idea, namely, a robust reinforcement learning defense framework ALO based on adversarial loss optimization.

[0091] In order to reduce the training cost, this embodiment uses an adversarial sample generation method based on gradient descent. When constructing an adversarial sample generation algorithm, it is necessary to define an objective function to guide the process of generating adversarial samples. The choice of objective function is critical because it determines the goal that the attacker wants to achieve and the characteristics of the adversarial sample. Assume that the state Next select action The probability of ,state The action probability distribution under ,state The worst action under , then the objective function that maximizes the single attack benefit is: in , negative logarithmic function Mapping smaller probability values ​​to larger objective function values ​​is done to encourage the generation of adversarial samples in such a way that the probability distribution deviates from the expected distribution of the original samples. Adversarial probability distribution ,for: The loss function of deep reinforcement learning models plays a vital role in the training process. They have a significant impact on the learning and performance of the model. The loss function of a DRL model usually consists of two parts: policy loss and value function loss.

[0092] This embodiment optimizes the loss function based on adversarial loss, which consists of three parts: in Represents the original loss function of the DRL algorithm, represents the anti-loss term to be designed in this embodiment, It represents the policy entropy introduced in this embodiment. is a hyperparameter that trades off between standard performance and robust performance, with a value between 0 and 1. The model is undergoing standardized training. The adversarial loss term and policy entropy mechanism are introduced below.

[0093] The adversarial loss of this embodiment It is designed based on the neural network robustness certification boundary, which is a technique for neural network verification and robustness analysis. It derives the range of neural network output by maintaining the input and output boundaries in each layer of the neural network. For a given neural network, suppose that in the first layer of the neural network Layer Input The output after ,The goal of the robust certification bound is to compute the layer-wise lower and upper bounds of the neural network, expressed as and , and for the input Any satisfaction The disturbance , both have: After introducing the adversarial loss, the robustness of the model is improved but some performance is sacrificed. This is because the weight of the original loss function of the algorithm is reduced, which will inevitably affect the standard performance of the model under noise-free conditions. Therefore, the policy entropy introduced in this embodiment , which is used to encourage the agent to explore the state and action space. Entropy refers to the measure of uncertainty or information content of a random variable. In information theory, entropy is used to measure the average amount of information or average uncertainty of information. For a discrete random variable, its entropy can be calculated by the following formula: in, is the set of values ​​of the random variable, is a random variable taking the value The higher the entropy, the greater the uncertainty of the random variable and the greater the amount of information. When the probability distribution of the random variable is more concentrated, the entropy will decrease, indicating a decrease in the amount of information and a decrease in uncertainty.

[0094] For reinforcement learning, at the beginning of training, the policy network is randomly initialized, so the probability distribution of the policy for all actions is equal, which means that the entropy of the policy is the largest. As the training proceeds, the policy will gradually learn a better action selection strategy, resulting in different selection probabilities for different actions, and the entropy of the policy will gradually decrease. This phenomenon makes the agent more inclined to explore the environment in the early stages of learning, and gradually tend to choose a certain action in the later stages. However, some reinforcement learning algorithms often face the problem of too fast entropy drop of the policy during training, resulting in premature convergence of the policy, which limits the exploration of the environment. In order to solve this problem, this embodiment uses the entropy of the policy as part of the loss function to maximize the entropy of the policy while learning to maximize the reward. The purpose of this is to maintain the diversity and uncertainty of the policy to promote the exploratory nature of the agent.

[0095] In this embodiment, the policy entropy represents the ,Strategy The degree of confusion of the different actions that may be output, namely: in is the state space set, normal distribution .

[0096] The "decision-control" algorithm of intelligent connected vehicles based on deep reinforcement learning technology has the advantage of autonomous learning and is suitable for complex modeling and large-scale scenarios; however, since the vehicle's "decision-control" model is easily disturbed by environmental changes during the learning process, it faces the practical problem of poor adaptability to adversarial disturbance scenarios in large-scale and complex traffic environments. Therefore, improving the robustness and generalization of deep reinforcement learning algorithms to perceived environmental disturbances is of great significance to improving the safety of intelligent connected vehicles. The basis of adversarial disturbances is the design of reinforcement learning models and algorithms. The key issue is to use adversarial attack and defense theory to explain the relationship between reinforcement learning algorithms and environmental disturbances, so as to design and optimize the corresponding reinforcement learning algorithms according to the operational needs of large-scale and complex traffic scenarios, and improve the robustness and generalization performance of the "decision-control" model.

[0097] The safety protection device of the intelligent connected vehicle provided by the present invention is described below. The safety protection device of the intelligent connected vehicle described below and the safety protection method of the intelligent connected vehicle described above can be referenced to each other.

[0098] Figure 6 Schematic diagram of the structure of the safety protection device of the intelligent network-connected vehicle provided by the present invention. Figure 6 As shown, the device includes the following modules: The detection module 600 is used to detect abnormal vehicle control instructions during the communication between the target vehicle and the cloud based on the vehicle control behavior characterization model and the abnormal behavior classification learning model; and to identify hidden malicious traffic during the communication between the target vehicle and the cloud based on the malicious traffic identification model; The training module 610 is used to train a decision control model according to abnormal vehicle control instructions and hidden malicious traffic through a deep reinforcement learning adversarial defense method; the trained decision model is used to provide safety protection for intelligent networked vehicles; Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; the malicious traffic identification model is based on the traffic interaction graph and graph neural network training.

[0099] According to a safety protection device for an intelligent networked vehicle provided by the present invention, a vehicle control behavior characterization model and an abnormal behavior classification learning model are constructed in the following manner: Through the protocol restoration process and reverse engineering process, the structured information of vehicle control instructions is obtained; Analyze attack methods on structured information and identify security vulnerabilities of structured information; Based on security vulnerabilities, a vehicle control behavior characterization model and an abnormal behavior classification learning model are constructed.

[0100] According to a safety protection device for an intelligent network-connected vehicle provided by the present invention, a malicious traffic identification model is constructed in the following manner: Extract the statistical characteristics and behavioral characteristics of the traffic respectively; Construct traffic interaction graph based on statistical and behavioral characteristics; Perform graph reduction, flow clustering and flow fitting operations on the traffic interaction graph to obtain a traffic feature library; Based on the traffic feature library, the graph neural network is trained to obtain a malicious traffic identification model.

[0101] According to a safety protection device for an intelligent networked vehicle provided by the present invention, a decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic through a deep reinforcement learning confrontation defense method, specifically including: Determine the adversarial attack framework of deep reinforcement learning based on the time selection function, random noise and gradient perturbation, and determine the adversarial defense framework of deep reinforcement learning based on the adversarial sample generation method based on gradient descent and the adversarial loss function; Based on the adversarial attack framework and adversarial defense framework, the decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic.

[0102] According to a safety protection device for an intelligent networked vehicle provided by the present invention, a deep reinforcement learning adversarial attack framework is determined according to a time selection function, random noise and gradient perturbation, specifically including: Based on the strategy distribution or Q-value distribution of the current state of the deep reinforcement learning agent model, the time selection function is calculated; the time selection function is used to determine the time to trigger the attack; Add random noise to the gradient perturbation to obtain adversarial perturbation; Based on the adversarial perturbation, the current state of the agent model is modified.

[0103] According to a safety protection device for an intelligent connected vehicle provided by the present invention, an adversarial defense framework for deep reinforcement learning is determined according to an adversarial sample generation method based on gradient descent and an adversarial loss function, specifically comprising: Generate adversarial samples based on gradient descent method; Training deep reinforcement learning models based on adversarial examples and comprehensive adversarial loss functions; Among them, the comprehensive adversarial loss function includes strategy loss, value function loss, adversarial loss and strategy entropy. The strategy entropy represents the degree of confusion of different actions output by the strategy corresponding to any state.

[0104] Figure 7 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communications interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the safety protection method of the intelligent networked vehicle, and the method includes: Based on the vehicle control behavior characterization model and the abnormal behavior classification learning model, abnormal vehicle control instructions during the communication between the target vehicle and the cloud are detected; and based on the malicious traffic identification model, hidden malicious traffic during the communication between the target vehicle and the cloud is identified; Through the adversarial defense method of deep reinforcement learning, the decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic; the trained decision model is used to provide security protection for intelligent connected vehicles; Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; the malicious traffic identification model is based on the traffic interaction graph and graph neural network training.

[0105] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0106] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the safety protection method of the intelligent connected vehicle provided by the above methods, the method includes: Based on the vehicle control behavior characterization model and the abnormal behavior classification learning model, abnormal vehicle control instructions during the communication between the target vehicle and the cloud are detected; and based on the malicious traffic identification model, hidden malicious traffic during the communication between the target vehicle and the cloud is identified; Through the adversarial defense method of deep reinforcement learning, the decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic; the trained decision model is used to provide security protection for intelligent connected vehicles; Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; the malicious traffic identification model is based on the traffic interaction graph and graph neural network training.

[0107] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for protecting the safety of an intelligent connected vehicle provided by the above methods is implemented, and the method comprises: Based on the vehicle control behavior characterization model and the abnormal behavior classification learning model, abnormal vehicle control instructions during the communication between the target vehicle and the cloud are detected; and based on the malicious traffic identification model, hidden malicious traffic during the communication between the target vehicle and the cloud is identified; Through the adversarial defense method of deep reinforcement learning, the decision control model is trained according to abnormal vehicle control instructions and hidden malicious traffic; the trained decision model is used to provide security protection for intelligent connected vehicles; Among them, the vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle and form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; the malicious traffic identification model is based on the traffic interaction graph and graph neural network training.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A safety protection method for an intelligent connected vehicle, characterized in that: include: Based on the vehicle control behavior characterization model and abnormal behavior classification learning model, detect abnormal vehicle control instructions during the communication between the target vehicle and the cloud; And, based on the malicious traffic identification model, identify hidden malicious traffic in the communication process between the target vehicle and the cloud; Through the adversarial defense method of deep reinforcement learning, a decision control model is trained according to the abnormal vehicle control instructions and the hidden malicious traffic; the trained decision model is used to provide safety protection for intelligent networked vehicles; The vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle to form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; The malicious traffic identification model is obtained based on traffic interaction graph and graph neural network training.

2. The safety protection method for an intelligent connected vehicle according to claim 1, characterized in that: The vehicle control behavior characterization model and the abnormal behavior classification learning model are constructed in the following manner: Through the protocol restoration process and reverse engineering process, the structured information of vehicle control instructions is obtained; Analyzing attack methods on the structured information to identify security vulnerabilities of the structured information; Based on the security vulnerabilities, a vehicle control behavior characterization model and an abnormal behavior classification learning model are constructed.

3. The safety protection method for an intelligent connected vehicle according to claim 1, characterized in that: The malicious traffic identification model is constructed in the following way: Extract the statistical characteristics and behavioral characteristics of the traffic respectively; Based on the statistical features and the behavioral features, construct a traffic interaction graph; Performing graph reduction, flow clustering and flow fitting operations on the flow interaction graph to obtain a flow feature library; Based on the traffic feature library, the graph neural network is trained to obtain a malicious traffic identification model.

4. The safety protection method for an intelligent connected vehicle according to claim 1, characterized in that: Through the adversarial defense method of deep reinforcement learning, according to the abnormal vehicle control instructions and the hidden malicious traffic, the decision control model is trained, specifically including: Determine the adversarial attack framework of deep reinforcement learning based on the time selection function, random noise and gradient perturbation, and determine the adversarial defense framework of deep reinforcement learning based on the adversarial sample generation method based on gradient descent and the adversarial loss function; Based on the adversarial attack framework and the adversarial defense framework, a decision control model is trained according to the abnormal vehicle control instructions and the hidden malicious traffic.

5. The safety protection method for an intelligent networked vehicle according to claim 4, characterized in that: The adversarial attack framework of deep reinforcement learning is determined based on the time selection function, random noise and gradient perturbation, including: Calculate a time selection function based on the strategy distribution or Q-value distribution of the current state of the deep reinforcement learning agent model; the time selection function is used to determine the time to trigger the attack; Add random noise to the gradient perturbation to obtain adversarial perturbation; Based on the adversarial perturbation, a current state of the agent model is modified.

6. The safety protection method for an intelligent connected vehicle according to claim 4, characterized in that: The adversarial defense framework for deep reinforcement learning is determined based on the adversarial sample generation method based on gradient descent and the adversarial loss function, which specifically includes: Generate adversarial samples based on gradient descent method; Based on the adversarial sample and the comprehensive adversarial loss function, training the deep reinforcement learning model; Among them, the comprehensive adversarial loss function includes strategy loss, value function loss, adversarial loss and strategy entropy, and the strategy entropy represents the degree of confusion of different actions output by the strategy corresponding to any state.

7. A safety protection device for an intelligent network-connected vehicle, characterized in that: include: A detection module, which is used to detect abnormal vehicle control instructions during the communication between the target vehicle and the cloud based on the vehicle control behavior characterization model and the abnormal behavior classification learning model; And, based on the malicious traffic identification model, identify hidden malicious traffic in the communication process between the target vehicle and the cloud; A training module, used for training a decision control model according to the abnormal vehicle control instructions and the hidden malicious traffic through a deep reinforcement learning adversarial defense method; the trained decision model is used for safety protection of intelligent networked vehicles; The vehicle control behavior characterization model is used to obtain the abnormal behavior characteristics of the target vehicle to form an abnormal behavior data sequence; the abnormal behavior classification learning model is used to determine the control behavior type of the target vehicle; The malicious traffic identification model is obtained based on traffic interaction graph and graph neural network training.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the safety protection method for the intelligent connected vehicle as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the safety protection method for an intelligent connected vehicle as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the safety protection method for an intelligent connected vehicle as claimed in any one of claims 1 to 6 is implemented.

Images