An autonomous driving control method and system, and a storage medium

Abstract

This invention relates to an autonomous driving control method and system, and a storage medium, comprising: inputting original samples into a first autonomous driving model to predict the future driving trajectory points of a vehicle, obtaining a first predicted trajectory point sequence; inputting the original samples into an adversarial sample generation model to generate adversarial samples; inputting the adversarial samples into a first defense module for adversarial sample defense; if the defense is successful, generating vehicle control commands based on the first predicted trajectory point sequence; if the defense fails, a second defense module uses the adversarial samples to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model; inputting the original samples into a pre-trained autonomous driving model to predict the future driving trajectory points of a vehicle, obtaining a second predicted trajectory point sequence, and generating vehicle control commands based on the second predicted trajectory point sequence; controlling the vehicle's autonomous driving according to the generated vehicle control commands, thereby improving the active defense capability of the autonomous driving model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving control technology, specifically to an autonomous driving control method and system, and a storage medium. Background Technology

[0002] Traditional autonomous driving employs rule-based modular approaches, typically divided into perception, decision-making and planning, and control. End-to-end solutions, however, are closer to human nature, representing a pure AI approach of imitation learning with immense future potential. End-to-end autonomous driving models possess artificial intelligence characteristics, learning perception while simultaneously implementing control actions through purely data-driven learning. However, pure end-to-end autonomous driving models (image → steering wheel angle control) have limitations, lacking interpretability and security. Therefore, trajectory-based end-to-end autonomous driving is proposed as an extension and expansion, offering better results. In realizing this invention, the inventors discovered that current research on AI adversarial attacks and defenses focuses almost entirely on tasks such as image classification and segmentation, while research on regression tasks, especially end-to-end autonomous driving models, is very rare. Therefore, current trajectory-based end-to-end autonomous driving adversarial attacks and active defenses require further research and improvement. Summary of the Invention

[0003] The purpose of this invention is to propose an autonomous driving control method and system, and a computer-readable storage medium, which are applied to end-to-end autonomous driving based on trajectory learning, in order to solve the technical problem of poor active defense capability of end-to-end autonomous driving based on trajectory learning.

[0004] To achieve the above objectives, a first aspect of the present invention provides an automatic driving control method, comprising the following steps:

[0005] Obtain the original sample; wherein, the original sample is the information of the front environment detected by the vehicle sensing unit during vehicle driving;

[0006] The original sample is input into a pre-trained autonomous driving model to predict vehicle trajectory points and obtain the first predicted trajectory point sequence.

[0007] The original sample is input into a pre-trained adversarial sample generation model to generate adversarial samples;

[0008] The adversarial sample is input into the first defense module for adversarial sample defense. If the defense is successful, vehicle control commands are generated based on the first predicted trajectory point sequence. If the defense fails, the adversarial sample is input into the second defense module. The second defense module uses the adversarial sample to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model. The original sample is input into the new autonomous driving model to predict vehicle trajectory points to obtain a second predicted trajectory point sequence, and vehicle control commands are generated based on the second predicted trajectory point sequence.

[0009] The vehicle is controlled to drive automatically based on the generated vehicle control commands.

[0010] Optionally, the adversarial example generation model includes a feature extraction module, a noise module, a vector concatenation module, a GAN generator, and an attack module;

[0011] The feature extraction module is used to extract the sample feature vector of the original sample;

[0012] The noise module is used to randomly generate noise vectors;

[0013] The vector concatenation module is used to concatenate the sample feature vector with the noise vector to obtain the generator input vector;

[0014] The generator is used to generate initial adversarial samples based on the generator input vector;

[0015] The attack module is used to attack the autonomous driving model using the initial adversarial sample. If the attack is successful, the initial adversarial sample is output as an adversarial sample.

[0016] Optionally, the adversarial example generation model also employs a GAN discriminator during training, with the loss function for training being: L(G, D) = L GAN +αL adv +βL pert ;

[0017] in:

[0018] L GAN =E x [log D(x)+E x log(1-D(G(z|f(x)))],

[0019] L adv =E x [M(x)-M(G(z|f(x))]distance

[0020] L pert =E x ||xG(z|f(x))||2

[0021] In the above function, G is the GAN generator, D is the GAN discriminator, α and β are hyperparameters controlling the loss weights for each target, x is the original input sample, and E... x Let D(x) be the mean of the input original sample as the variable, z be the noise vector, f(x) be the sample feature vector, G(z|f(x)) be the initial adversarial sample output by the generator when z and f(x) are concatenated and input into the generator, M(x) be the output of the autonomous driving model when the original sample x is input into the autonomous driving model, M(G(z|f(x))) be the output of the autonomous driving model when G(z|f(x)) is used to attack the autonomous driving model, and [M(x)-M(G(z|f(x))] distance To find the distance between M(x) and M(G(z|f(x))).

[0022] Optionally, inputting the adversarial sample into the first defense module for adversarial sample defense includes: performing a first-level defense;

[0023] The first level of defense includes:

[0024] After feature compression of the adversarial sample, it is input into the autonomous driving model to predict vehicle trajectory points and obtain a third predicted trajectory point sequence. The adversarial sample is then input into the autonomous driving model to predict vehicle trajectory points and obtain a fourth predicted trajectory point sequence. The third predicted trajectory point sequence and the fourth predicted trajectory point sequence are compared. If the error between the two is greater than a preset error value, the defense is successful.

[0025] Optionally, the first level of defense further includes:

[0026] After the adversarial sample is compressed using various feature compression methods, it is input into the autonomous driving model to predict vehicle trajectory points and obtain multiple predicted trajectory point sequences. If the error between any one of the multiple predicted trajectory point sequences and the fourth predicted trajectory point sequence is greater than a preset error value, the defense is successful; otherwise, the defense fails.

[0027] The various feature compression methods include at least one of median filtering and Gaussian filtering.

[0028] Optionally, the step of inputting the adversarial sample into the first defense module for adversarial sample defense further includes: if the first-level defense fails, then performing a second-level defense;

[0029] The second level of defense includes:

[0030] The adversarial sample is input into the autonomous driving model. At the same time, a pre-trained detection model is used to detect whether the sample input into the autonomous driving model is an adversarial sample. If it is, the defense is successful; otherwise, the defense fails.

[0031] The autonomous driving model includes a feature extraction network for image feature extraction and a feature processing network for trajectory point prediction of the extracted image features. The input of the detection model is connected to the output of the feature extraction network, and the output of the detection model is the detection result of whether the sample input to the autonomous driving model is an adversarial example.

[0032] Optionally, the loss function used by the detection model during training is as follows:

[0033]

[0034] Where y is the output prediction value of the detection model during training. To detect the true output value of the model.

[0035] Optionally, the step of using the adversarial examples to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model includes:

[0036] The autonomous driving model is adversarially trained using the adversarial examples; simultaneously, an autonomous driving reference model is adversarially trained using the adversarial examples to determine whether the model's structural complexity affects its defense capability against the adversarial examples, and the model parameters of the autonomous driving model are adjusted according to the impact of the model's structural complexity on its defense capability against the adversarial examples, ultimately obtaining a new autonomous driving model; wherein, the autonomous driving reference model has the same function as the autonomous driving model but a different structure.

[0037] A second aspect of the present invention provides an autonomous driving control system, comprising:

[0038] The sample acquisition unit acquires the original sample; wherein, the original sample is the information about the front environment detected by the vehicle sensing unit during vehicle driving.

[0039] The trajectory prediction unit is used to input the original sample into a pre-trained autonomous driving model to predict the future driving trajectory points of the vehicle and obtain a first predicted trajectory point sequence.

[0040] The trajectory prediction unit is used to input the original sample into a pre-trained adversarial sample generation model to generate adversarial samples.

[0041] A defense unit is configured to input the adversarial sample into a first defense module for adversarial sample defense. If the defense is successful, vehicle control commands are generated based on the first predicted trajectory point sequence. If the defense fails, the adversarial sample is input into a second defense module. The second defense module uses the adversarial sample to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model. The original sample is then input into the new autonomous driving model to predict vehicle trajectory points and obtain a second predicted trajectory point sequence. Vehicle control commands are generated based on the second predicted trajectory point sequence.

[0042] The control unit is used to control the vehicle's autonomous driving based on the generated vehicle control commands.

[0043] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the autonomous driving control method described in the first aspect.

[0044] The aforementioned autonomous driving control method and system, as well as the computer-readable storage medium, have at least the following beneficial effects:

[0045] The aforementioned autonomous driving model is an end-to-end autonomous driving regression model based on trajectory learning. For this autonomous driving model, embodiments of the present invention generate adversarial examples by inputting the original samples into a pre-trained adversarial example generation model. A cascaded first defense module and a second defense module are set up. Firstly, the first defense module defends against the generated adversarial examples. If the defense is successful, the trajectory points output by the autonomous driving model are converted into control commands to control the vehicle's autonomous driving. If the first defense module fails, the second defense module uses the generated adversarial examples to perform adversarial training on the autonomous driving model, obtaining a new autonomous driving model. Based on the new autonomous driving model, trajectory points are predicted for the original samples, and the predicted trajectory points are converted into control commands to control the vehicle's autonomous driving. The defense method of the first defense module differs from the adversarial training method of the second defense module. That is, embodiments of the present invention employ a cascaded defense using different defense modules for joint adversarial training, effectively overcoming the technical deficiency that a single defense method is unlikely to achieve "herd immunity" against adversarial examples generated by all other attack methods, thereby improving the defense capability against adversarial example attacks on the trajectory learning-based autonomous driving regression model.

[0046] Other features and advantages of the above-described automatic driving control method and system, as well as the computer-readable storage medium, will be set forth in the following description. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of an autonomous driving control method according to an embodiment of the present invention.

[0049] Figure 2 This is a framework diagram of an autonomous driving model according to an embodiment of the present invention.

[0050] Figure 3 This is a framework diagram of the adversarial sample generation model according to an embodiment of the present invention.

[0051] Figure 4 This is a diagram illustrating the training principle of the adversarial sample generation model according to an embodiment of the present invention.

[0052] Figure 5 This is a structural diagram of an autonomous driving control system according to another embodiment of the present invention. Detailed Implementation

[0053] The various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. Furthermore, numerous specific details are set forth in the following detailed embodiments to better illustrate the invention. Those skilled in the art will understand that the invention can be practiced without certain specific details. In some instances, means well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.

[0054] See Figure 1 An embodiment of the present invention proposes an automatic driving control method, the method of which includes the following steps S10 to S50:

[0055] Step S10: Obtain the original sample; wherein the original sample is the information of the front environment detected by the vehicle sensing unit during vehicle driving.

[0056] Specifically, the forward environment information detected by the vehicle sensing unit includes a sequence of forward images captured by the forward-view camera. During vehicle driving, the forward-view camera installed at the front of the vehicle captures forward images of the road environment in real time. Multiple consecutive forward images form the forward image sequence, which is also a video frame image. Of course, it may also include environmental information obtained by other sensors such as millimeter-wave radar and lidar.

[0057] Step S20: Input the original sample into the pre-trained autonomous driving model to predict vehicle trajectory points and obtain the first predicted trajectory point sequence.

[0058] Specifically, the autonomous driving model in this embodiment is an end-to-end autonomous driving regression model based on trajectory learning. For example, the autonomous driving model predicts the vehicle's future driving trajectory over a certain distance, such as 30 meters, based on the input sequence of images ahead. The future driving trajectory can be determined by, for example, 30 trajectory points p. i The trajectory is composed of (x, y), where i represents the index of the trajectory point, and p... i (x, y) represents the i-th trajectory point. A cubic curve fitting of s(t) = a0 + a1t + a2t is performed on the 30 trajectory points in the vehicle coordinate system. 2 +a3t 3 Then, the coordinates are downsampled at equal intervals to 5 coordinate points. After downsampling, the y-values ​​of the coordinate points are fixed as (6, 12, 18, 24, 28). The x-coordinates corresponding to the y-coordinates are found respectively. These 5 (x, y) coordinate points constitute the trajectory of the future 30m distance predicted by the autonomous driving model.

[0059] In this embodiment, the autonomous driving model can be set as a deep learning model of CNN+FC or CNN+LSTM+FC, for example. Figure 2 The diagram shows a CNN+LSTM+FC network, which consists of a CNN (Network for Feature Extraction), an LSTM (Network for Feature Processing), and a fully connected FC layer connected sequentially. The CNN extracts image features from the preceding image sequence. The LSTM performs prediction processing on the extracted image features. The FC layer outputs trajectory points through a fully connected layer. For example, the CNN preferably uses, but is not limited to, the classic ResNet residual network structure, transferring a ResNet50 model pre-trained on ILSVRC2012 image classification. The last feature layer of the CNN has 2048 dimensions. The LSTM consists of two layers of LSTM cells with a neuron size of 256. After full connection, the FC layer uses the tanh activation function to activate and output 5 predicted coordinate point values.

[0060] Step S30: Input the original sample into the pre-trained adversarial sample generation model to generate adversarial samples.

[0061] Specifically, there are many ways to generate adversarial examples. This embodiment is not limited to any one of the generation methods. It should be understood that all of them are within the protection scope of this embodiment of the invention.

[0062] Step S40: Input the adversarial sample into the first defense module for adversarial sample defense. If the defense is successful, generate vehicle control commands based on the first predicted trajectory point sequence. If the defense fails, input the adversarial sample into the second defense module. The second defense module uses the adversarial sample to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model. Input the original sample into the new autonomous driving model to predict vehicle trajectory points to obtain a second predicted trajectory point sequence, and generate vehicle control commands based on the second predicted trajectory point sequence.

[0063] It should be noted that the first defense module adopts a different defense method than the adversarial training method of the second defense module. When the adversarial sample is used to perform adversarial training on the autonomous driving model, if the model parameters of the autonomous driving model change after adversarial training and the adversarial sample can be correctly identified, it means that a new autonomous driving model has been obtained. The new autonomous driving model can effectively defend against the adversarial sample. The new autonomous driving model is then used to predict vehicle trajectory points.

[0064] Step S50: Control the vehicle to drive automatically according to the generated vehicle control commands.

[0065] Specifically, the vehicle control command, such as the steering wheel angle, can be calculated using a pure tracking algorithm to determine the required front axle wheel deflection angle δ. The specific calculation method is as follows:

[0066]

[0067] Where L is the vehicle wheelbase, L fw l is the distance from the vehicle's center of gravity to the center of the rear axle. fw θ is the distance from the vehicle's center of gravity to the aiming point, and θ is the angle between the vehicle's orientation and the line connecting the following point. The aiming point refers to the forward reference point determined by the aiming mechanism of the vehicle's autonomous driving system. The aiming mechanism of the vehicle's autonomous driving system is a commonly used autonomous driving technology, so it will not be elaborated here.

[0068] The aforementioned autonomous driving model is an end-to-end autonomous driving regression model based on trajectory learning. For this autonomous driving model, embodiments of the present invention generate adversarial examples by inputting the original samples into a pre-trained adversarial example generation model. A cascaded first defense module and a second defense module are set up. Firstly, the first defense module defends against the generated adversarial examples. If the defense is successful, the trajectory points output by the autonomous driving model are converted into control commands to control the vehicle's autonomous driving. If the first defense module fails, the second defense module uses the generated adversarial examples to perform adversarial training on the autonomous driving model, obtaining a new autonomous driving model. Based on the new autonomous driving model, trajectory points are predicted for the original samples, and the predicted trajectory points are converted into control commands to control the vehicle's autonomous driving. The defense method of the first defense module differs from the adversarial training method of the second defense module. That is, embodiments of the present invention employ a cascaded defense using different defense modules for joint adversarial training, effectively overcoming the technical deficiency that a single defense method is unlikely to achieve "herd immunity" against adversarial examples generated by all other attack methods, thereby improving the defense capability against adversarial example attacks on the trajectory learning-based autonomous driving regression model.

[0069] In some embodiments, Figure 3 For a framework diagram of the adversarial example generation model, please refer to... Figure 3 The adversarial example generation model includes a feature extraction module 11, a noise module 12, a vector concatenation module 13, a GAN generator 14, and an attack module 15.

[0070] The feature extraction module 11 is used to extract the sample feature vector of the original sample; specifically, the feature extraction module 11 can be set to be the same as the feature extraction part of the autonomous driving model.

[0071] The noise module 12 is used to randomly generate noise vectors.

[0072] The vector concatenation module 13 is used to concatenate the sample feature vector with the noise vector to obtain the generator input vector.

[0073] The generator 14 is used to generate initial adversarial samples based on the generator input vector. Specifically, GAN stands for Generative Adversarial Network, which mainly consists of two neural networks: a generator G and a discriminator D. In this embodiment, the core idea of ​​using GAN to generate adversarial samples is to use the sample feature vector output by the feature extractor of the original sample as the prior information for generating adversarial samples. Then, the sample feature vector and the randomly generated noise vector are concatenated and input into the generator G to generate adversarial samples. It should be noted that in actual applications, only the generator G is used to generate adversarial samples. The discriminator D is used during the training of the model and is responsible for determining whether the input sample is an adversarial sample.

[0074] The attack module 15 is used to attack the autonomous driving model using the initial adversarial sample. If the attack is successful, the initial adversarial sample is output as an adversarial sample. Specifically, a successful attack means that when the initial adversarial sample is input into the autonomous driving model, the autonomous driving model outputs the corresponding trajectory point, that is, the recognition is incorrect and the autonomous driving model cannot correctly recognize the sample as an adversarial sample.

[0075] In some embodiments, Figure 4 For a diagram illustrating the training principle of the adversarial example generation model, please refer to [link / reference]. Figure 4 The adversarial example generation model training process also utilizes the GAN discriminator D, which analyzes the generated adversarial examples x. adv To distinguish it from the real original sample x, and at the same time deceive the autonomous driving model M with generated adversarial examples, since the autonomous driving model M is a regression model without targeted attacks, M(x) adv The prediction is for the lateral coordinates X of 5 trajectory points. i In order to successfully deceive the autonomous driving model M, this embodiment maximizes M(x) adv The distance (i.e., deviation) between the predicted result and the actual result is considered. Furthermore, to limit the perturbation magnitude of generated adversarial examples, this embodiment minimizes the adversarial example x. adv The L2 loss between the original sample x and the target sample x is used. Therefore, the overall objective loss function of the adversarial sample generation model in this embodiment is:

[0076] L(G,D)=L GAN +αL adv +βL pert ;

[0077] in:

[0078] L GAN =E x [log D(x)+E xlog(1-D(G(z|f(x)))]

[0079] L adv =E x [M(x)-M(G(z|f(x))] distance

[0080] L pert =E x ||xG(z|f(x))||2

[0081] In the above function, G is the GAN generator, D is the GAN discriminator, α and β are hyperparameters controlling the loss weights for each target, x is the original input sample, and E... x Let D(x) be the mean of the input original sample as the variable, z be the noise vector, f(x) be the sample feature vector, G(z|f(x)) be the initial adversarial sample output by the generator when z and f(x) are concatenated and input into the generator, M(x) be the output of the autonomous driving model when the original sample x is input into the autonomous driving model, M(G(z|f(x))) be the output of the autonomous driving model when G(z|f(x)) is used to attack the autonomous driving model, and [M(x)-M(G(z|f(x))] distance To find the distance between M(x) and M(G(z|f(x)), which is used to measure the difference between the two elements.

[0082] Based on the above loss function, the optimal parameters of the generator and discriminator can be obtained by pre-training the adversarial example generation model using sample data.

[0083] In some embodiments, inputting the adversarial sample into the first defense module for adversarial sample defense includes: performing a first-level defense;

[0084] The first level of defense includes:

[0085] After feature compression of the adversarial sample, it is input into the autonomous driving model to predict vehicle trajectory points and obtain a third predicted trajectory point sequence. The adversarial sample is then input into the autonomous driving model to predict vehicle trajectory points and obtain a fourth predicted trajectory point sequence. The third predicted trajectory point sequence and the fourth predicted trajectory point sequence are compared. If the error between the two is greater than a preset error value, the defense is successful.

[0086] In some embodiments, the first level of defense further includes:

[0087] After the adversarial sample is compressed using various feature compression methods, it is input into the autonomous driving model to predict vehicle trajectory points and obtain multiple predicted trajectory point sequences. If the error between any one of the multiple predicted trajectory point sequences and the fourth predicted trajectory point sequence is greater than a preset error value, the defense is successful; otherwise, the defense fails.

[0088] The various feature compression methods include at least one of median filtering and Gaussian filtering.

[0089] Specifically, the median filtering compresses sample features by smoothing adjacent pixels to make them more similar. The filter is moved across the original image, and the center pixel value is modified to the median of the pixel values ​​in the filter. The Gaussian filtering compresses sample features by smoothing over a larger area (not just adjacent pixels) to make pixels more similar. Of course, other feature compression methods can also be used, which are not described in detail in this embodiment, but should be understood to be within the protection scope of this invention.

[0090] It should be noted that the first defense module in this embodiment may only have a first-level defense, or it may have a second-level defense or more levels of defense cascaded with the first-level defense. Only when the first-level or multiple-level defenses in the first defense module fail will it switch to the adversarial training of the second defense module. Of course, the multiple defenses adopt different defense methods.

[0091] In some embodiments, the step of inputting the adversarial sample into the first defense module for adversarial sample defense further includes: if the first-level defense fails, then performing a second-level defense;

[0092] The second level of defense includes:

[0093] The adversarial sample is input into the autonomous driving model. At the same time, a pre-trained detection model is used to detect whether the sample input into the autonomous driving model is an adversarial sample. If it is, the defense is successful; otherwise, the defense fails.

[0094] The autonomous driving model includes a feature extraction network for image feature extraction and a feature processing network for trajectory point prediction of the extracted image features. The input of the detection model is connected to the output of the feature extraction network, and the output of the detection model is the detection result of whether the sample input to the autonomous driving model is an adversarial example.

[0095] Specifically, the detection model is a machine learning model. During the training of the detection model, the training labels for the original samples and adversarial samples are set to two categories, 0 and 1. This can be understood as the detection model and the autonomous driving model sharing the feature extraction network. The output of the feature extraction network is fed into the feature processing network and the detection model, respectively. The two are essentially two branch networks. In order to keep the trajectory prediction accuracy of the autonomous driving model unchanged, this embodiment of the invention fixes all the parameters of the autonomous driving model during the training of the detection model. That is, the parameters of the autonomous driving model are not updated during the training process, and only the network parameters of the detection model are trained.

[0096] Specifically, the loss function used during the training of the detection model is as follows:

[0097]

[0098] Where y is the output prediction value of the detection model during training. To detect the true output value of the model.

[0099] In some embodiments, the step of using the adversarial examples to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model includes:

[0100] The autonomous driving model is adversarially trained using the adversarial examples; simultaneously, an autonomous driving reference model is adversarially trained using the adversarial examples to determine whether the model's structural complexity affects its defense capability against the adversarial examples, and the model parameters of the autonomous driving model are adjusted according to the impact of the model's structural complexity on its defense capability against the adversarial examples, ultimately obtaining a new autonomous driving model; wherein, the autonomous driving reference model has the same function as the autonomous driving model but a different structure.

[0101] It should be noted that, unlike existing adversarial training methods, this embodiment trains an autonomous driving reference model simultaneously with the generated adversarial examples used to train the autonomous driving model. This reference model functions identically to the autonomous driving model, both being autonomous driving trajectory learning models; the input is information about the preceding environment, and the output is predicted vehicle trajectory points. The difference lies in their model structures. The purpose of simultaneously training the reference model is to determine whether the model's structural complexity affects its ability to defend against the adversarial examples. Therefore, this embodiment's adversarial training method considers not only the adversarial examples used for training but also the model's structural complexity, continuously adjusting and iterating the model's parameters to ultimately obtain a new autonomous driving model. Training ends when the trained autonomous driving model can defend against the adversarial example attacks.

[0102] For example, for an autonomous driving model with a CNN+LSTM structure, the CNNs of the autonomous driving model and the autonomous driving reference model can be configured to use pilotNet and ResNet50 network structures respectively, in order to verify whether the complexity of different network structures affects the self-defense capability of the autonomous driving trajectory learning model against adversarial example attacks.

[0103] See Figure 5 Another embodiment of the present invention provides an autonomous driving control system, which can be used to implement the autonomous driving control method described in the above embodiments. The system of this embodiment includes:

[0104] The sample acquisition unit 31 acquires the original sample; wherein, the original sample is the forward environment information detected by the vehicle sensing unit during vehicle driving.

[0105] The trajectory prediction unit 32 is used to input the original sample into a pre-trained autonomous driving model to predict the future driving trajectory points of the vehicle and obtain a first predicted trajectory point sequence.

[0106] The trajectory prediction unit 33 is used to input the original sample into a pre-trained adversarial sample generation model to generate adversarial samples.

[0107] Defense unit 34 is used to input the adversarial sample into a first defense module for adversarial sample defense. If the defense is successful, vehicle control commands are generated based on the first predicted trajectory point sequence. If the defense fails, the adversarial sample is input into a second defense module. The second defense module uses the adversarial sample to perform adversarial training on the autonomous driving model to obtain a new autonomous driving model; the original sample is input into the new autonomous driving model to predict vehicle trajectory points to obtain a second predicted trajectory point sequence, and vehicle control commands are generated based on the second predicted trajectory point sequence; and

[0108] Control unit 35 is used to control the vehicle's automatic driving according to the generated vehicle control commands.

[0109] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] It should be noted that the system described in the above embodiments corresponds to the method described in the above embodiments. Therefore, the parts of the system described in the above embodiments that are not described in detail can be obtained by referring to the content of the method described in the above embodiments. That is, the specific steps recorded in the method described in the above embodiments can be understood as the functions that the system of this embodiment can achieve, and will not be repeated here.

[0111] Furthermore, if the autonomous driving control system described in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0112] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the autonomous driving control method described in the above embodiments.

[0113] Specifically, the computer-readable storage medium may include any entity or recording medium capable of carrying the computer program instructions, such as a USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media.

[0114] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An automatic driving control method characterized by comprising: The method comprises the following steps: obtaining an original sample; wherein the original sample is the front environment information detected by the sensing unit of the vehicle during driving; inputting the original sample into a pre-trained automatic driving model to predict a vehicle trajectory point and obtain a first predicted trajectory point sequence; inputting the original sample into a pre-trained adversarial sample generation model to generate an adversarial sample; inputting the adversarial sample into a first defense module for adversarial sample defense; if the defense is successful, generating a vehicle control instruction according to the first predicted trajectory point sequence; if the defense fails, inputting the adversarial sample into a second defense module, using the adversarial sample to perform adversarial training on the automatic driving model to obtain a new automatic driving model, inputting the original sample into the new automatic driving model to predict a vehicle trajectory point and obtain a second predicted trajectory point sequence, and generating a vehicle control instruction according to the second predicted trajectory point sequence; controlling the vehicle to automatically drive according to the generated vehicle control instruction; wherein the step of inputting the adversarial sample into the first defense module for adversarial sample defense comprises a first level defense; the first level defense comprises: after feature compression of the adversarial sample, inputting the adversarial sample into the automatic driving model to predict a vehicle trajectory point and obtain a third predicted trajectory point sequence, inputting the adversarial sample into the automatic driving model to predict a vehicle trajectory point and obtain a fourth predicted trajectory point sequence, comparing the third predicted trajectory point sequence and the fourth predicted trajectory point sequence, and if the error between the two is greater than a preset error value, the defense is successful.

2. The automatic driving control method according to claim 1, characterized by, the adversarial sample generation model comprises a feature extraction module, a noise module, a vector connection module, a GAN generator and an attack module; the feature extraction module is used to extract a sample feature vector of the original sample; the noise module is used to randomly generate a noise vector; the vector connection module is used to connect the sample feature vector and the noise vector to obtain a generator input vector; the generator is used to generate an initial adversarial sample according to the generator input vector; the attack module is used to attack the automatic driving model with the initial adversarial sample, and if the attack is successful, the initial adversarial sample is output as an adversarial sample.

3. The automatic driving control method according to claim 2, characterized by, The adversarial sample generation model training process also applies a GAN discriminator, and the loss function of the training process is: ; wherein: , In the above functions, G is a GAN generator, D is a GAN discriminator, and is a hyperparameter for controlling the weight of each target loss, x is an input original sample, is a mean solution result with the input original sample as a variable, is an output of the discriminator, is a noise vector, is a sample feature vector, is z and is an initial adversarial sample of the generator output after the input of the generator, is the original sample x is an output of the autonomous driving model when the input is input into the autonomous driving model, is an output of the autonomous driving model when the input is attacked, is an output of the autonomous driving model when the input is attacked, is a distance between and .

4. The automatic driving control method according to claim 1, characterized by, the first level defense further comprises: using multiple feature compression methods to respectively compress the features of the adversarial sample, inputting the adversarial sample into the automatic driving model to predict a vehicle trajectory point, and correspondingly obtaining multiple predicted trajectory point sequences; if the error between any one of the multiple predicted trajectory point sequences and the fourth predicted trajectory point sequence is greater than a preset error value, the defense is successful, otherwise, the defense fails; wherein the multiple feature compression methods include at least one of median filtering and Gaussian filtering.

5. The automatic driving control method of claim 1, wherein the step of inputting the adversarial sample into the first defense module for adversarial sample defense further comprises: if the first level defense fails, performing a second level defense; the second level defense comprises: The adversarial sample is input into the automatic driving model, and a pre-trained detection model is used to detect whether the sample input into the automatic driving model is an adversarial sample; if yes, the defense is successful, and if no, the defense fails. The automatic driving model comprises a feature extraction network for image feature extraction and a feature processing network for trajectory point prediction of the extracted image features, and an input of the detection model is connected with an output of the feature extraction network, and an output of the detection model is a detection result of whether the sample input into the automatic driving model is an adversarial sample.

6. The automatic driving control method according to claim 5, characterized by, A loss function used by the detection model in the training process is as follows: wherein, is an output prediction value of the detection model during the training process, is an output true value of the detection model.

7. The automatic driving control method according to claim 5, characterized by, The second defense module uses the adversarial sample to perform adversarial training on the automatic driving model to obtain a new automatic driving model, comprising: The adversarial sample is used to perform adversarial training on the automatic driving model, and the adversarial sample is used to perform adversarial training on an automatic driving reference model to determine whether the model structure complexity affects the defense capability of the automatic driving model on the adversarial sample, and the model parameters of the automatic driving model are adjusted according to the influence of the model structure complexity on the defense capability of the adversarial sample, and finally a new automatic driving model is obtained; wherein the automatic driving reference model has the same function as the automatic driving model and different structures.

8. An automatic driving control system characterized by comprising: Comprising: A sample acquisition unit acquires an original sample; wherein the original sample is the front environment information detected by a vehicle sensing unit during vehicle driving; A trajectory prediction unit is configured to input the original sample into a pre-trained automatic driving model to predict a future driving trajectory point of the vehicle and obtain a first predicted trajectory point sequence; An adversarial sample generation unit is configured to input the original sample into a pre-trained adversarial sample generation model to generate an adversarial sample; A defense unit is configured to input the adversarial sample into a first defense module for adversarial sample defense, and if the defense is successful, a vehicle control instruction is generated according to the first predicted trajectory point sequence, and if the defense fails, the adversarial sample is input into a second defense module, the second defense module uses the adversarial sample to perform adversarial training on the automatic driving model to obtain a new automatic driving model, the original sample is input into the new automatic driving model to predict a vehicle trajectory point to obtain a second predicted trajectory point sequence, and a vehicle control instruction is generated according to the second predicted trajectory point sequence; and A control unit is configured to control the vehicle automatic driving according to the generated vehicle control instruction. The defense unit inputs the adversarial sample into the first defense module for adversarial sample defense, comprising first-level defense. The first-level defense comprises: After the adversarial sample is compressed, the compressed adversarial sample is input into the automatic driving model to predict a vehicle trajectory point to obtain a third predicted trajectory point sequence, and the adversarial sample is input into the automatic driving model to predict a vehicle trajectory point to obtain a fourth predicted trajectory point sequence, and the third predicted trajectory point sequence and the fourth predicted trajectory point sequence are compared, and if the error between the two is greater than a preset error value, the defense is successful. 9.A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the automatic driving control method according to any one of claims 1 to 7.

Images