Automatic driving control system and control method based on two-stage imitation learning

Abstract

The present invention relates to an autonomous driving control system based on two-stage imitation learning, comprising a target publishing module, an integrated navigation module, a perception module, a control parameter generation module, and an underlying control module. The target publishing module is used to provide navigation information of the target; the integrated navigation module is used to provide vehicle status information; the perception module is used to provide image information of the surrounding environment; the control parameter generation module is used to collect information collected by the integrated navigation module, the perception module, and the target publishing module to predict control parameters and generate vehicle control information; and the underlying control module is used to receive vehicle control information and control the vehicle using the control information. The present invention adopts an autonomous driving control system based on two-stage imitation learning to perform control by comprehensively analyzing the current external environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of end-to-end control of unmanned vehicles, and specifically relates to an automatic driving control system and a control method thereof based on two-stage imitation learning. Background Art

[0002] As one of the most influential modern technologies impacting human society, autonomous vehicles have become a hot topic of research both domestically and internationally. Research into autonomous driving technology holds significant implications for national defense, scientific and technological development, and economic and social development. However, achieving true automated driving still faces numerous difficulties and challenges.

[0003] Traditional autonomous driving control systems are implemented through modularization. A modular system structures the processes from sensor input to vehicle chassis control parameter output into independent modules, typically categorized as localization, mapping, perception, prediction, planning, and control. The basic logic of modularization is divide and conquer, breaking a complex task into several simpler subtasks. This approach first transforms sensor input into a predicted vehicle trajectory through the perception and planning modules. The control module then uses the resulting trajectory to derive control parameters and subsequently control the vehicle. This approach offers the advantage of applying constraints from various angles for special situations, such as emergency braking and speeding, ensuring reliable output even when some sensors exhibit deviations. Conversely, traditional control algorithms also significantly increase the probability of error. Errors in a preceding module can propagate throughout the process, resulting in significant deviations from the ideal control parameters, thus compromising vehicle control. For example, in the 2018 Tesla accident, the perception module mistakenly identified a white trailer as the sky. Even if subsequent control modules executed correctly, this error could not be avoided.

[0004] With the rapid development of neural networks, end-to-end approaches have become a hot topic in autonomous driving research. End-to-end refers to the direct generation of control parameters from sensor inputs via a neural network. This system is essentially a control algorithm that uses sensor information to directly derive control parameters and subsequently control the vehicle. Because the control algorithm directly inputs sensor information and does not require pre-processing by other modules, it effectively avoids errors that accumulate due to errors in a single module. There are two main common end-to-end control algorithms: imitation learning and reinforcement learning. Imitation learning, as a promising method for training autonomous driving systems, is gaining increasing interest. Human driving demonstrations are easily available on a large scale. With these demonstrations, imitation learning can be used to train a model that maps sensory inputs to control commands, thereby directly controlling the vehicle.

[0005] In 2016, NVIDIA proposed PilotNet, a CNN-based end-to-end autonomous driving control algorithm. This algorithm was the first to predict vehicle steering angles. The algorithm feeds camera images into a CNN network, compares the CNN's predictions with the actual control values, and adjusts the CNN's weights to bring the predictions closer to the actual values. After extensive training with data, the algorithm can directly generate vehicle control parameters based on the camera input. However, because NVIDIA's algorithm only predicts a single frame at a time, it ignores the relationships between multiple frames, specifically the temporal correlations between them. To address this issue, Eraqi HM et al. proposed a new end-to-end control algorithm, CIL, which integrates a CNN with an LSTM. While the CNN extracts spatial features, the LSTM incorporates temporal information between images. The trained algorithm can then more accurately predict control parameters using a video stream. While CIL considers temporal information, testing has shown that some spatial information is lost during feature extraction. To address this issue, Chi L et al. proposed a spatiotemporal convolutional short-term long-term memory (CIRL) network. The autonomous driving control algorithm combines the spatial and temporal information of the past few frames of images, and integrates the historical status of the vehicle to predict control parameters and achieve autonomous driving.

[0006] A research report indicates that despite the current computing power of powerful computers, it is still difficult for computers to learn to drive solely from camera images. Tests have shown that even with hundreds of thousands of frames of real-world data for training, ideal results are still difficult to achieve. To address this issue, Adrien Gaidon et al. proposed a method called Soft BEV Agent, which uses a bird's-eye view instead of camera images for training. This method first uses a point cloud image captured by a lidar radar to generate a bird's-eye view. This view is then fed into a CNN for training, thereby deriving vehicle control parameters. Experimental testing has shown that, like humans, computers can obtain more information from a bird's-eye view, enabling more accurate prediction of control parameters and thus vehicle control. However, this method requires converting sensor data into a bird's-eye view, which significantly increases the algorithm's runtime and affects the real-time nature of the predictions.

[0007] A search revealed that CN112904852B discloses an autonomous driving control method, device, and electronic device, and CN112904864A discloses an autonomous driving method and system based on deep reinforcement learning. Both the reinforcement learning-based control method and the imitation learning-based control method allow the computer to learn how to drive on its own, but the imitation learning-based control algorithm requires supervised training with a data set, while the reinforcement learning-based control algorithm builds a model in an environment and lets the model run in the environment. In the process of continuous trial and error, the model gradually learns the correct behavior. Therefore, the reinforcement learning-based method can currently only achieve some simple driving tasks. Reinforcement learning is difficult to learn autonomously in complex urban road environments, while imitation learning can learn through manually labeled data in the data set. Summary of the Invention

[0008] The purpose of the present invention is to address the defects of the existing technology and propose an autonomous driving control system and a control method based on two-stage imitation learning, which can realize the control function of autonomous driving.

[0009] To achieve the above objectives, the present invention provides an autonomous driving control system based on two-stage imitation learning, comprising a target publishing module, a combined navigation module, a perception module, a control parameter generation module, and a bottom-level control module; the target publishing module is used to provide navigation information of the target, which includes planning information and location information;

[0010] The combined navigation module is used to provide vehicle status information;

[0011] The perception module is used to provide image information of the surrounding environment, including an RGB image in front of the vehicle and a bird's-eye view of the surrounding environment;

[0012] The control parameter generation module is a vehicle control model obtained from a trained perception control network, which is connected to the integrated navigation module, the perception module and the target release module respectively, and is used to collect and process the information collected by the integrated navigation module, the perception module and the target release module to predict the control parameters and generate vehicle control information;

[0013] The bottom control module is connected to the control parameter generation module, and is used to receive vehicle control information and control the vehicle through the control information.

[0014] The present invention adopts an automatic driving control system based on two-stage imitation learning to perform control by comprehensively analyzing the current external environment.

[0015] The present invention chooses to study the vehicle control method based on imitation learning, and at the same time notices that the effect of predicting control parameters by using a bird's-eye view as the input of imitation learning is better. Therefore, the present invention will combine the imitation learning technology and the algorithm with a bird's-eye view as input to design an autonomous driving vehicle control algorithm based on imitation learning, so as to improve the robustness of the algorithm without affecting the real-time performance.

[0016] The present invention discloses a control method for an autonomous driving control system based on two-stage imitation learning. The control parameter prediction module performs control by comprehensively analyzing the current external environment. The specific steps are as follows:

[0017] Step 1: The target publishing module provides navigation information of the target, and the combined navigation module provides status information of the vehicle; proceed to step 2;

[0018] Step 2: The perception module provides image information of the surrounding environment, including an RGB image in front of the vehicle and a bird's-eye view of the surrounding environment; then proceed to step 3;

[0019] Step 3: The control parameter prediction module collects the target navigation information, vehicle status information, and image information of the surrounding environment, and predicts the control parameters based on the above information to generate specific vehicle control information, and then sends the generated vehicle control information to the bottom control module; then proceed to step 4;

[0020] Step 4: The bottom control module controls the vehicle according to the vehicle control information.

[0021] The present invention further adopts the following technical solution:

[0022] In step 3, the control parameter prediction module is composed of a trained perception control network. It collects the navigation information of the target, the status information of the vehicle, and the image information of the surrounding environment, and inputs them into the vehicle control model obtained by training the perception control network to predict the control parameters and then generate vehicle control information. The control information includes three control parameters: throttle coefficient, brake coefficient and steering wheel angle. The perception control network structure is as follows: Figure 2 As shown in Figure 1, it consists of four parts: input, feature extraction network, control parameter prediction network, and output. The steps for predicting control parameters are as follows:

[0023] Step 3-1: The control parameter prediction module starts the vehicle control model obtained from the trained perception control network, imports the RGB image information in front of the vehicle provided by the perception module, and inputs it to the feature extraction network. The network structure of the feature extraction network is ResNet50; then proceed to step 3-2;

[0024] Step 3-2: Use ResNet50 to extract features from the input RGB image in front of the vehicle to obtain a feature vector of the current RGB image; then proceed to step 3-3;

[0025] Step 3-3, concatenate the vehicle status information provided by the combined navigation module and the target navigation information provided by the target release module with the feature vector obtained in step 3-2 to obtain a fusion information vector; then proceed to step 3-4;

[0026] Step 3-4: Input the fusion information vector obtained in step 3-3 to the control parameter prediction network to predict the control parameters. The network structure of the control parameter prediction network is a 4-layer fully connected layer; then proceed to step 3-5;

[0027] Step 3-5: The control parameter prediction network completes the prediction of the control parameters. The predicted control parameters include the throttle coefficient, the brake coefficient, and the steering wheel angle.

[0028] In step 3, the specific steps of the perception control network training process are as follows:

[0029] Step 3-a: Use the dataset to train the supervision network. Input the bird's-eye view of the surrounding environment and the navigation instructions. The predicted control parameter value is obtained through the feature extraction network and the control parameter prediction network. The predicted control parameter value and the true value corresponding to this frame of data in the dataset are used to calculate the loss through the loss function. Based on the loss value, backpropagation is performed through the Adam optimization function to adjust the weight coefficients of the feature extraction network and the control parameter prediction network so that the predicted control parameter value can be infinitely close to its true value. The training of the supervision network is completed through this process; go to step 3-b;

[0030] Step 3-b: After training the supervisory network, the perception control network is trained next. The data in the dataset is used for the first stage of the perception control network training. The input is the RGB image in front of the vehicle and the corresponding navigation instruction in the dataset. The predicted control parameter value is obtained through the feature extraction network and the control parameter prediction network. The predicted control parameter value is compared with the true value corresponding to this frame of data in the dataset to calculate the loss. Then, backpropagation is performed to update the weight coefficients of the feature extraction network and the control parameter prediction network to reduce the difference between the predicted control parameter value and the true value, thereby completing the first stage of the perception control network training. Go to step 3-c.

[0031] Step 3-c: After the first stage of the perception control network is trained, the second stage of training is carried out. The second stage uses the previously trained supervisory network for supervised training. The RGB image in front of the vehicle and the navigation instructions different from those in the first stage are input to the perception control network trained in the first stage. That is, if the navigation instruction for this frame of data in the first stage is to turn left, then in the second stage of training for this frame of data, the navigation instructions are to turn right, go straight, or stop. The control parameter value under the current input is predicted by the perception control network trained in the first stage. At the same time, the bird's-eye view corresponding to this frame of data and the newly selected navigation instruction (i.e., the navigation instruction input in 3-c) are input into the previously trained supervisory network to obtain the control parameter. The control parameter is used as the true value and compared with the previously predicted control parameter value to calculate the loss. Then, backpropagation is performed to update the weight coefficient of the perception control network. The perception control network trained in the first stage is further trained to complete the second stage of the perception control network training. Go to step 3-d.

[0032] Step 3-d: The perception control network trained in the second phase is used as the vehicle control model for predicting control parameters; proceed to step 3-e;

[0033] Step 3-e: Complete the training of the perception control network.

[0034] The advantages of the present invention are as follows:

[0035] (1) The control parameters predicted by the model are closer to the true values ​​and have higher accuracy;

[0036] (2) The algorithm can adapt to different driving scenarios and is more robust;

[0037] (3) The model can directly predict control parameters end-to-end, which improves real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention will be further described below with reference to the accompanying drawings.

[0039] Figure 1This is a system framework diagram of the present invention.

[0040] Figure 2 Flowchart for predicting control parameters of a vehicle control model obtained from a trained perception control network in the present invention.

[0041] Figure 3 Flowchart of the perceptual control network training process in the present invention; (a) is a flowchart of the supervisory network training process, (b) is a flowchart of the first-stage training process of the perceptual control network, and (c) is a flowchart of the second-stage training process of the perceptual control network. DETAILED DESCRIPTION

[0042] Example 1

[0043] like Figure 1 As shown, an autonomous driving control system based on two-stage imitation learning includes a target release module, an integrated navigation module, a perception module, a control parameter generation module, and a bottom-level control module. The target release module provides navigation information of the target, including planning information and location information; the integrated navigation module provides vehicle status information, including the vehicle's current position, velocity, and acceleration; the perception module provides image information of the surrounding environment, including an RGB image in front of the vehicle and a bird's-eye view of the surrounding environment; the control parameter generation module, which is a vehicle control model derived from a trained perception control network, is connected to the integrated navigation module, the perception module, and the target release module. It collects and processes the information provided by the integrated navigation module, the perception module, and the target release module to predict control parameters and generate vehicle control information; and the bottom-level control module is connected to the control parameter generation module and controls the vehicle based on the generated vehicle control information. The autonomous driving control system based on two-stage imitation learning controls the vehicle by comprehensively analyzing the current external environment.

[0044] like Figure 1 As shown in the figure, a control method for an autonomous driving control system based on two-stage imitation learning is proposed. The control parameter prediction module performs control by comprehensively analyzing the current external environment. The specific steps are as follows:

[0045] Step 1: The target publishing module provides navigation information of the target, and the combined navigation module provides status information of the vehicle; proceed to step 2;

[0046] Step 2: The perception module provides image information of the surrounding environment, including an RGB image in front of the vehicle and a bird's-eye view of the surrounding environment; then proceed to step 3;

[0047] Step 3: The control parameter prediction module collects the target navigation information, vehicle status information, and image information of the surrounding environment, and predicts the control parameters based on the above information to generate specific vehicle control information, and then sends the generated vehicle control information to the bottom control module; then proceed to step 4;

[0048] Step 4: The bottom control module controls the vehicle according to the vehicle control information.

[0049] In step 3, if Figure 2 As shown in the figure, the control parameter prediction module is composed of a trained perception control network. It collects the target navigation information, vehicle status information, and image information of the surrounding environment, and inputs them into the vehicle control model obtained by training the perception control network to predict the control parameters and then generate vehicle control information. The control information includes three control parameters: throttle coefficient, brake coefficient, and steering wheel angle. The perception control network structure is shown in the figure. Figure 2 As shown in Figure 1, it consists of four parts: input, feature extraction network, control parameter prediction network, and output. The steps for predicting control parameters are as follows:

[0050] Step 3-1: The control parameter prediction module starts the vehicle control model obtained from the trained perception control network, imports the RGB image information (3×144×256) in front of the vehicle provided by the perception module, and inputs it to the feature extraction network. The network structure of the feature extraction network is ResNet50. Then proceed to step 3-2.

[0051] Step 3-2: Use ResNet50 to extract features from the input RGB image (3×144×256) of the front of the vehicle through 50 convolution operations to obtain a feature vector (1×256) of the current RGB image; then proceed to step 3-3.

[0052] Step 3-3: Concatenate the vehicle status information provided by the combined navigation module and the target navigation information (1×4) provided by the target release module with the feature vector (1×256) obtained in step 3-2 to obtain a fused information vector (1×260); then proceed to step 3-4.

[0053] Step 3-4: Input the fused information vector (1×260) obtained in step 3-3 into the control parameter prediction network to predict the control parameters (1×3). The network structure of the control parameter prediction network is a 4-layer fully connected layer; proceed to step 3-5;

[0054] Step 3-5: The control parameter prediction network completes the prediction of the control parameters (1×3). The predicted control parameters include the throttle coefficient, the brake coefficient, and the steering wheel angle. The specific parameters of the entire process are shown in the following table:

[0055] .

[0056] like Figure 3 As shown, the training process of the perception control network first uses the collected dataset to train the supervisory network. The trained supervisory network is equivalent to a vehicle control model that uses a bird's-eye view as input. This model can predict relatively accurate control parameters by inputting the bird's-eye view and navigation instructions. The perception control network is then trained using the collected dataset, and finally, the trained supervisory network is used as a supervisor to further train the perception control network. The trained supervisory network is equivalent to the teacher, and the perception control network is equivalent to the student. For each frame, the perception control network asks the supervisory network what the control parameters are for going straight, turning left, turning right, and parking. The supervisory network then obtains the corresponding control parameters by inputting different navigation instructions, further training the perception control network. The supervisory network's role is limited to the training process. The trained perceptual control network serves as the vehicle control model for predicting control parameters in the present invention. This model can accurately obtain control parameters by inputting RGB images and navigation instructions. The bird's-eye view is only used during the training process, ensuring the real-time performance of the model's control parameter prediction. Each frame of the pre-collected data set includes an RGB image of the front of the car, a bird's-eye view, the car's own status, and the car's control parameters (throttle coefficient, brake coefficient, and steering wheel angle).

[0057] The specific training steps are as follows:

[0058] Step 3-a: Use the dataset to train the supervisory network. The structural parameters of the supervisory network are shown in the table:

[0059] Input the bird's-eye view of the surrounding environment and the navigation instructions, and obtain the predicted control parameter value through the feature extraction network and the control parameter prediction network. The predicted control parameter value and the true value corresponding to this frame of data in the dataset are used to calculate the Loss through the loss function. The calculation formula is: , where represents the weight of the neural network, n represents the number of frames of training data, x represents the input sample, and Y represents the true value of the sample label. Backpropagation is performed using the Adam optimization function based on the loss value to adjust the weight coefficients of the feature extraction network and the control parameter prediction network, so that the predicted control parameter values ​​are as close as possible to their true values. This process completes the training of the supervised network. Proceed to step 3-b.

[0060] Step 3-b: After training the supervisory network, the next step is to train the perception and control network. The first phase of the perception and control network training is performed using the data from the dataset. The input is the RGB image of the vehicle in front of the vehicle and the corresponding navigation instructions from the dataset. The feature extraction network and the control parameter prediction network generate predicted control parameter values. These predicted control parameter values ​​are compared with the true values ​​corresponding to that frame of data in the dataset to calculate the loss. Backpropagation is then performed to update the weight coefficients of the feature extraction network and the control parameter prediction network to minimize the difference between the predicted control parameter values ​​and the true values, thus completing the first phase of the perception and control network training. Proceed to step 3-c.

[0061] After the first phase of training of the perception control network is complete in step 3-c, the second phase of training proceeds. This phase uses the previously trained supervisory network for supervised training. The perception control network trained in the first phase is fed an RGB image of the vehicle's front and a navigation command different from the first phase. For example, if the navigation command for a frame in the first phase was to turn left, the navigation command for this frame in the second phase will be to turn right, go straight, or stop. The perception control network trained in the first phase predicts the control parameter values ​​for the current input. The corresponding bird's-eye view image for this frame and the selected navigation command (i.e., the navigation command input in step 3-c) are then fed into the previously trained supervisory network to obtain the control parameters. This control parameter is used as the true value and compared with the previously predicted control parameter value to calculate the loss. Backpropagation is then performed to update the weight coefficients of the perception control network, further training the first-phase trained perception control network, completing the second phase of training for the perception control network. Proceed to step 3-d.

[0062] In step 3-d, the perception control network trained in the second phase is used as the vehicle control model for predicting control parameters. Then proceed to step 3-e.

[0063] Step 3-e: Complete the training of the perception control network.

[0064] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

Claims

1. A control method for an autonomous driving control system based on two-stage imitation learning, characterized in that: The specific steps are as follows: Step 1: The target publishing module provides navigation information of the target, and the combined navigation module provides status information of the vehicle; proceed to step 2; Step 2: The perception module provides image information of the surrounding environment, including an RGB image in front of the vehicle and a bird's-eye view of the surrounding environment; then proceed to step 3; Step 3: The control parameter prediction module collects target navigation information, vehicle status information, and image information of the surrounding environment, and inputs them into the vehicle control model obtained by training the perception control network. It predicts the control parameters and generates vehicle control information, which is then sent to the underlying control module. The specific steps of the perception control network training process are as follows: Step 3-a: Use the dataset to train the supervision network. Input the bird's-eye view of the surrounding environment and the navigation instructions. The predicted control parameter values ​​are obtained through the feature extraction network and the control parameter prediction network. The predicted control parameter values ​​and the true values ​​corresponding to the frame data in the dataset are used to calculate the loss through the loss function. Based on the loss value, backpropagation is performed through the Adam optimization function to adjust the weight coefficients of the feature extraction network and the control parameter prediction network to complete the training of the supervision network; then proceed to step 3-b. Step 3-b: Input is the RGB image in front of the vehicle and the corresponding navigation command in the dataset. The predicted control parameter value is obtained through the feature extraction network and the control parameter prediction network. The predicted control parameter value is compared with the true value corresponding to the data frame in the dataset to calculate the loss. Then, backpropagation is performed to update the weight coefficients of the feature extraction network and the control parameter prediction network, completing the first stage of training of the perception control network. Then proceed to step 3-c. Step 3-c: Input the RGB image of the vehicle in front and the navigation instructions different from those in the first stage into the perception control network trained in the first stage. The perception control network trained in the first stage predicts the control parameter value under the current input. At the same time, the bird's-eye view corresponding to this frame of data and the newly selected navigation instructions are input into the previously trained supervision network to obtain the control parameter. This control parameter is used as the true value and compared with the previously predicted control parameter value to calculate the loss. Then, backpropagation is performed to update the weight coefficient of the perception control network, completing the second stage of training of the perception control network; then proceed to step 3-d. Step 3-d: The perception control network trained in the second phase is used as the vehicle control model for predicting control parameters; proceed to step 3-e; Step 3-e: Complete the training of the perception control network Go to step 4; Step 4: The bottom control module controls the vehicle according to the vehicle control information.

2. The control method of the automatic driving control system based on two-stage imitation learning according to claim 1 is characterized in that: In step 3, the vehicle control model obtained by training the perception control network performs the following steps to predict the control parameters: Step 3-1: The control parameter prediction module starts the vehicle control model obtained from the trained perception control network and imports the RGB image information in front of the vehicle provided by the perception module; then proceeds to step 3-2; Step 3-2: Use the feature extraction network to extract features from the RGB image in front of the vehicle to obtain a feature vector of the current RGB image; then proceed to step 3-3; Step 3-3, concatenate the vehicle status information provided by the combined navigation module and the target navigation information provided by the target release module with the feature vector obtained in step 3-2 to obtain a fusion information vector; then proceed to step 3-4; Step 3-4: Input the fusion information vector obtained in step 3-3 to the control parameter prediction network to predict the control parameters; then proceed to step 3-5; Step 3-5: The control parameter prediction network completes the prediction of the control parameters.

3. A system based on the two-stage imitation learning-based autonomous driving control method according to claim 1, characterized in that: It includes a target publishing module, a combined navigation module, a perception module, a control parameter generation module and a bottom control module; the target publishing module is used to provide navigation information of the target, and the navigation information includes planning information and location information; The combined navigation module is used to provide vehicle status information; The perception module is used to provide image information of the surrounding environment, including an RGB image in front of the vehicle and a bird's-eye view of the surrounding environment; The control parameter generation module is connected to the integrated navigation module, the perception module and the target publishing module respectively, and is used to collect information collected by the integrated navigation module, the perception module and the target publishing module to predict the control parameters and generate vehicle control information; The bottom control module is connected to the control parameter generation module, and is used to receive vehicle control information and control the vehicle through the control information.

Images