Vehicle end-to-end control method and system, storage medium and computing equipment

Abstract

The invention relates to the field of unmanned vehicle control, and discloses a vehicle end-to-end control method and system, a storage medium and computing equipment, and the method comprises the steps: collecting a single-path RGB image and a measurement vector of a running vehicle in real time through a road simulation map constructed by a CARLA simulator, and constructing a data set after preprocessing; inputting the single-path RGB image features into a GCViT image feature extraction model based on a cross attention mechanism, realizing space interaction and feature sharing through local and global self-attention calculation, and outputting image features; and constructing a control action prediction model based on reinforcement learning, training an RL expert by using a Roach model, and outputting vehicle control parameters through an IL agent by taking the image features and the measurement vectors as input. According to the invention, safe automatic driving in a simple driving environment can be realized, and robustness and real-time performance in scenes such as a temporary cone barrel road and the like are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous vehicle control technology, and in particular to an end-to-end vehicle control method, system, storage medium, and computing device based on cross-attention mechanism and reinforcement learning. Background Technology

[0002] Autonomous vehicles differ from ordinary road vehicles, characterized by high-speed driving, dynamic changes in track environment, high requirements for real-time and precision control, and the need to cope with complex scenarios such as temporary cone roads. They also face challenges such as low-adhesion road surfaces and winding tracks, which place extremely high demands on the robustness and environmental adaptability of vehicle control algorithms.

[0003] Existing literature shows that traditional modular autonomous driving systems design perception, planning, and control modules independently. While this offers some interpretability, it suffers from problems such as inter-module error accumulation, low computational efficiency, and poor coordination in complex scenarios. End-to-end models based on imitation learning can quickly learn from expert data, but their generalization ability in unfamiliar scenarios is limited by data diversity. Reinforcement learning-based models can optimize strategies through environmental interaction, but their sample utilization efficiency is low, and their interpretability is poor when directly outputting high-dimensional control parameters. Some studies have attempted to improve model interpretability through auxiliary outputs (such as BEV semantic maps and surrounding vehicle trajectories), but safety still needs improvement, and most models are only suitable for urban scenarios, lacking real-time performance and failing to meet the demands of high-speed dynamic scenarios in Formula One racing.

[0004] Since vehicles need to achieve safe and efficient autonomous driving in complex track environments, it is crucial to design an end-to-end control model that combines high real-time performance, strong generalization ability, and interpretability. Currently, most end-to-end models lack effective fusion of global and local information when processing image features, and the prediction of control actions does not fully incorporate the long-term decision-making advantages of reinforcement learning, resulting in insufficient control accuracy and stability in scenarios such as temporary road tracking and dynamic path planning. Summary of the Invention

[0005] To address the aforementioned issues, the present invention aims to provide a vehicle end-to-end control method, system, storage medium, and computing device to achieve safe autonomous driving in simple driving environments and improve the robustness and real-time performance of the model in scenarios such as temporary cone roads.

[0006] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: a vehicle end-to-end control method, comprising: constructing a road simulation map using a CARLA simulator; acquiring single-channel RGB images and measurement vectors of the operating vehicle in real time; constructing a dataset after preprocessing; inputting the single-channel RGB image features into a GCViT image feature extraction model based on a cross-attention mechanism; achieving spatial interaction and feature sharing through local and global self-attention calculations; and outputting image features; constructing a control action prediction model based on reinforcement learning; training an RL expert using a Roach model; taking image features and measurement vectors as input; and outputting vehicle control parameters through an IL agent.

[0007] Furthermore, the single-channel RGB image features are input into the GCViT image feature extraction model based on the cross-attention mechanism. Spatial interaction and feature sharing are achieved through local and global self-attention calculations, and the output image features include: Sub-images are generated by a complementary partitioning method of ordinary window and sliding window. Attention is calculated in sub-images through a local multi-head self-attention layer. A global query token is generated by a global lexical extraction layer and participates in the global multi-head self-attention calculation of each sub-image, thereby achieving the fusion of global information and local features. The feature map is reduced in dimensionality by a downsampling layer, resulting in image features with halved width and height and doubled number of channels.

[0008] Furthermore, the GCViT image feature extraction model based on the cross-attention mechanism includes: Input layer: The input is a single-channel RGB image, which is then normalized and segmented. Local multi-head self-attention layer: The image is divided into multiple local windows using a fixed window partitioning method. The image blocks within each local window constitute a local feature subset. Self-attention is computed in parallel for each local feature subset through a multi-head mechanism. Global multi-head self-attention layer: It fuses global information across local windows, captures global structural features, generates learnable global query tokens through a global lexical extraction layer, performs cross-window attention calculation on the key-value pairs of the global query tokens and all local windows, and concatenates and fuses the global attention output with local features. Feedforward neural network layer: Adds non-linearity and restores feature dimensions through two fully connected layers and activation functions; Downsampling layer: Reduces the spatial resolution of the feature map, expands the receptive field, and increases the number of channels to capture richer semantic information.

[0009] Furthermore, a control action prediction model based on reinforcement learning is constructed. An RL expert is trained using a Roach model, taking image features and measurement vectors as input, and outputting vehicle control parameters through an IL agent, including: A simulation environment was built in the CARLA simulator. Data was collected on the virtual driving platform to train the Roach model policy network and value network. The network parameters were updated using the proximal policy optimization algorithm, and maximum entropy loss was introduced to encourage exploration. The IL agent is trained using expert data generated by the Roach model. The IL agent processes the input data through image encoding and measurement encoding, generates latent parameters through a fully connected layer, and outputs throttle, brake, and steering control parameters. The model is optimized through L1 loss and speed prediction regularization.

[0010] Furthermore, reinforcement learning-based control action prediction models include: Feature fusion layer: fuses the image features extracted by GCViT with the vehicle state vector; Policy network layer: Generates continuous control actions; Value network layer: Evaluates the value of state-action pairs to assist in policy optimization; The feature fusion layer achieves intermodal information exchange through gated fusion units; The policy network layer uses a three-layer fully connected network, with each layer containing multiple neurons and ReLU activation function; control actions are generated through independent output heads. The value network layer shares the parameters of the first two layers with the policy network, reducing computational overhead; the state features and action vectors are concatenated and then input into the value head.

[0011] Furthermore, after outputting vehicle control parameters through the IL agent, the process also includes a real-vehicle verification step: constructing a temporary cone road simulation map based on the CARLA simulator, collecting data to train the model, and then optimizing the model using real-vehicle data before conducting real-vehicle verification.

[0012] Furthermore, the specific process of real-vehicle verification includes: Cameras, Hall sensors, and CAN bus devices are mounted on Formula One cars to collect cone road data under different weather and lighting conditions; Emergency braking safety protection is achieved through remote control. The effectiveness of the model in real-world scenarios is verified using route completion rate, violation rate, and control accuracy as indicators.

[0013] Secondly, the technical solution adopted by the present invention is as follows: a vehicle end-to-end control system, comprising: a data acquisition module, which acquires single-channel RGB images and measurement vectors of the operating vehicle in real time through a road simulation map constructed by the CARLA simulator, and constructs a dataset after preprocessing; an image feature extraction module, which inputs the single-channel RGB image features into a GCViT image feature extraction model based on a cross-attention mechanism, realizes spatial interaction and feature sharing through local and global self-attention calculation, and outputs image features; and a control action prediction module, which constructs a control action prediction model based on reinforcement learning, trains an RL expert using a Roach model, takes image features and measurement vectors as input, and outputs vehicle control parameters through an IL agent.

[0014] Thirdly, the technical solution adopted by the present invention is: a computer-readable storage medium for storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0015] Fourthly, the technical solution adopted by the present invention is as follows: a computing device, comprising: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0016] The present invention has the following advantages due to the adoption of the above technical solutions: The cross-attention mechanism employed in this invention effectively integrates the global structure and local details of an image. Compared to traditional convolutional networks, feature extraction is more targeted and can accurately identify temporary road boundaries. The end-to-end framework combining reinforcement learning and imitation learning improves model convergence efficiency through expert data supervision, while leveraging the long-term decision-making advantages of RL to enhance adaptability to complex scenarios. CARLA simulations and real-vehicle tests have verified that the model outperforms traditional modular algorithms in terms of route completion rate, violation rate, and control latency, meeting the real-time and accuracy requirements of Formula One racing.

[0017] This invention provides a new technical approach for end-to-end control of autonomous vehicles and can be extended to other types of autonomous driving scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart of the vehicle end-to-end control method in an embodiment of the present invention; Figure 2 This is a flowchart of image feature extraction in an embodiment of the present invention; Figure 3 This is a diagram of the control action prediction model architecture based on reinforcement learning in an embodiment of the present invention; Figure 4 This is a flowchart of the model training process in an embodiment of the present invention; Figure 5 This is a diagram showing the sensor deployment of a Formula One race car in an embodiment of the present invention. Detailed Implementation

[0019] To address the issues of insufficient control precision, poor interpretability, and weak adaptability to complex scenarios in high-speed dynamic racing environments, this invention provides an end-to-end vehicle control method, system, storage medium, and computing device. It includes: extracting single-channel RGB image features using a GCViT network based on a cross-attention mechanism, and combining local and global self-attention to achieve spatial interaction and feature sharing; constructing a reinforcement learning-based control action prediction model, training an expert-supervised IL agent using a Roach model, and using images and measurement vectors as input and output control parameters. The model is trained by collecting data from a temporary cone-road simulation map constructed using a CARLA simulator, and then validated on a real vehicle. This invention improves model interpretability and computational efficiency, enabling safe autonomous driving in simple driving environments. This invention can be applied to the field of end-to-end control technology for autonomous vehicles, providing a new solution for optimizing autonomous driving systems.

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] In one embodiment of the present invention, an end-to-end vehicle control method is provided. This invention focuses on the application of an end-to-end control model in Formula One racing scenarios, with terms such as "image features" and "control parameters" defined based on the racing driving environment. The invention enhances the global-local interaction of image features through a cross-attention mechanism and improves decision-making capabilities in complex scenarios by combining reinforcement learning; it provides an efficient solution for autonomous driving in Formula One racing and can be extended to other unmanned vehicle control scenarios.

[0023] In this embodiment, as Figure 1 As shown, the method includes the following steps: 1) A road simulation map is built using the CARLA simulator. Real-time acquisition of single-lane RGB images and measurement vectors of running vehicles is performed. After preprocessing, a dataset is constructed and used as the training set. The measurement vectors include data such as vehicle speed and turning angle.

[0024] Specifically, while acquiring single-channel RGB images and measurement vectors, sensor data is parsed via the CAN bus using test season data, and a dataset is formed after outlier processing and normalization.

[0025] 2) Input the single-channel RGB image features into the GCViT image feature extraction model based on the cross-attention mechanism, realize spatial interaction and feature sharing through local and global self-attention calculation, and output image features.

[0026] 3) Construct a control action prediction model based on reinforcement learning, use the Roach model to train RL experts (reinforcement learning experts), take image features and measurement vectors as input, and output vehicle control parameters such as throttle, brake, and steering through IL agents (imitation learning agents) to realize end-to-end mapping from images to control parameters such as throttle, brake, and steering.

[0027] In step 1) above, because the end-to-end control model training requires massive amounts of data from multiple scenarios of Formula One racing cars, it is necessary to first collect real vehicle data. Sensor data transmitted in message form via the CAN bus is parsed and preprocessed according to the CAN communication protocol to obtain the real physical signals of parameters such as vehicle speed and steering angle, which are then stored wirelessly on a remote server as a dataset.

[0028] Specifically, the process involves collecting and preprocessing real-time vehicle operating data, including the following steps: 1.1) Use the camera catalog to record single-channel RGB images, perform preprocessing such as cropping and image enhancement on the image data, delete image frames that are invalid for training according to the timestamp, and stitch together the processed image data segments; In this embodiment, since the image resolution is much higher than the input resolution of the designed image feature extraction network, the recorded image needs to be preprocessed.

[0029] 1.2) Record measurement vector data such as vehicle speed, throttle, and vehicle steering angle; In this embodiment, vehicle speed, throttle, and steering angle data are acquired via a bus or wireless transmission device. The collected data is then processed through deep learning to obtain corresponding parameters, which are then transmitted to the Formula ECU via a host computer to enable Formula ECU autonomous driving.

[0030] 1.3) The dataset uses the speed of the drive motor and RGB images as input for deep learning training of the model.

[0031] In step 1.1) above, to meet the design requirements of the image feature extraction network, the image data needs to be scaled, similar to the ViT design, and the image is divided into equal-sized blocks (e.g., 16×16 pixels). GCViT processes the blocks through a local window attention mechanism, reducing global computational complexity. Pixel values ​​are standardized by channel to accelerate model convergence. Multi-scale features are pre-extracted using convolutional layers (such as CNN Backbone) and then input into GCViT for global relation modeling, taking into account both local details and global context.

[0032] In step 1.3) above, the collected data is preprocessed and a dataset is created, including the following steps: 1.3.1) Extract image data and its corresponding timestamps from the dataset, delete invalid images such as vehicles that are stationary based on the timestamps, and perform data augmentation and other processing on the image data; 1.3.2) Based on time soft synchronization, complete data alignment and parsing; 1.3.3) The image and the rotational speed of the drive motor are used as data, and the torque of the drive motor and the rotation angle of the steering motor are used to label the data to create a dataset.

[0033] In step 2) above, the single-channel RGB image features are input into the GCViT image feature extraction model based on the cross-attention mechanism. Spatial interaction and feature sharing are achieved through local and global self-attention calculations, and the image features are output. This includes the following steps: 2.1) Sub-images are generated by using a complementary partitioning method of ordinary window and sliding window. Attention is calculated in sub-images through a local multi-head self-attention layer. A global query token is generated by a global lexical extraction layer to participate in the global multi-head self-attention calculation of each sub-image, thereby realizing the fusion of global information and local features. 2.2) The feature map is reduced in dimensionality by a downsampling layer, and the output image features are halved in width and height and doubled in number of channels.

[0034] In this embodiment, as Figure 2 As shown, the GCViT image feature extraction model based on the cross-attention mechanism includes: (1) Input layer: The input is a single-channel RGB image, which is then normalized and segmented.

[0035] First, standardization is performed: (1) in, I Represents the pixel values ​​of the original image; This represents the image pixel values ​​after standardization. and ... N Five non-overlapping image patches, each patch being [size missing]. Each image patch is transformed into a linear mapping layer with dimension 1. D Embedded vector At the same time, add learnable positional embeddings. To preserve spatial information, the final input vector for: (2) (2) Local multi-head self-attention layer: The image is divided into multiple local windows using fixed window partitioning (e.g., (Number of images within each local window constitute a subset of local features, and self-attention is computed in parallel for each subset of local features using a multi-head mechanism).

[0036] Among them, local feature subset for: (3) Indicates by OK, The space formed by matrices of real elements of columns. For each local feature subset, self-attention is computed in parallel using a multi-head mechanism. The computation process for each head is as follows: (4) in, Indicates the i-th head; These are the query, key, and value vectors obtained after linear projection of the input features; express The dimension of a vector; Indicates the first The corresponding size Learnable linear projective weight matrix; Indicates the first The corresponding size Learnable linear projective weight matrix; Indicates the first The corresponding size The learnable linear projection weight matrix.

[0037] Scaled Dot-Product Attention is used to calculate the correlation of image patches within a local window, highlighting the feature responses of the barrel region. To alleviate the vanishing gradient problem in deep networks, residual connections and LayerNorm are introduced. (5) in, Indicates local features.

[0038] (3) Global multi-head self-attention layer: Global information is fused across local windows to capture global structural features. A learnable global query token is generated through the global lexical extraction layer. Cross-window attention is calculated between the global query token and the key-value pairs of all local windows. The global attention output is then spliced ​​and fused with local features.

[0039] Specifically, it is generated through the Global Token Generator layer. G A learnable global query token This layer consists of two fully connected layers, with the input being the mean feature value of all local windows. The output is the global query token.

[0040] Link the global query token to the key-value pairs of all local windows. Perform cross-window attention calculation: V(6) in, This is the key-value vector of the output features from the local multi-head self-attention layer after linear projection. Through this operation, the global query token can obtain feature summaries for each local window, thereby achieving modeling of the global structure of the track.

[0041] Output global attention With local features Perform splicing and fusion: (7) And adjust the dimension to using a fully connected layer. D Maintain consistency in feature dimensions.

[0042] (4) Feedforward Neural Network Layer: Enhances feature representation through nonlinear transformation, further refining the key features of cone-shaped recognition. Nonlinearity is increased and feature dimension restoration is achieved through two fully connected layers and activation functions; (5) Downsampling layer: Reduces the spatial resolution of the feature map, expands the receptive field, and increases the number of channels to capture richer semantic information.

[0043] Convolutional layers are used to downsample the feature maps, halving the output size and doubling the number of channels. The feature maps output by the convolution are then reshaped into a sequence. This information is used to input subsequent global self-attention layers or control action predictions.

[0044] In step 3) above, a control action prediction model based on reinforcement learning is constructed. An RL expert is trained using the Roach model, taking image features and measurement vectors as input, and outputting vehicle control parameters through an IL agent. This includes the following steps: 3.1) Construct a simulation environment in the CARLA simulator, collect data on the virtual driving platform to train the Roach model policy network and value network, use the proximal policy optimization algorithm to update the network parameters, and introduce maximum entropy loss to encourage exploration; 3.2) The IL agent is trained with expert data generated by the Roach model. The IL agent processes the input data through image encoding and measurement encoding, generates hidden parameters through a fully connected layer, and outputs throttle, brake and steering control parameters. The model is optimized through L1 loss and speed prediction regularization.

[0045] In this embodiment, as Figure 3 As shown, the control action prediction model based on reinforcement learning includes: Feature fusion layer: fuses the image features extracted by GCViT with the vehicle state vector; Strategy network layer (Actor): Generates continuous control actions (steering, throttle, braking); Value network layer (Critic): Evaluates the value of state-action pairs to assist in policy optimization.

[0046] The overall data flow can be represented as: in, for Control your actions at all times. For state vectors, For image features, For vehicle speed, This is the current steering angle.

[0047] The feature fusion layer achieves intermodal information exchange through gated fusion units: (8) (9) in, For the Sigmoid function, For element-wise multiplication, and For learnable parameters, The vehicle state feature vector is represented by the vehicle speed. Steering angle This consists of vehicle status information. This represents the learnable bias parameters of the gated fusion unit. This represents the gating coefficient (value range [0,1]). This represents the learnable weight matrix of the gated fusion unit. This represents the final feature after gating fusion. This mechanism allows the model to adaptively weigh the importance of visual and state information.

[0048] The policy network layer uses a three-layer fully connected network, with each layer containing multiple neurons, and the activation function is ReLU; control actions are generated through independent output heads. Steering angle:

[0049] accelerator:

[0050] in, This represents the learnable weight matrix of the steering angle prediction branch. This represents the output characteristics of the third layer of the policy network (a three-layer fully connected network). The learnable bias parameters represent the steering angle prediction branch. This indicates the maximum value of the steering angle. This represents the learnable weight matrix of the throttle prediction branch. This represents the learnable bias parameter of the throttle prediction branch.

[0051] The value network layer shares the parameters of the first two layers with the policy network, reducing computational overhead. The state features and action vectors are concatenated and then input into the value header. ,in, This represents the value estimate of a state-action pair, used to calculate the advantage function.

[0052] In this embodiment, the imitation learning agent uses a lightweight three-layer fully connected network to predict actions directly from fused features: The MLP output layer has two dimensions (steering and throttle).

[0053] Training process and hyperparameter configuration: Training process as follows Figure 4 As shown, the training phase first involves RL expert pre-training, collecting data through a simulation experimental platform, creating a dataset as required, and training in the CARLA simulation environment. Then, IL agent distillation is performed, using samples generated by RL experts to train the IL agent. Finally, real vehicle data is used to fine-tune the IL agent, combined with RL reward signals for optimization.

[0054] Using unmanned Formula One racing cars as a platform, a test platform with multimodal data acquisition and real-time control capabilities will be built, such as... Figure 5 As shown, the core components include: a binocular camera, an embedded computer, and an end-to-end control model for image feature extraction and control command generation. The sensor arrangement in the equation is as follows. Figure 5 As shown.

[0055] In the above embodiments, after outputting vehicle control parameters through the IL agent in step 3), a real vehicle verification step 4 is also included: constructing a temporary cone road simulation map based on the CARLA simulator, collecting data to train the model, optimizing the model through real vehicle data collection, and then performing real vehicle verification to statistically analyze indicators such as route completion rate and control delay.

[0056] In this embodiment, the model performance is verified in the CARLA simulator, and high-speed tracking experiments are conducted after optimization using data collected from real vehicles. The specific process of real-vehicle verification includes: Cameras, Hall sensors, and CAN bus devices are mounted on Formula One cars to collect cone road data under different weather and lighting conditions; Emergency braking safety protection is achieved through remote control. The effectiveness of the model in real-world scenarios is verified using route completion rate, violation rate, and control accuracy as indicators.

[0057] In one embodiment of the present invention, a vehicle end-to-end control system is provided, comprising: The data acquisition module uses the road simulation map built by the CARLA simulator to collect single-lane RGB images and measurement vectors of running vehicles in real time, and then constructs a dataset after preprocessing. The image feature extraction module takes the single-channel RGB image features as input to the GCViT image feature extraction model based on the cross-attention mechanism, realizes spatial interaction and feature sharing through local and global self-attention calculation, and outputs image features. The control action prediction module constructs a control action prediction model based on reinforcement learning, trains an RL expert using the Roach model, takes image features and measurement vectors as input, and outputs vehicle control parameters through an IL agent.

[0058] In the above embodiments, single-channel RGB image features are input into the GCViT image feature extraction model based on a cross-attention mechanism. Spatial interaction and feature sharing are achieved through local and global self-attention calculations, and the output image features include: Sub-images are generated by a complementary partitioning method of ordinary window and sliding window. Attention is calculated in sub-images through a local multi-head self-attention layer. A global query token is generated by a global lexical extraction layer and participates in the global multi-head self-attention calculation of each sub-image, thereby achieving the fusion of global information and local features. The feature map is reduced in dimensionality by a downsampling layer, resulting in image features with halved width and height and doubled number of channels.

[0059] In this embodiment, the GCViT image feature extraction model based on the cross-attention mechanism includes: Input layer: The input is a single-channel RGB image, which is then normalized and segmented. Local multi-head self-attention layer: The image is divided into multiple local windows using a fixed window partitioning method. The image blocks within each local window constitute a local feature subset. Self-attention is computed in parallel for each local feature subset through a multi-head mechanism. Global multi-head self-attention layer: It fuses global information across local windows, captures global structural features, generates learnable global query tokens through a global lexical extraction layer, performs cross-window attention calculation on the key-value pairs of the global query tokens and all local windows, and concatenates and fuses the global attention output with local features. Feedforward neural network layer: Adds non-linearity and restores feature dimensions through two fully connected layers and activation functions; Downsampling layer: Reduces the spatial resolution of the feature map, expands the receptive field, and increases the number of channels to capture richer semantic information.

[0060] In the above embodiments, a control action prediction model based on reinforcement learning is constructed. A Roach model is used to train an RL expert. Image features and measurement vectors are taken as input, and vehicle control parameters are output through an IL agent, including: A simulation environment was built in the CARLA simulator. Data was collected on the virtual driving platform to train the Roach model policy network and value network. The network parameters were updated using the proximal policy optimization algorithm, and maximum entropy loss was introduced to encourage exploration. The IL agent is trained using expert data generated by the Roach model. The IL agent processes the input data through image encoding and measurement encoding, generates latent parameters through a fully connected layer, and outputs throttle, brake, and steering control parameters. The model is optimized through L1 loss and speed prediction regularization.

[0061] In this embodiment, the control action prediction model based on reinforcement learning includes: Feature fusion layer: fuses the image features extracted by GCViT with the vehicle state vector; Policy network layer: Generates continuous control actions; Value network layer: Evaluates the value of state-action pairs to assist in policy optimization; The feature fusion layer achieves intermodal information exchange through gated fusion units; The policy network layer uses a three-layer fully connected network, with each layer containing multiple neurons and ReLU activation function; control actions are generated through independent output heads. The value network layer shares the parameters of the first two layers with the policy network, reducing computational overhead; the state features and action vectors are concatenated and then input into the value head.

[0062] In the above embodiments, after outputting vehicle control parameters through the IL agent, the method further includes a real vehicle verification step: constructing a temporary cone road simulation map based on the CARLA simulator, collecting data to train the model, and then optimizing the model using real vehicle data before conducting real vehicle verification.

[0063] The specific process of real-vehicle verification includes: Cameras, Hall sensors, and CAN bus devices are mounted on Formula One cars to collect cone road data under different weather and lighting conditions; Emergency braking safety protection is achieved through remote control. The effectiveness of the model in real-world scenarios is verified using route completion rate, violation rate, and control accuracy as indicators.

[0064] The system provided in this embodiment is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0065] The system provided in this embodiment is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0066] This application is described with reference to the technical architecture diagram and training flowchart of the method, system, and computer program product according to embodiments of this study. It should be understood that the feature extraction module and control prediction module in the architecture diagram, as well as the data acquisition, model training, and real-vehicle verification stages in the flowchart, can be coordinated through algorithmic logic. This algorithmic logic can be compiled into computer-executable instructions and deployed to the processor of a general-purpose computer, an in-vehicle embedded system, or other programmable computing devices to build a dedicated autonomous driving computing platform. This enables the instructions executed by the processor to achieve feature encoding of track images by the GCViT network in the architecture diagram, policy generation of control actions by the reinforcement learning model, and full automation of the process from simulation training to real-vehicle deployment in the flowchart.

[0067] These algorithmic logics can also be stored in onboard solid-state drives or cloud storage media. Such media can guide computers or edge computing devices to run according to end-to-end control logic, so that the stored instructions form a software artifact containing a complete algorithm chain. This artifact can realize the functional definitions of core links such as simulation data acquisition, model parameter optimization, and real vehicle control command generation in the flowchart, ensuring the vehicle's trajectory tracking and safe obstacle avoidance in temporary cone road scenarios.

[0068] These visual feature vectors can also be embedded in a high-dimensional feature space that guides deep learning models to operate with specific logic, enabling the vectors embedded in this feature space to generate a knowledge graph that includes abstract representations. This knowledge graph is implemented in the architecture. Figure 1 One or more modules and processes Figure 1The semantic associations specified in one or more steps.

[0069] These reinforcement learning strategies can also be deployed in onboard computing units that can guide vehicles to run along optimal control trajectories, enabling the strategies deployed in the computing units to generate control sequences including steering / throttle commands, which can predict specified actions in one or more corners of a test track and one or more segments of a real race track.

[0070] These multimodal sensor data can also be fused into a spatiotemporal data structure that can guide the decision-making system to operate robustly, so that the information fused into the data structure can generate state estimates including environmental perception, which can realize risk assessment in one or more scenarios in a simulation environment and one or more operating conditions in a real vehicle test.

[0071] These end-to-end training methods can also be applied to parameter spaces that can guide neural networks to learn in a collaborative optimization manner, enabling methods applied in this parameter space to produce joint models that include feature extraction and action prediction, achieving specified accuracy improvements in one or more dimensions of validation metrics and one or more aspects of performance evaluation.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle end-to-end control method, characterized in that, include: The road simulation map is built using the CARLA simulator, and the single-lane RGB images and measurement vectors of the running vehicles are collected in real time. After preprocessing, a dataset is constructed. The single-channel RGB image features are input into the GCViT image feature extraction model based on the cross-attention mechanism. Spatial interaction and feature sharing are achieved through local and global self-attention calculations, and the image features are output. A control action prediction model based on reinforcement learning is constructed. The Roach model is used to train RL experts. The vehicle control parameters are output through an IL agent with image features and measurement vectors as input.

2. The vehicle end-to-end control method as described in claim 1, characterized in that, The single-channel RGB image features are input into the GCViT image feature extraction model based on the cross-attention mechanism. Spatial interaction and feature sharing are achieved through local and global self-attention calculations, and the output image features include: Sub-images are generated by a complementary partitioning method of ordinary window and sliding window. Attention is calculated in sub-images through a local multi-head self-attention layer. A global query token is generated by a global lexical extraction layer and participates in the global multi-head self-attention calculation of each sub-image, thereby achieving the fusion of global information and local features. The feature map is reduced in dimensionality by a downsampling layer, resulting in image features with halved width and height and doubled number of channels.

3. The vehicle end-to-end control method as described in claim 2, characterized in that, The GCViT image feature extraction model based on the cross-attention mechanism includes: Input layer: The input is a single-channel RGB image, which is then normalized and segmented. Local multi-head self-attention layer: The image is divided into multiple local windows using a fixed window partitioning method. The image blocks within each local window constitute a local feature subset. Self-attention is computed in parallel for each local feature subset through a multi-head mechanism. Global multi-head self-attention layer: It fuses global information across local windows, captures global structural features, generates learnable global query tokens through a global lexical extraction layer, performs cross-window attention calculation on the key-value pairs of the global query tokens and all local windows, and concatenates and fuses the global attention output with local features. Feedforward neural network layer: Adds non-linearity and restores feature dimensions through two fully connected layers and activation functions; Downsampling layer: Reduces the spatial resolution of the feature map, expands the receptive field, and increases the number of channels to capture richer semantic information.

4. The vehicle end-to-end control method as described in claim 1, characterized in that, A reinforcement learning-based control action prediction model is constructed. An RL expert is trained using a Roach model, taking image features and measurement vectors as input, and outputting vehicle control parameters through an IL agent, including: A simulation environment was built in the CARLA simulator. Data was collected on the virtual driving platform to train the Roach model policy network and value network. The network parameters were updated using the proximal policy optimization algorithm, and maximum entropy loss was introduced to encourage exploration. The IL agent is trained using expert data generated by the Roach model. The IL agent processes the input data through image encoding and measurement encoding, generates latent parameters through a fully connected layer, and outputs throttle, brake, and steering control parameters. The model is optimized through L1 loss and speed prediction regularization.

5. The vehicle end-to-end control method as described in claim 4, characterized in that, Reinforcement learning-based control action prediction models include: Feature fusion layer: fuses the image features extracted by GCViT with the vehicle state vector; Policy network layer: Generates continuous control actions; Value network layer: Evaluates the value of state-action pairs to assist in policy optimization; The feature fusion layer achieves intermodal information exchange through gated fusion units; The policy network layer uses a three-layer fully connected network, with each layer containing multiple neurons and ReLU activation function; control actions are generated through independent output heads. The value network layer shares the parameters of the first two layers with the policy network, reducing computational overhead; the state features and action vectors are concatenated and then input into the value head.

6. The vehicle end-to-end control method as described in claim 1, characterized in that, After outputting vehicle control parameters through the IL agent, the process also includes a real-vehicle verification step: building a temporary cone road simulation map based on the CARLA simulator, collecting data to train the model, and then optimizing the model using real-vehicle data before conducting real-vehicle verification.

7. The vehicle end-to-end control method as described in claim 6, characterized in that, The specific process of real vehicle verification includes: Cameras, Hall sensors, and CAN bus devices are mounted on Formula One cars to collect cone road data under different weather and lighting conditions; Emergency braking safety protection is achieved through remote control. The effectiveness of the model in real-world scenarios is verified using route completion rate, violation rate, and control accuracy as indicators.

8. A vehicle end-to-end control system, characterized in that, include: The data acquisition module uses the road simulation map built by the CARLA simulator to collect single-lane RGB images and measurement vectors of running vehicles in real time, and then constructs a dataset after preprocessing. The image feature extraction module takes the single-channel RGB image features as input to the GCViT image feature extraction model based on the cross-attention mechanism, realizes spatial interaction and feature sharing through local and global self-attention calculation, and outputs image features. The control action prediction module constructs a control action prediction model based on reinforcement learning, trains an RL expert using the Roach model, takes image features and measurement vectors as input, and outputs vehicle control parameters through an IL agent.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images