End-to-end control system and method fusing multi-modal perception and strategy collaborative optimization mechanism

Abstract

The invention specifically discloses an end-to-end control system and method fusing multi-modal sensing and a strategy collaborative optimization mechanism. The system comprises a multi-modal sensing module; the key frame selection module and the time sequence modeling module are used for dynamically screening key frames from the historical sensing frames and carrying out time sequence modeling and feature coding by adopting a time sequence Transform model to obtain time sequence state features with semantic continuity; the strategy optimization module is used for constructing an Actor-Critic strategy network based on a maximum entropy reinforcement learning algorithm; the system is composed of a multi-mode sensing module, a time sequence modeling module, a strategy learning module and an end-to-end control module, a sensing-decision-control integrated closed-loop structure is formed, the system takes multi-source sensor data in a real traffic scene as input, and the multi-source sensor data in the real traffic scene are subjected to uniform feature coding and time sequence modeling, so that the real traffic scene is obtained. The method has the advantages of being short in information processing link, high in response speed, high in adaptive capacity and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation and automatic driving, and particularly relates to an end-to-end control system and method fusing multi-modal perception and strategy collaborative optimization mechanism. BACKGROUND

[0002] The existing automatic driving system adopts a phased architecture in which perception, prediction, decision, and control modules are connected in series. Although it has certain interpretability and stability, it has problems such as accumulation of perception errors, delay of decision response, and insufficient overall system robustness in dynamic traffic scenarios. The end-to-end learning method has potential collaborative optimization capability, but often lacks controllability and generalization ability. How to realize effective collaboration between perception and strategy in complex and variable traffic environments and build an efficient, robust, and real-time response-capable end-to-end control system has become a current research hotspot and technical bottleneck.

[0003] In view of the above problems, an end-to-end control system and method fusing multi-modal perception and strategy collaborative optimization mechanism are provided. SUMMARY

[0004] The present application aims to provide an end-to-end control system and method fusing multi-modal perception and strategy collaborative optimization mechanism to solve the problems raised in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solutions: An end-to-end control system fusing multi-modal perception and strategy collaborative optimization mechanism comprises: A multi-modal perception module for acquiring multi-source perception data of the environment in which the vehicle is located; A key frame selection module and a time series modeling module for dynamically selecting key frames from historical perception frames and using a time series Transformer model for time series modeling and feature encoding to obtain time series state features with semantic continuity; A strategy optimization module for constructing an Actor-Critic strategy network based on a maximum entropy reinforcement learning algorithm, training using an ALM-PPO method, and introducing an attention mechanism and a priority experience replay structure to realize strategy generation and continuous optimization; An end-to-end control integration module for directly mapping control actions output by the strategy network into bottom-level control instructions executable by the vehicle, including steering angle, acceleration, and braking commands.

[0006] In an optional solution, the multi-modal perception module includes image data and point cloud data, and an improved Swin Transformer and Hybrid PETR network structure are used to fuse and process the data, realize spatial alignment of multi-modal information and three-dimensional target detection, and output high-dimensional semantic feature representation.

[0007] In an optional solution, the multi-modal perception module further comprises a semantic segmentation head and a three-dimensional detection head, respectively outputting an aerial semantic map and a spatial target detection result, for enhancing the semantic richness of the input state of the strategy network.

[0008] In an optional solution, the key frame selection module adopts a key frame extraction strategy based on information entropy and traffic event dynamic measurement, for compressing redundant data and retaining important time sequence information.

[0009] In an optional solution, the Actor network and the Critic network in the strategy optimization module respectively adopt an LSTM and an attention mechanism fusion structure, for improving the modeling capability of the strategy for time dependence.

[0010] In an optional solution, the end-to-end control integration module is connected with a vehicle CAN bus interface, realizing direct mapping and execution of the perception data to the vehicle low-level control command.

[0011] In an optional solution, the system is deployed on a vehicle-mounted platform supporting heterogeneous computing, and is compatible with a GPU and an embedded chip for real-time inference and execution.

[0012] A control method of an end-to-end control system fusing multi-modal perception and strategy collaborative optimization mechanism, comprising the following steps: Step S1, collecting environment information from multi-modal sensors, inputting to a multi-modal perception module and outputting high-dimensional scene features; Step S2, extracting key frames from continuous perception frames, inputting to a time sequence modeling module, and constructing a dynamic semantic state sequence; Step S3, inputting the state features to a reinforcement learning strategy network, outputting a control action distribution and sampling; Step S4, converting the sampling result into an executable bottom-level control command, and sending to a vehicle control execution unit.

[0013] Compared with the prior art, the control method has the advantages that: In the present application, a multi-modal perception module, a time sequence modeling module, a strategy learning module and an end-to-end control module are combined to form a perception-decision-control integrated closed-loop structure, the system takes multi-source sensor data in a real traffic scene as input, and after unified feature coding and time sequence modeling, outputs decision commands directly applicable to vehicle control execution, and has the advantages of short information processing link, fast response speed and strong adaptability.

[0014] The end-to-end integrated design of the application improves system response efficiency, the application adopts an end-to-end perception decision fusion architecture, compared with a traditional module-level series system, reduces information transmission paths and interface dependence, significantly reduces perception error accumulation and decision delay, and improves the real-time performance and stability of the system as a whole.

[0015] In the application, multi-modal perception fusion enhances environmental understanding capability, the system introduces a Hybrid PETR-based image and point cloud joint encoding strategy, fully utilizes multi-source heterogeneous sensor information, realizes comprehensive perception of dynamic targets, road structures and traffic semantics, and has higher robustness and perception accuracy in complex scenes.

[0016] In the application, the combination of key frame driving and timing modeling strengthens the history-dependent modeling capability, through dynamic key frame extraction and multi-layer Transformer timing modeling module, the system can fully understand the evolution trend of the traffic situation, improve the prediction ability of the strategy network to the future scene change, and thus optimize the behavior planning and control strategy.

[0017] In the application, the reinforcement learning method based on the maximum entropy strategy improves the strategy robustness and generalization ability, combines the maximum entropy PPO algorithm and the priority experience replay mechanism, effectively solves the problems of low sample efficiency and local optimization in traditional strategy learning, can converge to the global strategy faster, and has strong generalization ability and real adaptability.

[0018] The application has strong deployability and supports multi-platform application, the system design fully considers the engineering landing demand, the perception and decision modules can run on mainstream automatic driving simulation platforms and embedded vehicle hardware, have good portability and scalability, and are convenient for popularization and application in different test scenes and product-level platforms.

[0019] The application is suitable for various typical dynamic traffic scenes, can be widely applied to various high-dynamic complex traffic situations such as lane changing, following, obstacle avoidance and overtaking on urban roads, and has good adaptability and control effect in actual tests, has significant engineering practical value and commercialization potential. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 It is a Transformer perception model structure diagram based on multi-view images and three-dimensional point clouds according to the application.

[0021] Figure 2 It is a strategy network structure diagram based on attention mechanism and time modeling in the application.

[0022] Figure 3 It is a multi-modal state space vector construction flowchart in the application.

[0023] Figure 4The ALM-PPO algorithm flowchart based on the maximum entropy strategy in the application.

[0024] Figure 5 The perception visualization result display diagram of the system in a typical dynamic traffic scene in the application.

[0025] Figure 6 The performance comparison diagram of the ALM-PPO different module variants in the application in terms of average reward, convergence step number and reward variance. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the application.

[0027] Please refer to Figures 1-6 , Figure 1 The specific implementation path of the perception module is shown, including the double-channel processing flow of images and point clouds, the feature-guided position encoder, the Transformer decoder, and the output form of the segmentation head (Seg Head) and the detection head (DetHead) for generating semantic maps and three-dimensional detection results.

[0028] Figure 2 The state space features are first extracted by the MBConv module to obtain a basic representation, then input into a multi-layer attention module (including Multi-Attention, Linear, Dropout, Norm, etc.), complete time series modeling through LSTM, and finally connected to the Actor and Critic networks to realize policy output and value estimation.

[0029] Figure 3 It is shown in the application that the RGB images from the camera and the laser radar point cloud complete spatial feature fusion through the Hybrid PETR model to output semantic segmentation maps and BEV bird's eye view maps, and at the same time, combine vehicle sensor information, global instruction signals and state normalization processing, and finally generate a unified state vector that can be used as input of the reinforcement learning policy network through the VAE module.

[0030] Figure 4The automatic driving environment interaction process, the construction method of the experience replay buffer pool, the training and updating path of the Actor-Critic network, the advantage function estimation and the loss function construction process are described in detail. In particular, the core update logic of the policy ratio truncation and the Clipped Surrogate Objective is displayed, which is one of the key implementation methods of the decision optimization module in the application.

[0031] Figure 5 The left side of the middle is a multi-view RGB image result, covering the front, back, left and right directions of the vehicle (FrontLeft, Front, Front Right, Back Left, Back, Back Right), the middle is the corresponding laser radar point cloud image (Lidar), and the right side is the semantic projection result under the high-precision map (Map).

[0032] Figure 6 Model A is the complete structure, B to F respectively remove the maximum entropy mechanism, priority experience replay, Bayesian value function, multi-head attention mechanism and LSTM time series modeling, and G is the original PPO baseline model. The results show that the complete structure (model A) is better than other variants in all indicators, verifying the effectiveness of each module in improving the strategy performance, training efficiency and decision stability. In this embodiment, an end-to-end control system integrating multi-modal perception and strategy collaborative optimization mechanism comprises: A multi-modal perception module for acquiring multi-source perception data of the environment in which the vehicle is located; A key frame selection module and a time series modeling module for dynamically selecting key frames from historical perception frames and using a time series Transformer model for time series modeling and feature coding to obtain time series state features with semantic continuity; A strategy optimization module based on the maximum entropy reinforcement learning algorithm to construct an Actor-Critic strategy network, using the ALM-PPO method for training, and introducing an attention mechanism and a priority experience replay structure to realize strategy generation and continuous optimization; An end-to-end control integration module for directly mapping the control actions output by the strategy network into bottom-level control instructions executable by the vehicle, including steering angle, acceleration and brake commands.

[0033] The multi-modal perception module is responsible for collecting and fusing data from different sensors, including forward-facing RGB cameras, LiDARs, millimeter wave radars, etc. It extracts image features through an improved Swin Transformer network and combines a Hybrid PETR structure to complete multi-modal spatial alignment and three-dimensional target detection tasks. This module can accurately obtain the position information of dynamic and static targets in the environment and generate a unified high-dimensional semantic expression, significantly enhancing the system's perception robustness in complex environments and the efficiency of utilizing multi-source information.

[0034] The multi-modal perception module (Module A) uses an improved Transformer network architecture to extract features and fuse semantics from each modality data. The fusion strategy includes a spatial alignment-based point cloud-image joint encoding mechanism and a cross-modal attention mechanism, thereby obtaining accurate and structured environmental representation information. The high-dimensional perception vector output by this module serves as the main input for subsequent strategy decision-making.

[0035] In the key frame selection module and the time series modeling module, considering the strong temporal dynamics and frequent state changes in traffic environments, a key frame selection strategy based on an environmental dynamic scoring mechanism is used to extract only perception frames with high information density and dramatic state changes, reducing the burden of redundant information. On this basis, a multi-layer time series Transformer network is used to model the temporal dependence of perception history, enhancing the system's understanding of event development and improving the prediction accuracy and decision stability of the strategy module for future states.

[0036] In the key frame selection module and the time series modeling module (Module B), the multi-modal perception results are buffered and then entered into the key frame selection module. The system dynamically selects representative information frames based on the degree of traffic environment change (such as target speed change rate, obstacle density mutation, etc.), and uses a time series Transformer structure for time series modeling and state embedding. This module can effectively depict the dynamic evolution trend of the traffic situation, providing semantic coherent time series feature input for strategy decision-making.

[0037] In the strategy optimization module, an improved Proximal Policy Optimization algorithm (ALM-PPO) under the maximum entropy framework is used as the core strategy learning algorithm. An attention mechanism-based state-action embedding fusion mechanism is introduced to realize the collaborative input of information from different modalities and time scales. During training, a priority experience replay strategy is combined to assign higher learning weights to rare or critical traffic events, thereby improving the sample efficiency and generalization ability of the strategy network. The final output action instructions include the vehicle's longitudinal acceleration, lateral steering angle, and brake control signals, which can be directly applied to the vehicle execution system.

[0038] During the training process of the strategy optimization module (module C), the priority experience replay mechanism is combined to enhance the model's response capability to rare traffic events (such as sudden lane changes and sudden braking), and to improve the generalization and robustness of the overall strategy. In the reasoning stage, the system outputs specific control actions, including acceleration, steering angle, and brake amplitude, based on the current state.

[0039] In the end-to-end control integration module, a unified modeling paradigm is adopted, and a single neural network model is used for training and reasoning from perception to control. This avoids error accumulation and interface mismatch problems in multi-stage information transmission. The control system has online reasoning and rapid adaptation capabilities, and can operate stably in various typical scenarios such as urban streets, highways, and traffic junctions.

[0040] The end-to-end control integration module (module D) directly maps the strategy output to low-level vehicle control commands, eliminating the intermediate steps in traditional path planning and behavior decision-making. It establishes a direct mapping relationship from environmental perception to control execution, significantly reducing response delay and error accumulation. The module supports vehicle chassis control calls based on CAN bus interfaces and is compatible with mainstream automatic driving simulation platforms (such as CARLA) for debugging and testing.

[0041] The system architecture supports both module-level decoupling and end-to-end debugging, allowing independent verification of perception and strategy module performance, as well as overall deployment on a vehicle-mounted computing platform. The system is compatible with mainstream automatic driving simulation platforms (such as CARLA) and actual sensor interfaces, and has good engineering portability and real-world potential.

[0042] The multi-modal perception module includes image data and point cloud data, and uses improved Swin Transformer and Hybrid PETR network structures for data fusion processing, achieving spatial alignment of multi-modal information and three-dimensional target detection, and outputting high-dimensional semantic feature representations.

[0043] The multi-modal perception module also includes a semantic segmentation head and a three-dimensional detection head, which output an aerial semantic map and spatial target detection results, respectively, to enhance the semantic richness of the strategy network input state.

[0044] The key frame selection module uses a key frame extraction strategy based on information entropy and traffic event dynamic measurement to compress redundant data and retain important temporal information.

[0045] The Actor network and Critic network in the strategy optimization module use LSTM and attention mechanism fusion structures, respectively, to improve the model's ability to model temporal dependencies.

[0046] The end-to-end control integration module is connected with a vehicle CAN bus interface to realize direct mapping and execution of perception data to vehicle low-level control commands.

[0047] The system is deployed on a vehicle platform supporting heterogeneous computing, and is compatible with GPU and embedded chips for real-time inference and execution.

[0048] When the system is applied to a loader, the multi-modal perception module can be responsible for comprehensively perceiving the state of the loader itself and the surrounding dynamic environment. The key frame selection module and the timing modeling module dynamically filter key frames in historical perception data in a complete loading cycle of the loader. The system focuses on retaining state frames at key moments such as "the bucket just touches the material pile", "the maximum digging resistance", "lifting to a safe height", and "starting to turn", and filtering out a large number of similar frames in straight driving. The strategy optimization module can make optimal operation decisions based on the understood environmental state. The end-to-end control integration module converts the decisions into specific actions of the loader hydraulic and transmission system and sends them to the corresponding actuators.

[0049] A control method of an end-to-end control system integrating multi-modal perception and strategy collaborative optimization mechanism, comprising the following steps: Step S1, collecting environmental information from multi-modal sensors, inputting to a multi-modal perception module and outputting high-dimensional scene features; Step S2, extracting key frames from continuous perception frames, inputting to a timing modeling module, and constructing a dynamic semantic state sequence; Step S3, inputting state features into a reinforcement learning strategy network, outputting control action distribution and sampling; Step S4, converting the sampling results into executable bottom-level control commands and sending them to a vehicle control execution unit.

[0050] The above is only a preferred embodiment of the present application, and is not intended to limit the other forms of the present application. Any person skilled in the art can use the disclosed technical content to make changes or modifications to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made in accordance with the technical essence of the present application to the above embodiments shall fall within the protection scope of the present application.

Claims

1. An end-to-end control system integrating multimodal perception and policy collaborative optimization mechanisms, characterized in that, include: The multimodal perception module is used to acquire multi-source perception data of the vehicle's environment; The keyframe selection module and the temporal modeling module are used to dynamically filter keyframes from historical sensing frames and use the temporal Transformer model for temporal modeling and feature encoding to obtain temporal state features with semantic continuity. The policy optimization module constructs an Actor-Critic policy network based on the maximum entropy reinforcement learning algorithm, trains it using the ALM-PPO method, and introduces an attention mechanism and a priority experience replay structure to achieve policy generation and continuous optimization. The end-to-end control integration module is used to directly map the control actions output by the policy network into low-level control commands that can be executed by the vehicle, including steering angle, acceleration, and braking commands.

2. The end-to-end control system integrating multimodal perception and policy collaborative optimization mechanism according to claim 1, characterized in that: The multimodal perception module includes image data and point cloud data, and performs data fusion processing through an improved SwingTransformer and Hybrid PETR network structure to achieve spatial alignment of multimodal information and 3D target detection, and output high-dimensional semantic feature representation.

3. The end-to-end control system integrating multimodal perception and policy collaborative optimization mechanism according to claim 2, characterized in that: The multimodal perception module also includes a semantic segmentation head and a 3D detection head, which output a bird's-eye view semantic map and spatial target detection results, respectively, to enhance the semantic richness of the input state of the policy network.

4. The end-to-end control system integrating multimodal perception and policy collaborative optimization mechanism according to claim 1, characterized in that: The keyframe selection module employs a keyframe extraction strategy based on information entropy and dynamic traffic event measurement to compress redundant data and retain important temporal information.

5. The end-to-end control system integrating multimodal perception and policy collaborative optimization mechanism according to claim 1, characterized in that: The Actor network and Critic network in the policy optimization module adopt a fusion structure of LSTM and attention mechanism to improve the policy's ability to model temporal dependencies.

6. The end-to-end control system integrating multimodal perception and policy collaborative optimization mechanism according to claim 1, characterized in that: The end-to-end control integration module is connected to the vehicle's CAN bus interface, enabling direct mapping and execution of sensing data into low-level vehicle control commands.

7. The end-to-end control system integrating multimodal perception and policy collaborative optimization mechanism according to claim 1, characterized in that: The system is deployed on an in-vehicle platform that supports heterogeneous computing and is compatible with GPUs and embedded chips for real-time inference and execution.

8. A control method for an end-to-end control system integrating multimodal perception and policy collaborative optimization mechanisms as described in any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Collect environmental information from multimodal sensors, input it into the multimodal perception module, and output high-dimensional scene features; Step S2: Extract keyframes from continuous sensing frames, input them into the temporal modeling module, and construct a dynamic semantic state sequence; Step S3: Input the state features into the reinforcement learning policy network, output the distribution of control actions, and sample it; Step S4: Convert the sampling results into executable low-level control commands and send them to the vehicle control execution unit.

Images