Automatic driving control system and method based on world model and intelligent chassis

Abstract

The invention provides an automatic driving control system and method based on a world model and an intelligent chassis, and belongs to the field of automatic driving, and the method comprises the steps: obtaining multi-source data of an external environment, and constructing a time sequence feature sequence according to the multi-source data; performing explicit probability modeling according to the time sequence feature sequence to obtain scene features; generating a world query vector according to the scene features, and performing prediction through a world model according to world query and a time sequence feature sequence to obtain a future Gaussian scene state; generating a track point sequence according to a future Gaussian scene state, and performing confidence evaluation on the track point sequence to obtain track point confidence; and generating a control instruction according to the track point sequence and the track point confidence coefficient, performing coordination control according to the control instruction, generating a final control vector, and controlling the chassis through the final control vector. Feedback data is obtained, evaluation is carried out according to the feedback data, and the final control vector and the world model are adjusted and optimized, so that effective chassis control optimization is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving. Specifically, it relates to an autonomous driving control system and method based on a world model and an intelligent chassis, belonging to the technical direction of intelligent driving decision-making and control systems. Background Art

[0002] Most current autonomous driving systems adopt a modular structure, with functions such as perception, prediction, decision-making, and control designed separately. Although this type of architecture has clarity in engineering implementation, due to the dependence on fixed interfaces for information transfer between modules, it is difficult to achieve system-level linkage and collaborative optimization. Especially in a dynamic traffic environment, problems such as information lag, error amplification, and inconsistent system responses are prone to occur. At the same time, it is relatively difficult for modular systems to capture long-term temporal dependencies in the environment, restricting their generalization ability and continuous understanding ability for complex driving tasks.

[0003] In order to improve the system's overall understanding ability of the environment, the industry has begun to explore a unified expression method centered on scene semantic modeling, enabling the system to construct an internal world representation based on historical states and current observations and perform future state predictions. However, existing methods generally have a disconnection problem at the execution level, and it is difficult to directly convert the world modeling results into high-precision and controllable underlying control behaviors. At the same time, the behavior decision-making module often lacks linkage feedback with the actuator state and cannot perform online correction under complex working conditions, resulting in the expansion of trajectory deviation and even inducing safety risks. In addition, the lack of a confidence evaluation mechanism for trajectory planning results also weakens the decision-making credibility of the system in unstructured scenarios.

[0004] In terms of execution control, as the ultimate carrier of autonomous driving decision results, the control accuracy, response timeliness, and stability of the intelligent chassis system play a decisive role in driving safety. However, existing chassis control systems mostly focus on path tracking and one-way instruction execution, lacking a joint control optimization mechanism for world state prediction results. At the same time, there is a lack of effective health monitoring and redundancy fault tolerance strategies to support the problem of fault accumulation and performance degradation during the long-term operation of actuators, making it difficult to ensure the stable and reliable operation of the system under high-frequency control cycles. Therefore, there is an urgent need to build an end-to-end control system from scene modeling to behavior reasoning, and then to intelligent execution and feedback regulation, truly realizing the collaborative modeling and closed-loop control of the world model and the intelligent chassis. Summary of the Invention

[0005] In view of this, in order to solve the technical problems of amplified control errors caused by information transmission lag, mismatch between world model semantic representation and chassis execution parameters, and failure to reverse the decision model due to actuator health status in modular autonomous driving architecture, this invention provides an end-to-end autonomous driving control system and method based on collaborative modeling of world model and intelligent chassis. By constructing a spatiotemporally aligned BEV-language-control joint coding system and a two-way dynamic feedback mechanism, it achieves closed-loop optimization from environmental semantic understanding to precise chassis execution under limited training data and limited computing power.

[0006] To achieve the above objectives, this invention proposes an autonomous driving control system and method based on a world model and an intelligent chassis, comprising: 1. An autonomous driving control system based on a world model and intelligent chassis, characterized in that it includes: Multimodal perception module, world model modeling and prediction module, behavior decision-making and trajectory generation module, chassis control execution module, and feedback learning and state monitoring module; Multi-source data of the external environment is acquired through a multi-modal perception module, and a time-series feature sequence is constructed based on the multi-source data; The world model modeling and prediction module performs explicit probabilistic modeling based on the temporal feature sequence to obtain scene features; a world query vector is generated based on the scene features; and the future Gaussian scene state is obtained by predicting the world query and the temporal feature sequence through the world model. The behavior decision-making and trajectory generation module generates a sequence of trajectory points based on the future Gaussian scene state, and evaluates the confidence of the trajectory point sequence to obtain the trajectory point confidence. The chassis control execution module generates control commands based on the trajectory point sequence and trajectory point confidence, performs coordinated control based on the control commands, generates a final control vector, and controls the chassis through the final control vector. Feedback data is obtained through the feedback learning and state monitoring module, and the final control vector and world model are adjusted and optimized based on the feedback data.

[0007] Optionally, in the multimodal sensing module, the process of constructing the temporal feature sequence includes: The multi-source data is calibrated, wherein the multi-source data includes data collected by visual sensors, lidar, millimeter-wave radar, and global positioning system; Feature extraction and cross-modal alignment are performed on the calibrated data to obtain fused features; The fused features are projected into the bird's-eye view representation space and combined with adjacent frames to generate a temporal feature sequence.

[0008] Optionally, in the world model modeling and prediction module, the process of explicitly probabilistically modeling the time-series feature sequence includes: Gaussian distribution is used to model the explicit probability of semantic units corresponding to time-series feature sequences to generate Gaussian scene representations; the Gaussian scene representations include the position mean, covariance matrix, and semantic category labels of the semantic units. The Gaussian scene representation is dynamically updated through a scene evolution mechanism to obtain scene features; the scene evolution mechanism includes static alignment, dynamic modeling, and region completion.

[0009] Optionally, in the world model modeling and prediction module, the process of obtaining the future Gaussian scene state includes: Obtain the semantic description of the Gaussian scene, encode the semantic description into a semantic vector, and fuse the semantic vector with the scene features to generate a world query vector; The world query vector and the temporal feature sequence are used as input to the world model. The world model is used to recursively predict the future temporal sequence to obtain the future Gaussian scene state.

[0010] Optionally, in the behavior decision-making and trajectory generation module, the process of generating trajectory point confidence includes: A semantic BEV-driven trajectory intent modeling network infers high-level driving intent based on the future Gaussian scene state and temporal feature sequence. Based on the high-level driving intention and the future Gaussian scene state, a trajectory point sequence is generated, wherein the trajectory point sequence includes the position of the trajectory point, the expected velocity, and the expected acceleration.

[0011] Optionally, in the chassis control execution module, the process of generating the final control vector includes: Trajectory analysis is performed based on the trajectory point sequence and trajectory point confidence to generate control commands. These commands are then coordinated and controlled by an intelligent chassis hierarchical model predictive controller to generate the final control vector. Specifically, within the intelligent chassis hierarchical model predictive controller, a corresponding cost function is calculated based on the error of the trajectory point sequence and the control commands. The control commands are optimized with the goal of minimizing the cost function, and the optimized control commands are adjusted to generate the final control vector.

[0012] Optionally, in the feedback learning and state monitoring module, the error between the final control command execution and the actual execution is calculated based on the feedback data, and the error in the control command execution is used to adjust the final control command or replan the trajectory; wherein the feedback data includes the vehicle's actual steering angle, acceleration, braking force and real-world scenario state; The world model is adjusted and optimized based on the real-world state and the future Gaussian state.

[0013] On the other hand, this invention provides an autonomous driving control method based on a world model and an intelligent chassis, including: Acquire multi-source data of the external environment, and construct a time-series feature sequence based on the multi-source data; Explicit probabilistic modeling is performed based on the temporal feature sequence to obtain scene features; a world query vector is generated based on the scene features; and the future Gaussian scene state is obtained by predicting the world query and the temporal feature sequence through the world model. Based on the future Gaussian scene state, a trajectory point sequence is generated, and the confidence of the trajectory point sequence is evaluated to obtain the trajectory point confidence. Control commands are generated based on the trajectory point sequence and trajectory point confidence, coordinated control is performed based on the control commands, a final control vector is generated, and the chassis is controlled through the final control vector; Obtain feedback data, evaluate it based on the feedback data, and adjust and optimize the final control vector and world model.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves end-to-end closed-loop control by constructing an end-to-end closed-loop control system, realizing full-process coordination from multimodal perception, world modeling, trajectory reasoning to chassis control and feedback fine-tuning. It avoids the problems of information transmission lag and error accumulation in sub-modules in modular architecture, and significantly improves the overall response efficiency and system robustness of autonomous driving system in complex scenarios.

[0015] This invention proposes an explicit Gaussian modeling and language enhancement fusion mechanism, which introduces a three-factor driven scene evolution strategy (static alignment, dynamic modeling, and new region completion) in the world model construction process. At the same time, it integrates language prior knowledge for conditional query and scene prediction, which significantly improves the system's modeling and generalization ability for unknown environments and long-tail tasks.

[0016] This invention implements an interpretable trajectory generation mechanism based on semantic BEV. By combining trajectory intent modeling and confidence evaluation modules, the output trajectory has clear semantic reasoning logic and quantifiable uncertainty expression, which improves the controllability and safe interpretability of the system in the behavior decision-making stage.

[0017] This invention constructs an intelligent chassis hierarchical control and feedback fine-tuning mechanism. Through the collaborative optimization of upper-layer MPC trajectory tracking and lower-layer actuator coordinated control, as well as actuator health monitoring and fault compensation mechanisms, dynamic adaptive and fault-tolerant redundant scheduling of the control layer is realized, effectively enhancing the system's stability and safety redundancy capabilities at the execution level. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1 This is a schematic diagram of the method flow in an embodiment of the present invention; Figure 2 This is a system structure diagram in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the multimodal perception and spatiotemporal BEV construction principle in an embodiment of the present invention. Figure 4 This is a schematic diagram of the language-enhanced world modeling and scene prediction structure in an embodiment of the present invention; Figure 5 This is a flowchart of the behavior decision-making and trajectory generation process in an embodiment of the present invention; Figure 6 This is a schematic diagram of robust chassis adaptive control and redundant actuator coordination in an embodiment of the present invention; Figure 7 This is a diagram showing the interface design of an end-to-end autonomous driving system in an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] To achieve the above objectives, the present invention provides the following technical solution: An end-to-end autonomous driving control method based on collaborative modeling of a world model and an intelligent chassis includes the following steps: Step S1, Multimodal Perception and Spatiotemporal BEV Construction: Vehicle environmental information is acquired through multi-source sensors such as vision, radar, and GPS. After preprocessing, multimodal semantic features are extracted and uniformly projected onto the bird's-eye view space. Based on this, a temporal BEV feature sequence between consecutive frames is constructed to provide dynamic semantic input for world model modeling.

[0021] Step S2, Language-Enhanced World Modeling and Scene Prediction, employs a Gaussian distribution to explicitly model semantic elements in the scene, integrating static alignment, dynamic modeling, and new region completion mechanisms to achieve scene evolution modeling. Simultaneously, language priors are introduced to generate world query vectors, which are injected into the model as modeling conditions to predict the scene state at several future moments.

[0022] Step S3, End-to-end behavioral decision reasoning and trajectory generation, based on the BEV semantic input of the current and predicted scenarios, infers the vehicle's intent and driving strategy, outputs a sequence of trajectory points through the control decoupling module, and evaluates the confidence of each trajectory point, thereby reducing computational complexity while supporting the interpretable execution of subsequent control modules.

[0023] Step S4: Robust chassis adaptive control. The trajectory point sequence is parsed into vehicle-level control commands, which are then processed by a hierarchical model predictive controller, including upper-level trajectory tracking optimization and lower-level actuator coordinated control. Simultaneously, redundant actuator scheduling and fault compensation mechanisms are introduced to address control redundancy and execution anomalies, ensuring system operational stability.

[0024] Step S5, Feedback Closed Loop and Continuous World Model Fine-tuning: Based on the chassis control execution status and external perception feedback, the trajectory execution error is dynamically evaluated, and online fine-tuning of the world model is performed under safety constraints to improve adaptability to complex scenarios. Long-term reliable operation of the control system is achieved through a chassis actuator health monitoring module.

[0025] Furthermore, step S1, multimodal perception and spatiotemporal BEV construction, includes: First, multi-source data on the external environment are collected using visual sensors, lidar, millimeter-wave radar, and GPS installed on the vehicle to form a raw perception dataset:

[0026] in, Indicates the time frame number. For camera image frames, For lidar point clouds, For millimeter-wave radar echo data, This is for location information.

[0027] Secondly, the multi-source data is synchronized in time and calibrated in space, and further processed to obtain multimodal fusion features in a unified coordinate system. :

[0028] in, This represents the feature extraction and cross-modal alignment operator.

[0029] Subsequently, the fusion features will be Projecting the space onto a bird's-eye view (BEV) Generate static spatial feature mapping And combine the features of adjacent frames to construct a temporal feature sequence.

[0030] in, The time window length, It is a temporal BEV feature sequence used to characterize the dynamic semantic information around the vehicle and provide input for explicit modeling of the world model.

[0031] Furthermore, the language-enhanced world modeling and scene prediction in step S2 include: First, based on the time-series BEV feature sequence obtained in step S1 Gaussian scene representation is constructed by explicitly probabilistically modeling each semantic unit in the scene using a Gaussian distribution. :

[0032] The semantic unit represents a temporal BEV feature sequence within the bird's-eye view space. The semantically consistent regions or target instances formed by aggregation correspond to environmental components such as road elements, traffic participants, or obstacles in the scene, and are used to represent environmental elements such as road structure, traffic participants, or passable space. Indicates the first The average position of each scene element in the BEV space Let covariance matrix be the variance matrix. For semantic category labels, This represents the total number of scene elements in the current frame.

[0033] Secondly, a three-factor driven scene evolution mechanism is adopted for Gaussian scenes. Dynamic updates are performed. Static alignment: using a fixed reference frame Global registration of the current Gaussian element positions reduces sensor drift error; Dynamic modeling: through time difference And velocity estimation predicts motion trends; New region completion: For regions that exceed the historical observation range, a Gaussian generation completion strategy based on interpolation and prior estimation is adopted to ensure the global integrity of the scene.

[0034] Furthermore, prior language knowledge is introduced to describe high-level semantics. Encoded as semantic vectors It is then fused with scene features to generate a world query vector. :

[0035] in, This represents a multimodal fusion operator used to map geometric and semantic priors to a unified query space, where the high-level semantic description... It is a high-level semantic constraint description of the current driving scenario and task objectives. It includes, but is not limited to: driving task intent, road structure semantics, traffic rules and constraints, risks and key concerns.

[0036] Finally, using the world query vector Conditional input prediction module Regarding the future Recursively predict the scene state at each time step:

[0037] Output Indicates the first time domain within the prediction time domain The Gaussian scene state at each moment realizes a closed-loop reasoning from current semantic understanding to future scene evolution, and the prediction module... A world model is used to perform time-recursive prediction of scene states under the conditional constraints of the world query vector QW.

[0038] Furthermore, step S3, end-to-end behavioral decision reasoning and trajectory generation, includes: First, based on the future scenario state sequence predicted in step S2 ; and the corresponding time-series BEV features Trajectory Intent Modeling Network Driven by Semantic BEV This paper employs a semantic modeling approach based on conditional reasoning. Under the conditional constraints of the world model's prediction results, it models and infers feasible driving strategies for the vehicle in the current scenario, thereby obtaining the vehicle's high-level driving intention. :

[0039] in, This indicates the type of driving strategy inferred under the current scenario conditions (such as going straight, overtaking, changing lanes, obstacle avoidance, etc.).

[0040] Secondly, based on the aforementioned high-level driving intentions And combined with the future scene state sequence predicted in step S2 Under scenario constraints, a sequence of control trajectory points is generated through condition generation or constraint optimization. :

[0041] in, Indicates the first Two-dimensional position of each trajectory point Expected speed and expected acceleration , The length of the trajectory.

[0042] Finally, the generated trajectory point sequence Perform interpretability analysis and confidence assessment. Interpretability analysis module. The trajectory is semantically associated with the input scene to provide a basis for decision-making. Specifically, by analyzing the spatial relationship and semantic category association between the trajectory point sequence and the corresponding semantic units in the Gaussian scene, a correspondence between trajectory decisions and scene elements is established, thus providing a semantic explanation for driving decisions. The confidence assessment module outputs the uncertainty quantification value for each trajectory point. :

[0043] in, Indicates the first The confidence level of each trajectory point is used for risk-weighted execution in subsequent control phases.

[0044] Furthermore, the robust chassis adaptive control in step S4 includes: First, based on the trajectory point sequence output in step S3 and its confidence set By trajectory analysis module The high-level trajectory point information is parsed into the vehicle's low-level control target using the trajectory-control mapping method:

[0045] in, Indicates the desired steering angle. Indicates an acceleration command. This indicates a braking command. Specifically, the trajectory parsing module outputs the trajectory point sequence in step S3. The input consists of the main input combined with the corresponding trajectory confidence set. Under the premise of satisfying the vehicle's geometric motion relationship and control constraints, the position, velocity and acceleration information in the trajectory are mapped into the underlying executable control target; among them, the trajectory confidence is used to perform risk modulation on the amplitude and response intensity of the control target.

[0046] Secondly, control commands Predictive controller based on intelligent chassis hierarchical model The controller includes the following processing features: Upper-level trajectory tracking optimizer Based on vehicle dynamics model The vehicle dynamics model is a low-dimensional, analytical vehicle prediction model used to describe the state evolution of a vehicle under control inputs. Within a model predictive control framework, this model predicts the vehicle's position, attitude, and velocity changes over multiple future time steps based on the current vehicle state and control inputs, thereby calculating the deviation between the predicted trajectory and the reference trajectory. (Predicting the future...) Step trajectory deviation And minimize the cost function:

[0047] in, As a reference trajectory point, To predict the location, These are the weighting coefficients. For control sequence The control input vector at time t+h; the confidence level affects the cost function weights in this content. The risk-weighted execution introduces the confidence level of the trajectory points output in step S3 into the trajectory analysis and upper-level model predictive control process, dynamically adjusting the cost function weights during the generation of control objectives and trajectory tracking optimization stages, thereby reducing the proportion of trajectory points with low confidence levels in control optimization, and thus achieving adaptive control execution based on risk level.

[0048] Lower-level actuator coordination controller The optimized steering, drive, and braking commands from the upper layer are distributed to multiple redundant actuators to ensure consistent response and optimal energy efficiency under multi-actuator collaboration.

[0049] Furthermore, to address actuator failures or performance degradation, a redundant actuator coordination and fault compensation module is introduced. ,in, This represents a redundant actuator coordination and fault compensation mapping operator, used to redistribute or compensate for control inputs based on actuator state feedback. Its functions include monitoring the deviation between the actual output and desired control input of each actuator; redistributing control commands or switching actuator operating modes when an anomaly is detected; and ensuring that the overall vehicle control objective is still met even in the event of actuator degradation or failure. (The text also mentions detecting actuator output deviations.) Exceeding the threshold When necessary, automatically switch to the standby actuator or adjust the actuator weight allocation matrix. To maintain vehicle control stability:

[0050] in, Assign a weight matrix to the actuator. For the number of actuators, This is the final control vector that is actually sent to each actuator.

[0051] Furthermore, the fine-tuning of the feedback loop and the continuous world model in step S5 includes: First, through the execution status and perception feedback acquisition module The actuator feedback signals and sensing system outputs from the intelligent chassis are uniformly collected and mapped to form an execution state vector:

[0052] and the external environment state vector from the sensing system

[0053] in, This represents a feedback mapping operator between execution status and perceived information, used to convert chassis sensor signals and environmental observation data into a unified feedback status representation. , , These represent the time of the vehicle. The actual steering angle, acceleration, and braking force. This represents the dimension of environmental state characteristics.

[0054] Secondly, utilizing the error assessment and dynamic adjustment module By adjusting relevant control parameters, triggering trajectory or strategy recalculation, and constraining the online learning amplitude, the error vector between the predicted execution instruction and the actual execution result is calculated:

[0055] in, This is the set of reference control variables output by the upper-level trajectory tracking optimizer in step S4. When ( When controlling the error threshold, The system adaptively adjusts control-related parameters, trajectory confidence weights, or prediction time domain length based on the error magnitude, and triggers trajectory regeneration or control parameter reconfiguration when necessary to suppress error accumulation.

[0056] Secondly, the world model adaptive learning module under security constraints. In the process, feedback data is used to analyze the parameters of the world model. Perform online fine-tuning:

[0057] The world model refers to the scene modeling and prediction model constructed in step S2, based on Gaussian representation and language prior enhancement. For learning rate, The future predicted by the model Step-by-step scene state sequence, This is a sequence of real-world scene states based on perceptual feedback. This is the loss function for the world model. Online fine-tuning of the world model parameters is essentially a joint adjustment of the parameters from step S2.

[0058] Finally, through the chassis actuator health status monitoring module The stability and response delay of the actuator output and energy consumption Conduct multi-indicator assessments to generate health indicators. :

[0059] in, It is the actuator output deviation in S4.

[0060] when Below the threshold At that time, the health status information is fed back to the redundant actuator coordination mechanism in step S4. This enables a secure closed loop across modules.

[0061] The present invention will be further described below with reference to the accompanying drawings and specific embodiments: The end-to-end autonomous driving control method based on collaborative modeling of a world model and an intelligent chassis, as described in this invention, can be divided into five sequentially executed sub-steps. Each step uses the output of the previous step as the input for the next step, such as... Figure 1 As shown, the processing flow is as follows: Step S1: Multimodal perception and spatiotemporal BEV construction. Environmental data is collected and preprocessed through multi-source sensors, unified multimodal features are extracted, and a continuous frame temporal BEV feature sequence is constructed to provide dynamic semantic input for subsequent world model modeling. This includes the following steps: S1.1: Multi-source sensing data acquisition and preprocessing S1.2: Multimodal Feature Extraction S1.3: Continuous Frame Timing Modeling Step S2: Language-enhanced world modeling and scene prediction. Based on BEV features, Gaussianized scene explicit modeling is performed. Scene evolution is driven by a combination of static alignment, dynamic modeling, and a new region completion mechanism. Language priors are introduced to generate world query vectors to achieve future scene prediction and evolution output. This includes the following steps: S2.1: Explicit Gaussian Scene Modeling S2.2: Three-Factor Driven Scene Evolution Modeling S2.3: Language Prior Injection and World Query Generation S2.4: Future Scenario Prediction and Evolution Output Step S3: End-to-end behavioral decision reasoning and trajectory generation. This step uses the semantic information of the current and predicted BEV scenarios to infer driving intentions, generates a sequence of trajectory points that satisfy constraints, and calculates interpretability and confidence information to provide a basis for control execution decisions. This includes the following steps: S3.1: Semantic BEV-driven trajectory intent modeling S3.2: Generation of control trajectory point sequence S3.3: Explainable Trajectory Output and Confidence Assessment Step S4: Robust chassis adaptive control, which parses the trajectory point sequence into low-level control variables, uses hierarchical model predictive control to achieve trajectory tracking and actuator coordination, and performs redundant coordination and fault compensation in case of actuator failure or performance degradation to ensure stable vehicle operation, including the following steps: S4.1: Trajectory Command Parsing and Low-Level Control Variable Mapping S4.2: Intelligent Chassis Layered Model Predictive Control S4.3: Redundant Actuator Coordination and Fault Compensation Step S5: Feedback loop and continuous world model fine-tuning, collecting execution status and perception feedback, evaluating trajectory execution error and making dynamic adjustments, fine-tuning the world model online under safety constraints, and monitoring the health status of the chassis actuators to maintain the long-term reliability of the system, including the following steps: S5.1: Execution Status and Perception Feedback Acquisition S5.2: Error Assessment and Dynamic Adjustment S5.3: Adaptive Learning of Continuous World Models under Security Constraints S5.4: Chassis Actuator Health Status Monitoring The system described in this invention for executing the aforementioned end-to-end autonomous driving control method based on collaborative modeling of a world model and an intelligent chassis, such as... Figure 2 As shown, it includes: a multimodal perception module, a world model modeling and prediction module, a behavior decision and trajectory generation module, a chassis control execution module, and a feedback learning and state monitoring module.

[0062] The multimodal perception module includes a sensor data acquisition unit, a multimodal feature extraction unit, and a temporal BEV construction unit. It is used to acquire multi-source environmental data, including data from cameras, LiDAR, millimeter-wave radar, GPS, etc., to perform data preprocessing and cross-modal feature fusion, and then project this data onto a bird's-eye view to generate a continuous frame temporal BEV semantic feature map.

[0063] The world modeling and prediction module includes: an explicit Gaussian modeling unit, a three-factor evolutionary modeling unit, a language prior injection unit, and a future scene prediction unit. It is used to construct a Gaussian representation of the environment state model based on BEV semantic features, and to model the scene evolution process by combining static alignment, dynamic modeling, and new region completion strategies. Simultaneously, it injects world query conditions through language encoding and outputs future environment prediction results for multiple time steps.

[0064] The behavior decision-making and trajectory generation module includes: an intent reasoning and semantic trajectory modeling unit, a trajectory point generation unit, and a trajectory confidence evaluation unit. It is used to identify driving intentions and generate a sequence of control trajectory points based on the current and predicted BEV scenario, while simultaneously evaluating trajectory confidence to support the reliability judgment of subsequent control execution.

[0065] The chassis control execution module includes a trajectory command parsing unit, a hierarchical model prediction and control unit, and a redundant actuator coordination and compensation unit. It maps trajectory point sequences to specific control quantities, optimizes trajectory tracking through upper-layer MPC, implements coordinated control among multiple actuators at the lower layer, and possesses redundant scheduling and fault-tolerant execution capabilities for abnormal states.

[0066] The feedback learning and status monitoring module includes: an execution status perception unit, an error assessment and model fine-tuning unit, and an actuator health status monitoring unit. It is used to collect chassis execution feedback and external environment perception data, perform online assessment of trajectory errors, continuously update world model parameters under safe conditions, and monitor the actuator operating status in real time to ensure system reliability.

[0067] like Figure 3 As shown, the multimodal sensing module of the present invention includes the following steps: Every time frame of the vehicle's movement The system collects raw observation data from cameras, lidar, millimeter-wave radar, and GPS / IMU modules to form a multi-source sensing dataset:

[0068] in For camera image frames, For 3D LiDAR point clouds, This is millimeter-wave radar echo data (position, radial velocity, signal-to-noise ratio). This is the positioning information (3D position + Euler angles pose).

[0069] Using hardware timestamps and unified reference time Synchronize:

[0070] in This is a linear interpolation time alignment operator. Subsequently, based on the extrinsic parameter matrix... Transform the modal data to the vehicle coordinate system. :

[0071] in The coordinates of the point in the sensor coordinate system. These are the coordinates of a point in the vehicle coordinate system.

[0072] Synchronized and aligned multimodal data are processed by feature extraction operators deal with:

[0073] Specifically, cross-modal fusion employs a weighted attention mechanism:

[0074] in For modal sets, This represents the feature vector extracted by the corresponding modal encoder. To merge query vectors, These are modal bond vectors. Obtained through Softmax normalization. The weights are non-negative and satisfy the following conditions: This is used to weight and fuse features from different modalities according to attention weights, thereby obtaining a unified fused feature. .

[0075] Fusion features Projecting the space onto a bird's-eye view BEV grid coordinates With vehicle coordinate system The mapping relationship is as follows:

[0076] in For raster resolution, The coordinates are the lower left corner of the BEV region. The static spatial feature map is obtained after projection. .

[0077] To characterize dynamic environmental information, a time window with a length of [length missing] is constructed. Time-series BEV sequences:

[0078] will sequence Input timing encoder Perform dynamic feature modeling:

[0079] To highlight the dynamic changes of key time frames during encoding, an attention-based temporal weighting mechanism is introduced. The temporal encoder employs a multi-head self-attention structure to model the BEV feature sequence within the time window. Self-attention calculations explicitly characterize the correlation between different time frames, thereby achieving adaptive weighting of key dynamically changing frames. Its core calculation is as follows:

[0080] in, These are the parameters of the time encoder. Employing a multi-head self-attention-based modeling mechanism, its time modeling can be further expressed as:

[0081] in, , , These are respectively composed of BEV feature sequences The query vector, key vector, and value vector are generated through linear mapping. This refers to the attention dimension. Specifically, Used to initiate feature relevance queries. Used to provide a reference for feature matching. This carries the feature information that needs to be aggregated or updated. After encoding... The sign remains unchanged, but it already contains dynamic feature information enhanced by temporal modeling, which serves as the input to the next module, providing a highly temporally consistent feature foundation for subsequent explicit Gaussian scene modeling and prediction.

[0082] like Figure 4 As shown, the world modeling and prediction module of this invention includes the following steps: First, the time-series BEV feature sequence from the multimodal sensing module is... As input, explicit Gaussian probability modeling is performed on each semantic unit in the scene. Specifically, for time... BEV feature map The geometric and category information of all semantic elements is extracted through an object detection and semantic segmentation network, and the Gaussian parameters are determined using maximum likelihood estimation.

[0083] in, Indicates the first The average position of each scene element in the BEV space Let be the covariance matrix of its location distribution. For semantic category labels, This represents the total number of scene elements in the current frame.

[0084] In obtaining Subsequently, the system dynamically updates itself using a three-factor driven scenario evolution mechanism: Static alignment: Selecting an initial reference frame Using global features as spatial references, rigid body transformations are performed. Will Registered to the reference frame, we obtain This is to reduce sensor drift error and cumulative positioning error.

[0085] Dynamic modeling: computation of time difference And estimate the velocity vector based on the assumption of uniform motion. In order to predict short-term movement trends .

[0086] New Region Completion: For new regions outside the current observation range, interpolation estimation based on the Gaussian distribution of historical neighborhoods is used to generate completion elements. ,in It is obtained by weighting near the Gaussian center. Then, the mean of the in-class covariance is used for initialization to ensure the continuity of the global semantics and geometry of the scene.

[0087] After completing the three-factor update, language priors are introduced. Mapped into semantic vectors by a language encoder Subsequently, Gaussian scene representation Fusion to generate world query vectors:

[0088] in, This represents a multimodal fusion operator, whose function is to combine geometrically distributed... With language semantics Projecting to a unified query space allows for the simultaneous use of physical geometric constraints and high-level semantic priors during the prediction process.

[0089] Finally, using the prediction module by As a conditional vector, and combined with historical time-series BEV feature sequences Recursive prediction of the future Gaussian scene state at each moment:

[0090] in, Indicates the prediction of the first time domain The Gaussian scene set of frames includes location prediction, distribution uncertainty estimation, and possible class transitions. This achieves a conditional prediction closed loop from current semantic understanding to future scene evolution, providing dynamic prior support for subsequent trajectory generation.

[0091] like Figure 5 As shown, the behavior decision-making and trajectory generation module includes the following steps: At every moment of vehicle travel The system receives the future scene state prediction sequence from the previous module.

[0092] and the corresponding time-series BEV feature sequences This serves as input for high-level decision-making reasoning.

[0093] First, the above inputs are fed into a semantic BEV-driven trajectory intent modeling network. The trajectory intent modeling network performs the following steps in the feature encoding stage: and Multi-layer convolution and temporal attention are fused to obtain a scene semantic representation that considers both global and local contexts. This representation is input into a decision classification head, which outputs the driving intent vector for the current frame.

[0094] in, This represents the category of driving strategy inferred under the current scenario conditions, and the confidence level of each strategy can be represented in the form of a probability distribution.

[0095] Next, the trajectory decoupling module A trajectory reasoning method based on conditional generation is used to determine driving intention. Under the constraints of future scenario states, the spatial path and kinematic parameters of the trajectory are decoupled for modeling and joint inference, thereby generating a sequence of control path points that satisfy semantic constraints.

[0096] in, For the trajectory length, Indicates the first The two-dimensional position of each path point in the vehicle coordinate system. For the desired speed, The desired acceleration is determined by a conditional generation mechanism in the trajectory decoupling process, which enables independent optimization of the spatial distribution of path points, velocity curves, and acceleration curves, thereby improving adaptability to different driving strategies.

[0097] Finally, trajectory interpretability analysis and confidence assessment are performed by modules. The implementation involves interpretability analysis, which establishes spatial and semantic consistency between trajectory points and semantic Gaussian elements in the prediction scene, providing a basis for trajectory generation decisions. Confidence assessment, on the other hand, integrates the uncertainty of the world model prediction, the degree of consistency between the trajectory and semantics, and temporal smoothness, outputting a quantitative value of uncertainty for each trajectory point.

[0098] in, Indicates the first Confidence of each trajectory point A higher value indicates a more reliable trajectory point, while a lower value suggests risk-weighted execution or trajectory correction in subsequent control phases.

[0099] This process enables a closed-loop decision-making process that covers the entire chain, from scenario prediction to driving intent reasoning, trajectory generation, and uncertainty quantification, providing interpretable and confident precise trajectory commands for chassis adaptive control.

[0100] like Figure 6 As shown, the main task of the chassis control execution module is to process the trajectory point sequence. With corresponding confidence level It is converted into low-level control commands that can be executed by the vehicle, and coordination and redundancy fault tolerance of multiple actuators are achieved during the execution process.

[0101] First, the trajectory instruction parsing unit receives the trajectory point sequence. The desired position, velocity, and acceleration parameters for each path point are extracted sequentially over time, and these high-level trajectory targets are mapped to low-level control requirements, including steering angle, longitudinal acceleration, and braking force. This process incorporates confidence levels. In the subsequent calculation of the cost function for upper-level model predictive control, the weights of the path tracking term are adaptively adjusted based on the confidence information corresponding to the trajectory points, thereby suppressing the influence of low-confidence path points on control decisions during control optimization.

[0102] In the upper-level control phase, the hierarchical model predictive controller (MPC) uses the analyzed trajectory target as a reference input, employs a vehicle dynamics model to predict state changes over a future period, and solves the control variable sequence through rolling optimization to ensure that the vehicle state closely follows the reference trajectory within the prediction time domain while satisfying safety constraints (such as tire slip limits and braking force limits). This MPC control layer can dynamically update the constraints based on real-time road adhesion coefficients and load changes to ensure trajectory tracking accuracy and driving stability.

[0103] The lower-level actuator coordination unit is responsible for distributing the control quantities obtained from the MPC to specific actuators (such as the front wheel steering system, drive motor, braking system, etc.). Under normal operating conditions, each actuator works collaboratively according to the optimal allocation matrix; when an actuator output deviation is detected... When the preset threshold is exceeded, the redundant actuator coordination and fault compensation mechanism will be triggered to adjust the actuator allocation matrix or switch to the backup actuator in order to maintain vehicle control performance and safety.

[0104] like Figure 7 As shown, the interface consists of four parts. Frame ① displays the vehicle's current operating status, including driving, stopped, braking, accelerating, and starting. Frame ② displays the current system time, vehicle signal status, and the current remaining battery percentage. Frame ③ provides function switching buttons for the feedback learning and status monitoring modules, including two interface entries: "Vehicle Trajectory Prediction" and "Actuator Health Monitoring." Users can click to switch to the corresponding function's visualization display page. Frame ④ displays the visualization results of the selected function interface in Frame ③.

[0105] The "Vehicle Trajectory Prediction" interface displays the calculation results based on the behavior decision and trajectory generation module and the chassis adaptive control module, fusing the predicted scene state with the vehicle dynamics model to generate the future trajectory. When the "Vehicle Trajectory Prediction" button is clicked, this area displays the trajectory prediction results based on the current environmental input, vehicle driving intention, and predicted Gaussian scene state, including the future trajectory. The location, expected velocity, and uncertainty range of each trajectory point are visualized, along with a comparison between the planned and predicted trajectories.

[0106] The “Actuator Health Monitoring” interface displays the operating status of the chassis actuators, including the current output values, output deviations, and health scores of actuators such as steering, drive, and braking. It also highlights a warning when the health score is below a preset threshold, making it easier to detect potential faults and perform maintenance in a timely manner.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An autonomous driving control system based on a world model and intelligent chassis, characterized in that, include: Multimodal perception module, world model modeling and prediction module, behavior decision-making and trajectory generation module, chassis control execution module, and feedback learning and state monitoring module; Multi-source data of the external environment is acquired through a multi-modal perception module, and a time-series feature sequence is constructed based on the multi-source data; The world model modeling and prediction module performs explicit probabilistic modeling based on the temporal feature sequence to obtain scene features; a world query vector is generated based on the scene features; and the future Gaussian scene state is obtained by predicting the world query and the temporal feature sequence through the world model. The behavior decision-making and trajectory generation module generates a sequence of trajectory points based on the future Gaussian scene state, and evaluates the confidence of the trajectory point sequence to obtain the trajectory point confidence. The chassis control execution module generates control commands based on the trajectory point sequence and trajectory point confidence, performs coordinated control based on the control commands, generates a final control vector, and controls the chassis through the final control vector. Feedback data is obtained through the feedback learning and state monitoring module, and the final control vector and world model are adjusted and optimized based on the feedback data.

2. The system according to claim 1, characterized in that, In the multimodal sensing module, the process of constructing the temporal feature sequence includes: The multi-source data is calibrated, wherein the multi-source data includes data collected by visual sensors, lidar, millimeter-wave radar, and global positioning system; Feature extraction and cross-modal alignment are performed on the calibrated data to obtain fused features; The fused features are projected into the bird's-eye view representation space and combined with adjacent frames to generate a temporal feature sequence.

3. The system according to claim 1, characterized in that, In the world model modeling and prediction module, the process of explicitly probabilistically modeling the time-series feature sequences includes: Gaussian distribution is used to model the explicit probability of semantic units corresponding to time-series feature sequences to generate Gaussian scene representations; the Gaussian scene representations include the position mean, covariance matrix, and semantic category labels of the semantic units. The Gaussian scene representation is dynamically updated through a scene evolution mechanism to obtain scene features; the scene evolution mechanism includes static alignment, dynamic modeling, and region completion.

4. The system according to claim 1, characterized in that, In the world modeling and prediction module, the process of obtaining the future Gaussian scene state includes: Obtain the semantic description of the Gaussian scene, encode the semantic description into a semantic vector, and fuse the semantic vector with the scene features to generate a world query vector; The world query vector and the temporal feature sequence are used as input to the world model. The world model is used to recursively predict the future temporal sequence to obtain the future Gaussian scene state.

5. The system according to claim 1, characterized in that, In the behavior decision-making and trajectory generation module, the process of generating trajectory point confidence includes: A semantic BEV-driven trajectory intent modeling network infers high-level driving intent based on the future Gaussian scene state and temporal feature sequence. Based on the high-level driving intention and the future Gaussian scene state, a trajectory point sequence is generated, wherein the trajectory point sequence includes the position of the trajectory point, the expected velocity, and the expected acceleration.

6. The system according to claim 1, characterized in that, In the chassis control execution module, the process of generating the final control vector includes: Trajectory analysis is performed based on the trajectory point sequence and trajectory point confidence to generate control commands. These commands are then coordinated and controlled by an intelligent chassis hierarchical model predictive controller to generate the final control vector. Specifically, within the intelligent chassis hierarchical model predictive controller, a corresponding cost function is calculated based on the error of the trajectory point sequence and the control commands. The control commands are optimized with the goal of minimizing the cost function, and the optimized control commands are adjusted to generate the final control vector.

7. The system according to claim 1, characterized in that, In the feedback learning and state monitoring module, the error between the final control command execution and the actual execution is calculated based on the feedback data. The error in the control command execution is used to adjust the final control command or replan the trajectory. The feedback data includes the vehicle's actual steering angle, acceleration, braking force, and real-world scenario state. The world model is adjusted and optimized based on the real-world state and the future Gaussian state.

8. An autonomous driving control method based on a world model and intelligent chassis, characterized in that, include: Acquire multi-source data of the external environment, and construct a time-series feature sequence based on the multi-source data; Explicit probabilistic modeling is performed based on the temporal feature sequence to obtain scene features; A world query vector is generated based on the scene features, and the future Gaussian scene state is predicted by the world model based on the world query and the temporal feature sequence. Based on the future Gaussian scene state, a trajectory point sequence is generated, and the confidence of the trajectory point sequence is evaluated to obtain the trajectory point confidence. Control commands are generated based on the trajectory point sequence and trajectory point confidence, coordinated control is performed based on the control commands, a final control vector is generated, and the chassis is controlled through the final control vector; Obtain feedback data, evaluate it based on the feedback data, and adjust and optimize the final control vector and world model.

Images