Driving control system and terrain-aware vehicle cooperative control method

By employing a vehicle cooperative control method based on terrain perception and multi-objective game equilibrium, the problems of insufficient control basis and lag response of driving control systems under complex road conditions are solved, achieving coordinated optimization of driving efficiency, comfort and stability.

CN123104640APending Publication Date: 2026-11-17ZHEJIANG GEELY HLDG GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611405826.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing vehicle driving control systems struggle to simultaneously balance driving efficiency, ride comfort, and vehicle stability under complex road conditions. Insufficient control data, delayed response, and independent decision-making by each controller lead to conflicting objectives.

Method used

By using a terrain-aware vehicle cooperative control method, road condition physical attributes are determined using multi-source perception data, multiple candidate joint control sequences are generated, a causal driving world model is invoked to predict future states, and a multi-objective game equilibrium strategy is adopted to select a cooperative control scheme, driving multiple execution units to work collaboratively.

Benefits of technology

To achieve forward-looking, multi-objective balanced vehicle driving control under complex road conditions, improve driving smoothness, stability and overall driving quality, and avoid response lag and objective conflict.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123104640A_ABST
    Figure CN123104640A_ABST
Patent Text Reader

Abstract

This invention provides a driving control system and a vehicle cooperative control method based on terrain perception. The method is applied to a vehicle driving control system and includes: determining state feature representations based on vehicle driving state data and determining environmental feature representations of the current driving scenario based on multi-source perception data, wherein the environmental feature representations include road condition physical attributes; generating multiple sets of candidate joint control sequences, each set including control commands corresponding to multiple execution units; invoking a causal driving world model to predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representations and environmental feature representations; evaluating the comprehensive impact of each set of candidate sequences on multiple game actors based on the future state evolution results using a multi-objective game equilibrium strategy to determine a target joint control sequence; and driving multiple execution units according to the control commands in the target joint control sequence to control vehicle driving.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of vehicle intelligent control technology, specifically to a driving control system and a vehicle cooperative control method based on terrain perception. Background Technology

[0002] With the development of intelligent driving technology, vehicle driving control systems typically need to adjust vehicle speed, body posture, and driving stability under different road conditions. Common vehicle driving control systems can include intelligent cruise control systems, suspension control systems, and vehicle stability control systems. Intelligent cruise control systems are mainly used to adjust the vehicle's drive and braking based on traffic targets ahead, maintaining a set speed or following the vehicle in front. Suspension control systems are mainly used to adjust suspension damping or vehicle height based on vehicle excitation. Vehicle stability control systems primarily perform stability control when the vehicle's posture, yaw rate, or lateral state meets preset intervention conditions.

[0003] In a typical intelligent cruise control implementation, vehicles typically use sensors such as millimeter-wave radar and forward-facing cameras to identify structured traffic targets such as vehicles ahead and lane markings. The controller constructs a longitudinal control target based on the target distance, relative speed, and the vehicle's own speed, and further adjusts the vehicle speed through the drive or braking system. However, when the vehicle encounters complex road conditions such as speed bumps, potholes, gravel, mud, undulating surfaces, or continuous slopes, the aforementioned structured traffic target identification methods often struggle to accurately obtain the physical properties of road obstacles, such as their geometric dimensions, material hardness, adhesion conditions, and impact transmission characteristics. Furthermore, the longitudinal control cost function primarily reflects the current following relationship and longitudinal speed adjustment requirements, failing to fully reflect the impact of road surface physical properties on the vehicle's response under different driving conditions. Therefore, in complex road conditions, relying solely on structured target identification and longitudinal error adjustment often fails to determine the potential body excitation, chassis impact, or attitude changes that may occur when the vehicle traverses the road in its current state, easily leading to a decrease in ride comfort and driving stability.

[0004] In suspension control, a typical approach involves acquiring the vehicle's excitation state using sensors such as vehicle height, vertical acceleration, and suspension displacement, and then adjusting the suspension damping or vehicle height according to preset rules. Because this type of control typically uses already occurred vehicle excitation, suspension displacement, or height changes as input, the suspension system often only adjusts after the wheels have contacted road obstacles or the vehicle has produced a noticeable response. For suspension height or damping changes that require a longer adjustment time, this control method based on the current local state is prone to response lag.

[0005] Furthermore, intelligent cruise control, suspension control, and vehicle stability control typically operate around different control objectives. Cruise control focuses more on traffic efficiency and following distance, suspension control focuses more on vehicle vibration and ride comfort, and vehicle stability control focuses more on posture safety and instability risk. Because each controller usually makes independent decisions based on local sensor information and fixed calibration parameters, longitudinal speed adjustment, suspension adjustment, and stability limiting can easily interfere with each other when the vehicle traverses complex road conditions. For example, improving traffic efficiency may increase the vehicle's response to road excitation, while reducing speed, although beneficial for reducing impact, may affect traffic efficiency; and suspension adjustments, while potentially improving comfort, may also affect posture stability.

[0006] Therefore, the aforementioned technologies often struggle to simultaneously achieve driving efficiency, ride comfort, and vehicle stability under complex road conditions. How to achieve coordinated control of multiple driving quality-related objectives under complex road conditions is a pressing technical problem that needs to be solved. Summary of the Invention

[0007] To overcome the aforementioned shortcomings of existing technologies, this invention provides a driving control system and a terrain-aware vehicle cooperative control method to address the problem in existing technologies of difficulty in coordinating the control of multiple driving quality-related objectives under complex road conditions. Specifically, this invention is achieved through the following technical solution: In a first aspect, the present invention provides a vehicle cooperative control method based on terrain perception, applied to a vehicle driving control system, comprising: The vehicle's state characteristics are determined based on the vehicle's driving state data, and the environmental characteristics of the current driving scenario are determined based on the vehicle's multi-source perception data. The environmental characteristics include the physical attributes of the road conditions in the current driving scenario. Multiple sets of candidate joint control sequences are generated, and each set of candidate joint control sequences includes control instructions corresponding to multiple execution units of the vehicle. The causal driving world model is invoked to predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representation and the environmental feature representation. By employing a multi-objective game equilibrium strategy, the comprehensive impact of each candidate joint control sequence on multiple game agents related to driving quality is evaluated based on the future state evolution results, so as to determine the target joint control sequence that satisfies the game equilibrium condition. The plurality of execution units are driven according to the control instructions in the target joint control sequence to control the vehicle's movement.

[0008] Secondly, the present invention provides a vehicle driving control system, comprising: The feature determination unit is used to determine the state feature representation of the vehicle based on the vehicle's driving state data, and to determine the environmental feature representation of the current driving scenario based on the vehicle's multi-source perception data, wherein the environmental feature representation includes the road condition physical attributes under the current driving scenario. A sequence generation unit is used to generate multiple sets of candidate joint control sequences, each set of candidate joint control sequences including control instructions corresponding to multiple execution units of the vehicle; The state prediction unit is used to invoke the causal driving world model and predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representation and the environmental feature representation. A multi-objective game unit is used to evaluate the comprehensive impact of each candidate joint control sequence on multiple game subjects related to driving quality based on the future state evolution results through a multi-objective game equilibrium strategy, so as to determine the target joint control sequence that satisfies the game equilibrium condition. The collaborative control unit is used to drive the plurality of execution units respectively according to the control instructions in the target joint control sequence, so as to control the vehicle driving.

[0009] Thirdly, the present invention also provides a vehicle having a driving control system deployed thereon, the system being used to implement the method described in the first aspect above.

[0010] Fourthly, the present invention also provides an electronic device, including a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; the memory stores machine-readable instructions, and the processor implements the method described in the first aspect above by invoking the machine-readable instructions.

[0011] Fifthly, the present invention also provides a machine-readable storage medium storing machine-readable instructions, which, when invoked and executed by a processor, implement the method described in the first aspect above.

[0012] The novel solution proposed in the foregoing embodiments of this invention incorporates road condition physical attributes into vehicle driving decisions, generates control commands from multiple execution units as a unified sequence, predicts the future impact of different control schemes through a causal driving world model, and selects a cooperative control scheme using a game-theoretic equilibrium approach. This achieves forward-looking, multi-objective balance, and multi-execution unit collaborative vehicle driving control under complex road conditions, improving driving smoothness, stability, and overall driving quality. Specifically: This solution determines environmental feature representations based on multi-source perception data, including the physical attributes of road conditions in the current driving scenario. This expands the control basis beyond structured traffic targets such as the vehicle ahead and lane lines, incorporating physical information about complex road surfaces like speed bumps, potholes, gravel, mud, and undulating surfaces. Consequently, the driving control system can identify the sources of influence of road conditions on vehicle response, providing an input basis for judging vehicle excitation, chassis impact, and attitude changes, thus alleviating the difficulty of obtaining road geometry, material, adhesion, and impact transmission characteristics in structured target recognition. Furthermore, multiple candidate joint control sequences, each including control commands corresponding to multiple execution units of the vehicle, mean that the decision object is not a single longitudinal speed control, but a combination of actions of multiple execution units in the future control process. This incorporates previously dispersed control quantities into the same candidate space, providing a prerequisite for unified comparison and coordinated execution.

[0013] Furthermore, this solution utilizes a causal driving world model to predict the future state evolution of each candidate joint control sequence based on state and environmental feature representations. Since there are response delays in actions such as suspension height and damping, and the impact of complex road conditions often only becomes fully apparent when the vehicle approaches or passes obstacles, predicting future state evolution allows the system to compare the vehicle responses that different control sequences might cause before execution, thus mitigating the problem of suspension control relying on pre-existing vehicle excitation and adjustment lag in the prior art. Finally, a multi-objective game equilibrium strategy is used to evaluate the comprehensive impact of each candidate sequence on multiple game actors, and a target joint control sequence that satisfies the game equilibrium condition is selected. Multiple execution units are then driven according to this sequence. Because multiple driving quality-related objectives are incorporated into the same decision-making framework for prediction, evaluation, and checks and balances, conflicts caused by independent operation of controllers based on local objectives and fixed parameters can be avoided, achieving a dynamic balance between driving efficiency, ride comfort, and vehicle stability.

[0014] It is evident that the novel technical solution proposed in this invention, through the technical chain of "road condition physical attribute perception - joint control sequence generation - causal prediction - game equilibrium - multi-execution unit collaborative driving", can effectively solve the problems of insufficient control basis, response lag and target conflict in complex road conditions in the background technology, and the improvement effect is significant. Attached Figure Description

[0015] To more clearly illustrate the technical solution of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the hardware architecture of a driving control system according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the software architecture of a driving control system according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating a vehicle cooperative control method based on terrain perception, as shown in an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating a collaborative control process based on multi-objective game equilibrium, as shown in an embodiment of the present invention. Figure 5 This is a flowchart illustrating another vehicle cooperative control method based on terrain perception, as shown in an embodiment of the present invention. Figure 6 This is a schematic structural diagram of an electronic device according to an embodiment of the present invention; Figure 7 This is a block diagram of a vehicle cooperative control device based on terrain perception, as shown in an embodiment of the present invention. Detailed Implementation

[0017] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present invention.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0020] The embodiments of the collaborative control method of the present invention will now be described in detail with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the hardware architecture of a driving control system according to an embodiment of the present invention. Figure 1 As shown, from a hardware perspective, the system can include only vehicle 11, or it can include both vehicle 11 and a cloud-based model runtime server 12 (i.e., a "cloud-edge collaborative" or "vehicle-road-cloud integrated" architecture). From a software perspective, the driving control system can include the driving control system deployed in vehicle 11, and can also include a cloud service module deployed in model runtime server 12 for interacting with the driving control system. Specifically, the driving control system of vehicle 11 can be deployed in a domain controller 111 (such as an intelligent driving domain controller ADCU or an intelligent chassis domain controller ACDC) installed in vehicle 11. It is understood that the driving control system is an in-vehicle system, which can operate in conjunction with the aforementioned cloud service module or operate independently offline. If the driving control system includes model runtime server 12 or there is a need to interact with model runtime server 12, vehicle 11 can establish a network connection with model runtime server 12 through a wireless communication module to interact with the server.

[0022] In addition to the domain controller 111, the vehicle 11 may also be equipped with various sensors, such as environmental sensing devices for collecting data on the external environment and behavior sensing devices for detecting the driver's behavior inside the vehicle. This application embodiment does not limit the number of these devices or their installation location. For example, the environmental sensing devices may include image acquisition devices (such as front-view cameras, side-view cameras, rear-view cameras, etc.), ranging radars (such as LiDAR, millimeter-wave radar, ultrasonic radar, etc.); the behavior sensing devices may include cameras installed above the driver's seat to detect the driver's eye direction (such as visible light cameras, infrared cameras, etc.), and / or torque sensors, pressure sensors, etc., installed on the steering wheel to collect the driver's hand movements (this sensor is used to implement the hands-off detection (HOD) function). The aforementioned behavior sensing devices and corresponding control devices can constitute the vehicle's DMS system (Driver Monitoring System).

[0023] Furthermore, this solution may also involve a cloud-based model management server 13. For example, the model management server 13 can deploy a fully functional, large-scale driving control system (hereinafter referred to as a driving control system copy), while the driving control system deployed locally in the vehicle can be a lightweight system with smaller parameter scale obtained by lightweighting the driving control system copy. In this regard, after running the cooperative control scheme described in this invention, the vehicle 11 can upload the corresponding experience data packaged and uploaded to the model management server 13 via network 10, so that the latter can iteratively train the cloud-deployed driving control system copy based on the data, realizing continuous learning and evolution of the cloud model. Further, the model management server 13 can generate a corresponding OTA (Over-The-Air) upgrade package based on the iterated driving control system copy and download it to the vehicle (including vehicle 11), so that vehicle 11 can complete the upgrade and update of its local driving control system based on the upgrade package.

[0024] Furthermore, the vehicles described in this application (such as vehicle 11) can be pickup trucks, sedans, SUVs (Sport Utility Vehicles), motorhomes, trucks, etc., in terms of their functional form; and can be fuel vehicles or new energy vehicles (such as hybrid vehicles, pure electric vehicles, hydrogen fuel cell vehicles, methanol fuel cell vehicles, etc.) in terms of their power form. This invention does not limit the specific form of the vehicles described.

[0025] Figure 2 This is a schematic diagram of the software architecture of a driving control system according to an embodiment of the present invention. Figure 2 As shown, from a software architecture perspective, the driving control system can include a multimodal perception large model layer, a decision-making and game theory deduction layer, a control and actuator layer, and an edge-cloud layer. This architecture, based on information flow and control flow, can be divided into a data access layer, a feature representation layer, a candidate control sequence generation layer, a causal driving world model deduction layer, a multi-objective game theory decision-making layer, a collaborative control layer, and an execution and feedback layer. In the diagram, each box represents a software functional module, data processing unit, or control interface, and arrows indicate data flow, model call relationships, or control command issuance relationships. The overall connection relationship between each layer is as follows: the data access layer inputs the vehicle's own state and the external driving scenario into the feature representation layer; the feature representation layer outputs state feature representations and environmental feature representations; the candidate control sequence generation layer generates multiple sets of candidate joint control sequences; the causal driving world model deduction layer predicts the future state evolution results of each set of candidate sequences; the multi-objective game theory decision-making layer selects the target joint control sequence; the collaborative control layer drives multiple execution units; and the execution results are then returned to the data access layer through the feedback layer, forming a closed loop. Wherein: The data access layer includes a driving state data acquisition module and a multi-source perception data acquisition module. The driving state data acquisition module acquires the vehicle's current driving state data. This data can include vehicle speed, longitudinal acceleration, vertical acceleration, vehicle attitude, suspension status, wheel speed, etc., reflecting the vehicle's own motion state and chassis status. The multi-source perception data acquisition module acquires multi-source perception data for the current driving scenario. This data can come from perception devices such as cameras, LiDAR, and millimeter-wave radar, reflecting the road environment in front of and around the vehicle. The outputs of both modules are connected to the feature representation layer. The role of the data access layer is to complete the acquisition of raw data, providing input for subsequent unified feature representation.

[0026] The core block of the feature representation layer is the feature determination unit. This unit can be further divided into a state feature representation submodule and an environmental feature representation submodule. The state feature representation submodule determines the vehicle's state features based on its driving state data, converting information such as the vehicle's motion state and chassis state into structured features that can be processed by the subsequent prediction model. The environmental feature representation submodule determines the environmental features of the current driving scenario based on the vehicle's multi-source perception data. This environmental feature representation includes the physical attributes of the road conditions in the current driving scenario. In other words, this module not only processes general driving scenario information but also incorporates the physical attributes of the road surface that affect vehicle movement into the feature expression. The feature representation layer connects to the candidate control sequence generation layer, providing scenario and state basis for the generation of candidate control sequences; it also directly connects to the causal driving world model inference layer, providing feature input for future state prediction. The technical function of this layer is to uniformly transform the raw sensor data and vehicle state data into the feature foundation required for subsequent inference and decision-making.

[0027] The candidate control sequence generation layer includes a sequence generation unit, which generates multiple sets of candidate joint control sequences. Each set of candidate joint control sequences includes control commands corresponding to multiple actuators of the vehicle. Instead of generating single-point control quantities for a single actuator, this layer generates combined control schemes for multiple actuators. This layer receives state feature representations and environmental feature representations from the feature representation layer and outputs the generated multiple sets of candidate joint control sequences to the causal driving world model inference layer. Its function is to construct a comparable joint control search space, enabling subsequent decisions to predict and select among multiple control paths.

[0028] The causal driving world model deduction layer includes a state prediction unit, whose core block is the causal driving world model. The state prediction unit invokes the causal driving world model, predicting the future state evolution results corresponding to multiple sets of candidate joint control sequences based on state feature representations and environmental feature representations. The input to this layer consists of two parts: one part is the state feature representation and environmental feature representation output from the feature representation layer, and the other part is the multiple sets of candidate joint control sequences output from the sequence generation unit. The role of the causal driving world model is to infer the potential future vehicle state changes caused by each set of candidate control sequences before the actual issuance of control commands. The future state evolution results output by this layer are connected to the multi-objective game decision layer. Through this layer, the software architecture extends control decision-making from an immediate response based on the current state to a forward-looking assessment of future state changes.

[0029] The multi-objective game decision layer comprises multi-objective game units. Each unit may further include a comprehensive impact assessment module and an equilibrium solution module. The comprehensive impact assessment module evaluates the combined impact of each candidate joint control sequence on multiple game actors related to driving quality, based on the future state evolution results. The equilibrium solution module determines the joint control sequence that satisfies the game equilibrium conditions based on the assessment results. This layer receives the future state evolution results output by the causal driving world model deduction layer and outputs the selected joint control sequence to the collaborative control layer. Its technical function is to incorporate multiple driving quality-related objectives into the same decision framework, balancing and selecting among multiple candidate control schemes to avoid the final control outcome being dominated by a single objective or a single subsystem objective.

[0030] The collaborative control layer includes a collaborative control unit, which drives multiple execution units according to the control instructions in the target joint control sequence. Since the target joint control sequence itself contains control instructions corresponding to each execution unit, the collaborative control unit can distribute the control instructions from different execution units according to a unified timing and a unified control objective. This layer receives the target joint control sequence output by the multi-objective game decision layer and connects multiple execution units in the execution and feedback layers. Its function is to enable multiple execution units to coordinate their actions around the same target joint control sequence, rather than responding independently to local control objectives.

[0031] The execution and feedback layer comprises multiple execution units and a state feedback module. The execution units drive the vehicle according to control commands issued by the cooperative control unit. The state feedback module collects the vehicle's actual driving state after executing control commands and returns this state to the data access layer as input for the next round of state feature representation, candidate sequence generation, future state prediction, and game-theoretic decision-making. This layer enables... Figure 2The software architecture shown forms a closed-loop operating relationship. Changes in the actual driving state of the vehicle will continuously affect the feature representation and control decisions in the next cycle, thereby improving the system's continuous adaptability under dynamic road conditions.

[0032] Based on the data and control relationships at each layer, the feature determination unit determines the vehicle's state characteristics based on its driving state data and the environmental characteristics of the current driving scenario based on the vehicle's multi-source perception data. These environmental characteristics include the physical attributes of the road conditions in the current driving scenario. The sequence generation unit generates multiple candidate joint control sequences, each containing control commands corresponding to multiple execution units of the vehicle. The state prediction unit invokes a causal driving world model to predict the future state evolution results corresponding to the multiple candidate joint control sequences based on the state and environmental characteristics. The multi-objective game unit evaluates the comprehensive impact of each candidate joint control sequence on multiple game actors related to driving quality based on the future state evolution results using a multi-objective game equilibrium strategy, thereby determining the target joint control sequence that satisfies the game equilibrium conditions. The cooperative control unit drives multiple execution units according to the control commands in the target joint control sequence to control vehicle movement.

[0033] visible, Figure 2 The software architecture shown integrates road condition physical attribute perception, candidate joint control sequence generation, future state evolution prediction, multi-objective game equilibrium decision-making, and collaborative driving of multiple execution units into a unified control link through sequential connections and closed-loop feedback between layers, thereby achieving collaborative control of vehicle driving quality-related objectives under complex road conditions.

[0034] Corresponding to the foregoing embodiments, the present invention proposes an in-vehicle driving control system, which includes multiple functional units. Among them, The feature determination unit is used to determine the state feature representation of the vehicle based on the vehicle's driving state data, and to determine the environmental feature representation of the current driving scenario based on the vehicle's multi-source perception data, wherein the environmental feature representation includes the road condition physical attributes under the current driving scenario. A sequence generation unit is used to generate multiple sets of candidate joint control sequences, each set of candidate joint control sequences including control instructions corresponding to multiple execution units of the vehicle; The state prediction unit is used to invoke the causal driving world model and predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representation and the environmental feature representation. A multi-objective game unit is used to evaluate the comprehensive impact of each candidate joint control sequence on multiple game subjects related to driving quality based on the future state evolution results through a multi-objective game equilibrium strategy, so as to determine the target joint control sequence that satisfies the game equilibrium condition. The collaborative control unit is used to drive the plurality of execution units respectively according to the control instructions in the target joint control sequence, so as to control the vehicle driving.

[0035] The specific functional logic of each of the above functional units can be found in the following embodiments, and will not be repeated here.

[0036] Figure 3 This is a flowchart illustrating a vehicle cooperative control method based on terrain perception, as shown in an embodiment of the present invention. The method is applied to the vehicle driving control system and includes the following steps 302-310.

[0037] Step 302: Determine the state feature representation of the vehicle based on the vehicle's driving state data, and determine the environmental feature representation of the current driving scenario based on the vehicle's multi-source perception data. The environmental feature representation includes the physical attributes of the road conditions in the current driving scenario.

[0038] This step, by performing feature processing on the vehicle's driving status data and multi-source perception data, can uniformly express the vehicle's own motion state and the physical attributes of the road conditions in the current driving scenario as structured features, providing an input basis for subsequent generation of candidate joint control sequences and calling the causal driving world model to predict future states.

[0039] The driving state data refers to data reflecting the current motion and chassis status of the vehicle. This data can be acquired using hardware such as vehicle speed sensors, inertial measurement units, vehicle height sensors, suspension displacement sensors, and wheel speed sensors. Specifically, it can include vehicle speed, longitudinal acceleration, vertical acceleration, yaw rate, vehicle attitude angles (pitch angle, roll angle), suspension height, suspension displacement, suspension damping, steering wheel angle, throttle opening, braking pressure, and wheel speed. The state feature representation refers to the features obtained after encoding, normalizing, fusing, or structuring the driving state data. This can be implemented using vector, tensor, time series features, or neural network encoding results.

[0040] The multi-source perception data refers to data from various sensing devices on the vehicle, including sensors such as LiDAR, cameras, millimeter-wave radar, and ultrasonic sensors. For example, LiDAR can be used to generate 3D point clouds, high-definition cameras can be used to acquire road surface images and semantic information, and millimeter-wave radar can be used to detect targets ahead. The environmental feature representation refers to features obtained based on the multi-source perception data to characterize the current driving scenario. These features can take the form of physical cognitive feature maps, 3D grid features, semantic maps, road surface attribute maps, or obstacle object attribute lists. The road condition physical attributes refer to the physical characteristics of the road surface that may affect the vehicle's driving response, such as road surface geometry, obstacle height, pothole depth, material hardness, friction coefficient, moisture content, slope, roughness, and impact transmission characteristics. Specifically, the physical properties of the road conditions can be divided into: geometric dimension parameters (such as the height, width, length, slope, radius of curvature, etc. of obstacles), surface material parameters (such as material types and corresponding friction coefficients of asphalt, concrete, gravel, soil, ice and snow, water accumulation, etc.), adhesion condition parameters (such as road surface adhesion coefficient, slip ratio, etc.), and impact transmission characteristic parameters (such as road surface stiffness, damping, energy absorption characteristics, etc.).

[0041] The state feature representation output in this step can reflect the current motion state of the vehicle, and the environmental feature representation can reflect the possible impact of the road surface ahead on the vehicle. The state feature representation and the environmental feature representation together provide feature basis for the subsequent step 304 to generate candidate joint control sequences and step 306 to predict future state evolution results.

[0042] During vehicle operation, the physical properties of road conditions may vary spatially at different road locations. To facilitate precise processing of specific road locations for subsequent prediction and game-theoretic decision-making, a physical cognitive feature map mechanism can be employed to structurally represent the physical properties of road conditions according to the location of the road ahead. In one embodiment, the environmental feature representation may include a physical cognitive feature map, which characterizes the distribution of physical properties at various road locations ahead of the vehicle. The physical cognitive feature map refers to a structured feature map indexed by spatial location and carrying the physical properties of the road surface. It can be a two-dimensional feature map with the spatial coordinates of the road surface ahead of the vehicle as the horizontal axis and the corresponding physical property values ​​as the vertical axis, or a three-dimensional feature map presented in BEV (Bird's Eye View) format, where each pixel or voxel corresponds to a road location and stores one or more physical property values ​​for that location. For example, the physical cognitive feature map can be a bird's-eye view feature, a three-dimensional raster feature, a point cloud voxel feature, or an image and point cloud fusion feature, etc. The physical property distribution refers to the spatial distribution of geometric, material, adhesion, or impact-related properties corresponding to different road surface locations. It can be generated by collecting data through lidar, cameras, millimeter-wave radar, and then through multimodal encoders, semantic-physical decoders, rasterization modules, or attribute mapping rules.

[0043] Taking a speed bump approximately 10 centimeters high located 50 meters in front of the vehicle as an example, in this scenario, LiDAR and cameras can identify the location, height, width, and material hardness of the speed bump on the road surface. A physical cognitive feature map can then be generated to represent the corresponding physical attributes at that location. Subsequently, a causal driving world model can predict the state evolution at different vehicle speeds and attitudes based on these location attributes. The multi-objective game unit can also select a joint control sequence based on this, such as deceleration, adjusting suspension height, or adjusting damping.

[0044] This embodiment allows environmental feature representation to be refined from an overall scene description to a physical attribute expression organized by road surface location. On the one hand, it helps to distinguish the physical differences between different locations such as speed bumps, potholes, gravel, and ramps; on the other hand, it can provide more specific spatial input for future state prediction, further contributing to improving the pertinence of control decisions, driving smoothness, and vehicle stability under complex road conditions.

[0045] Of course, besides physical cognitive feature maps, a list of road surface object attributes can also be used to represent the road surface. The location, size, material, and impact attributes of each road obstacle can be treated as object entries for subsequent prediction and decision-making. This approach reduces the amount of data processing in irrelevant areas and facilitates targeted control of discrete obstacles. Alternatively, a two-dimensional grid physical attribute map or a three-dimensional occupancy grid map can be used, dividing the road surface ahead into multiple grids and filling in the corresponding physical attributes. This approach can represent the spatial changes of continuous road surfaces and helps the world model extrapolate vehicle responses based on location.

[0046] While physical cognitive feature maps can express the static properties of a road surface, the impact at the same road location can differ under different vehicle speeds and vehicle postures. In one embodiment, an energy impact potential field can be used as the physical cognitive feature map. This energy impact potential field characterizes the energy distribution of the impact generated on the vehicle at a specific location on the road surface. The value of the energy impact potential field can vary with the vehicle's speed and / or posture when passing through the specific location on the road surface. Specifically, the energy impact potential field refers to the risk distribution of impact energy that the vehicle may experience when passing through a road surface location, indexed by that location. It can be implemented using vehicle dynamics models, suspension models, neural network regression models, or simulation lookup tables. The energy distribution of the impact can be determined based on parameters such as road surface geometry, material hardness, vehicle speed, pitch angle, and roll angle. Generally, for the same road surface location, the higher the vehicle speed or the more tilted the vehicle posture, the larger the value of the energy impact potential field can be.

[0047] Taking a speed bump approximately 10 centimeters high located 50 meters in front of the vehicle as an example, the energy impact potential field can be used to calculate the impact energy value at that location based on the speed bump's position, height, material, current vehicle speed, and vehicle attitude. If the vehicle passes over it at 80 km / h, the potential field value can be higher; if it passes at 15 km / h, the potential field value can be lower; and if the pitch or roll angle is large, the potential field value can also increase. Subsequent world models and game units can then use this data to compare different candidate joint control sequences.

[0048] As can be seen, the energy impact potential field can transform static road surface properties into a dynamic impact risk expression related to vehicle speed and attitude. This not only helps quantify the impact differences at the same road surface location under different passage conditions, but also provides a basis for suitability evaluation; a larger potential field value indicates greater potential vehicle vibration and impact. Furthermore, this expression can provide risk input for prediction and game theory decision-making, helping to balance traffic efficiency, ride comfort, and vehicle stability.

[0049] Alternatively, an impact risk level map indexed by vehicle speed and attitude can be used to pre-store the impact levels for different speed and attitude ranges. This approach reduces online computation and facilitates the screening of high-risk passage conditions. Alternatively, a road surface object impact risk curve can be used to associate obstacle attributes with impact energy values ​​at different vehicle speeds and attitudes. This approach enables precise evaluation of discrete obstacles and facilitates comparison of candidate control sequences.

[0050] In one embodiment, the energy impact potential field can be determined based on multi-source sensing data of the vehicle using any of the following methods.

[0051] Method 1: Based on the multi-source sensing data, determine the physical properties of a specific location on the road surface. Input these physical properties into a multimodal sensing model for inference, and obtain the energy impact potential field output by the multimodal sensing model. The multimodal sensing model can be a trained deep learning model (such as a large multimodal model), whose input is the physical properties of the road surface (such as geometric dimensions, material, friction coefficient, etc.), and whose output is the corresponding energy impact potential field. The multimodal sensing model refers to a deep learning model that, after training, can infer the impact risk distribution from the physical properties of the road surface. The input of this model can include physical properties such as geometric dimensions, obstacle height, pothole depth, material hardness, friction coefficient, moisture content, and roughness at a specific location on the road surface, and can also be further combined with state information such as the vehicle's current speed, vehicle attitude, and suspension height. The output of this model can be the energy impact potential field, i.e., the impact energy distribution that may occur at different road surface locations under different passage conditions. In specific implementations, the multimodal sensing model can employ a large multimodal model, a Transformer model, a convolutional neural network, a point cloud network, an image and point cloud fusion network, or other deep learning models. Training samples can come from real-world road data, suspension vibration data, vehicle acceleration data, high-fidelity simulation data, or historical experience data. The model learns the mapping relationship between road surface properties and vehicle impact response, and can directly output an energy impact potential field.

[0052] The advantage of this approach is that the generation of the energy impact potential field can be completed end-to-end by the model, enabling it to express the nonlinear relationships between road geometry, material, adhesion conditions, and vehicle state. For complex or unstructured road surfaces such as speed bumps, potholes, gravel, and mud, this approach can learn the impact patterns that are difficult to fully describe with simple formulas through training data, thereby helping to improve adaptability and inference efficiency in complex scenarios.

[0053] Method 2: Based on the multi-source sensing data, determine the physical properties of a specific location on the road surface. Input these physical properties into a multimodal sensing model for inference, obtain an initial potential field estimate, and then call a physical calculation model to correct the initial potential field estimate based on the vehicle's dynamic parameters to obtain the energy impact potential field. The physical calculation model can calculate the actual impact energy of the vehicle passing through the road surface location based on dynamic parameters such as vehicle mass, suspension stiffness, tire stiffness, and damping coefficient, according to mechanical formulas, thereby correcting the initial potential field estimate output by the multimodal sensing model. The physical calculation model refers to a model that calculates the impact response of the vehicle when passing through the road surface location based on vehicle dynamics. This model can calculate the vertical acceleration, suspension travel, wheel end impact, or impact energy that the vehicle may generate when passing through a specific road surface location based on dynamic parameters such as vehicle mass, load distribution, suspension stiffness, damping coefficient, tire stiffness, unsprung mass, wheelbase, center of gravity height, current vehicle height, pitch angle, and roll angle. In this approach, the multimodal perception model first outputs an initial potential field estimate based on the road surface's physical properties. This initial potential field estimate reflects the approximate impact risk distribution at the road surface location. Subsequently, the physical calculation model can combine current vehicle dynamics parameters to calculate an impact energy that better reflects the vehicle's actual mechanical state, and use this energy to correct the initial potential field estimate. Correction methods can include proportional correction, residual correction, weighted fusion, Kalman filtering, Bayesian fusion, or physical constraint optimization, etc.

[0054] The advantage of this approach is that it retains the ability of multimodal perception models to represent complex road surface properties, while also enhancing the consistency between the energy impact potential field and the actual dynamic state of the vehicle through physical calculation models. When vehicle load, tire condition, suspension height, or damping condition changes, the physical calculation model can make targeted corrections to the initial potential field estimate, thereby helping to improve the reliability, interpretability, and vehicle-specific adaptability of the potential field values.

[0055] Method 3: Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined. These physical properties are then input into a physical calculation model, which calculates the energy impact potential field based on the physical properties and the vehicle's dynamic parameters. This method is entirely based on physical model calculations and does not rely on the inference of the multi-modal sensing model. In this method, multi-source sensing data can still be used to determine the physical properties of a specific location on the road surface, but the generation of the energy impact potential field mainly depends on the physical calculation model. The physical calculation model can directly calculate the impact energy distribution when the vehicle passes through that location on the road surface, based on physical properties such as road geometry, material hardness, friction coefficient, obstacle height, or pothole depth, combined with dynamic parameters such as vehicle mass, suspension stiffness, damping coefficient, and tire stiffness.

[0056] The advantage of this approach lies in the fact that the generation process of the energy impact potential field has a relatively clear physical meaning, and the calculation results can be directly linked to vehicle dynamics parameters. Because this approach does not heavily rely on large-scale training samples, it can still generate the energy impact potential field based on mechanical relationships even when adapting to new vehicle models, experiencing changes in actuator parameters, dealing with special road conditions, or when training data is insufficient. Furthermore, this approach facilitates subsequent interpretation and verification of indicators such as impact energy, suspension travel, and vehicle acceleration, thus improving the verifiability of the solution.

[0057] Besides the three methods mentioned above, other methods can also be used to determine the energy impact potential field. For example, a combination of table lookup and interpolation can be used. For instance, impact energy tables can be pre-generated for typical road surface types such as speed bumps, potholes, gravel, and ramps under different heights, materials, vehicle speeds, and vehicle attitudes. During online operation, the system can determine road surface attributes based on multi-source sensing data, then look up the corresponding table and generate the energy impact potential field through interpolation. This approach reduces online computation and helps improve real-time performance and stability. Another example is the use of physical information neural networks or residual learning models. For instance, the physical calculation model can first output the basic impact energy, and the neural network can then learn the residual between the real vehicle response and the physical model result to obtain the corrected energy impact potential field. This approach balances physical consistency and the fitting ability of the data-driven model, helping to improve prediction accuracy under complex road conditions. Yet another example is the use of high-fidelity simulation to generate a proxy model. For instance, a large number of road surface attributes, vehicle states, and impact response samples can be generated through vehicle dynamics simulation in the offline stage, and then a lightweight proxy model can be trained and deployed on the vehicle. This approach can cover extreme or rare operating conditions that are difficult to collect in large quantities on real roads, and helps reduce the online computing burden on the vehicle. For example, it can also be implemented using historical experience statistical models. The system can collect data on vehicle acceleration, suspension displacement, damping state, and comfort feedback when the vehicle actually passes through similar road surfaces, and establish a statistical mapping between road surface properties and impact response. This approach can reflect the actual vehicle usage and actuator state changes, helping to improve the consistency between the energy impact potential field and the actual riding experience. Another example is the generation of the energy impact potential field with uncertainty output. For instance, the model or physical calculation process can output not only the mean impact energy but also confidence level, variance, or risk interval. Subsequent prediction and game theory decisions can select more robust control strategies based on uncertainty. This approach can express uncertainties in perception or model prediction, helping to improve the robustness of control decisions under complex road conditions.

[0058] Step 304: Generate multiple sets of candidate joint control sequences, each set of candidate joint control sequences including control instructions corresponding to multiple execution units of the vehicle.

[0059] This step generates multiple sets of candidate joint control sequences covering different control possibilities, which can provide multiple control schemes for subsequent causal driving world models to be deduced and compared, thus enabling subsequent decisions to choose among multiple joint control paths.

[0060] The candidate joint control sequence refers to a set of time-sequential control schemes used to control the vehicle over a future period. These schemes can be generated through sampling, enumeration, rule template generation, optimization algorithm generation, model predictive control candidate set generation, or policy network output. Each candidate joint control sequence can cover multiple future control cycles, and there can be differences in control amplitude, control order, control timing, or control intensity between different candidate sequences. The multiple execution units refer to the actuators in the vehicle that can respond to control commands and affect the vehicle's driving state. These can be implemented as cruise control units, power units, braking units, steering assist units, suspension adjustment units, damping adjustment units, or energy recovery units. The control commands are instructions used to drive the execution units, and can be implemented as target acceleration, target deceleration, suspension height adjustment amount, continuously variable damping force target value, steering angle correction amount, braking force distribution command, or driving torque command. In a more suitable implementation, a set of candidate joint control sequences can simultaneously include target deceleration, air spring height adjustment amount, damper damping force target value, and steering angle correction amount.

[0061] The multiple candidate joint control sequences generated in this step are used in step 306. Since each candidate sequence includes control commands corresponding to multiple execution units, the subsequent prediction target is not the local action of a single actuator, but the future state change of the vehicle under the coordinated action of multiple execution units.

[0062] In one embodiment, multiple candidate joint control sequences can be generated in various ways depending on the actual situation. For example, sampling generation can be used, i.e., random sampling or Monte Carlo sampling can be performed within the feasible space of control commands to generate multiple candidate joint control sequences. For example, for the control commands of each execution unit, random sampling can be performed within its physical feasible range to form candidate sequences. Here, sampling generation refers to selecting multiple sets of control commands within the feasible space of control commands using random or statistical sampling methods to form candidate joint control sequences. The feasible space refers to the range of values ​​allowed for each execution unit under physical, safety, and comfort constraints, such as vehicle speed 0 to 120 km / h, suspension height 0 to 100 mm, and steering angle -30° to +30°. Monte Carlo sampling refers to a method of approximately covering the control space through a large number of random samples. In implementation, the system can first determine the value range of each execution unit, and then sample control quantities such as target deceleration, suspension height adjustment, and damping force target values ​​in several future control cycles. For example, for the three execution units of longitudinal control, suspension height control, and damping control, a set of control commands can be generated in 10 control cycles within the next second, and multiple candidate sequences can be obtained through repeated sampling. To avoid obviously infeasible solutions, the maximum deceleration, suspension adjustment rate, or steering angle change rate can also be limited. This method is relatively simple to implement, has a wide coverage, and is suitable for continuous and high-dimensional control spaces. It does not rely on a precise control model and can quickly form a diverse set of candidates, facilitating subsequent screening using a causal driving world model and multi-objective game equilibrium strategies.

[0063] For example, enumeration generation can also be used. This involves enumerating all possible combinations of the discretized control command space to generate candidate joint control sequences. For instance, vehicle speed can be discretized into several gears, suspension height into several gears, and braking intensity into several gears, enumerating all combinations. Here, enumeration generation refers to first discretizing the continuous control command space into a finite number of gears, then traversing each gear combination to generate candidate joint control sequences. Discretization refers to dividing continuous variables into several representative values ​​or intervals. In implementation, the system can set a finite number of gears for each execution unit. For example, vehicle speed can be discretized into 40km / h, 50km / h, 60km / h, and 70km / h; suspension height into low, medium, and high; and braking intensity into none, light, medium, and heavy. Combining these yields 48 candidate sequences. If the candidate sequences need to cover multiple future control cycles, a segmented constant value strategy can be used. For example, maintaining one set of control variables for the first 0.5 seconds and switching to another set for the next 0.5 seconds to control the growth of the number of candidates. This approach offers a well-defined candidate set with strong interpretability, facilitating calibration, verification, and safety review. For scenarios with positional actuators or highly discrete control variables, it can comprehensively cover feasible control combinations.

[0064] For example, optimization generation can also be used, which involves searching for candidate joint control sequences in the control command space based on optimization algorithms (such as genetic algorithms, particle swarm optimization, simulated annealing, etc.), prioritizing the generation of candidate sequences that are likely to have better performance. Here, optimization generation refers to searching for candidate joint control sequences with better performance in the control command space based on preliminary evaluation objectives or constraints. Genetic algorithms, particle swarm optimization, and simulated annealing are all optimization algorithms that find better solutions in complex search spaces. In implementation, the system can first construct a preliminary objective function, for example, using vehicle vertical acceleration, pitch angle change, travel time, and control quantity smoothness as evaluation indicators. Subsequently, each group of candidate sequences is encoded as an individual, particle, or state solution, and the candidate solutions are updated through iterative evaluation, selection, crossover, mutation, or neighborhood search. After several rounds of iteration, the top few groups of sequences with better evaluation can be retained as candidate joint control sequences for subsequent input into the causal driving world model. This method generates high-quality candidates, reducing the number of obviously inferior solutions entering subsequent prediction and game decision-making stages, and helping to reduce the computational pressure of world model deduction and game solution. For scenarios with multiple constraints, nonlinearity, and multiple coupled execution units, it is possible to search for a better control region in a more targeted manner.

[0065] For example, a policy network output can also be used. The state feature representation and the environment feature representation are input into a trained policy network (such as a deep reinforcement learning policy network), which then outputs multiple sets of candidate joint control sequences. This policy network can be based on an Actor-Critic architecture, where the Actor network outputs the mean and variance of control commands, from which multiple candidate sequences are sampled. Here, the policy network refers to a neural network that, after training, can output control commands or control command distributions based on state and environment feature representations. In the Actor-Critic architecture, the Actor network outputs control actions or action distributions, and the Critic network evaluates the value of actions or states. In implementation, the system can input the state and environment feature representations into the policy network. The Actor network can output the mean and variance of control quantities such as target deceleration, suspension height adjustment, and damping force target values ​​over several future control cycles. The system can sample multiple sets of candidate joint control sequences from this distribution and trim the results to the feasible range of the execution unit. To improve candidate quality, the evaluation results of the Critic network can be combined to retain candidate sequences with higher value. This method generates sequences quickly, and the candidate sequences can match the current road conditions and vehicle states to a high degree. Because the policy network can learn control experience from historical data or simulation training, it is suitable for handling complex, high-dimensional, and continuously changing all-terrain control scenarios.

[0066] For example, rule-based generation can also be used, that is, generating candidate joint control sequences based on preset control rules. For instance, based on the current vehicle speed and road obstacle type, several sets of candidate control sequences can be generated according to preset rules (such as "high-speed passage + stiff suspension", "medium-speed passage + soft suspension", "low-speed passage + raised suspension", etc.). Rule generation refers to the method of generating candidate joint control sequences based on preset control rules, scenario templates, or driving experience. Preset control rules can be obtained from engineering calibration, typical scenario libraries, safety boundaries, or summaries of driving common sense. In implementation, the system can first identify the current road type and vehicle state based on environmental characteristics, and then call the corresponding rules to generate several candidate control schemes. For example, in a scenario where there is a high speed bump ahead and the current vehicle speed is high, the rule library can generate schemes such as "low-speed passage + raised suspension + medium damping", "medium-speed passage + medium suspension height + slightly stiff damping", and "maintain current vehicle speed + standard suspension setting", and expand each scheme into specific control instructions for several future control cycles, forming candidate joint control sequences. This method has strong interpretability, easy-to-control safety boundaries, and low computational load. For typical scenarios such as speed bumps, potholes, slopes, and muddy roads, it can quickly provide candidate solutions that match driving experience and serve as a basic or fallback candidate set for complex scenarios.

[0067] In practice, the above generation methods can also be used in combination. For example, the system can first generate a basic candidate set through rule generation, then expand the candidate diversity through sampling, subsequently use optimization to filter better candidates, and finally use a policy network to supplement or accelerate the generation of candidate sequences. This combined approach can balance the coverage, generation efficiency, interpretability, and candidate quality of the candidate set, thereby providing a more sufficient basis for subsequent causal driving world model predictions and multi-objective game equilibrium decisions.

[0068] Step 306: Invoke the causal driving world model and predict the future state evolution results corresponding to the multiple candidate joint control sequences based on the state feature representation and the environmental feature representation.

[0069] This step involves using a causal driving world model to extrapolate the future states of multiple candidate joint control sequences. This allows for the assessment of potential vehicle state changes caused by different control schemes before the actual issuance of control commands, providing a predictive basis for selecting the target joint control sequence in the subsequent multi-objective game equilibrium strategy.

[0070] The causal driving world model refers to a predictive model used to characterize the causal relationship between control input, environmental state, and future vehicle state. It can be implemented using a causal Transformer architecture, an autoregressive neural network, a vehicle dynamics model, a data-driven predictive model, or a hybrid model combining data-driven and vehicle dynamics models. In an implementation more suited to this scheme, the causal driving world model can receive current state feature representations, environmental feature representations, and candidate joint control sequences, and predict the evolution of vehicle state and scene response within one to two seconds. Unlike traditional data-driven end-to-end predictive models, the causal driving world model explicitly models the causal relationship between control commands (cause) and the future vehicle state (effect), accurately predicting the evolution trend of the vehicle's future state under different control commands.

[0071] The future state evolution result refers to the possible state changes of the vehicle at several future moments under the action of a candidate joint control sequence. This can be achieved through state trajectories, vehicle attitude change curves, vertical acceleration change curves, suspension travel change curves, pitch angle change curves, roll angle change curves, impact risk indicators, or comfort indicators. This result can reflect the potential impact of a candidate control sequence on traffic efficiency, vehicle response, attitude changes, or impact risk. Specifically, the future state evolution result may include parameters reflecting the vehicle's future driving state, such as vehicle position, velocity, acceleration, yaw rate, vehicle attitude angles (pitch angle, roll angle), suspension displacement, vehicle vibration acceleration, tire contact force, and slip ratio.

[0072] The future state evolution result output from this step can be input into step 308. Through this step, the system can first compare the potential impact of different candidate joint control sequences in the future, and then select a control scheme that is more suitable for the current road conditions and vehicle state, thereby helping to reduce the lag problem caused by real-time control based solely on the current local state.

[0073] Multiple candidate joint control sequences may contain control commands for multiple future time steps. If the prediction model struggles to characterize the temporal causal relationship between control input, road environment, and vehicle state, the future state evolution results of different candidate sequences may be difficult to compare accurately. Therefore, in one embodiment, a causal Transformer autoregressive prediction scheme can be employed. For example, the causal driving world model can use a causal Transformer architecture to predict the future state evolution results corresponding to the multiple candidate joint control sequences in an autoregressive manner. The causal Transformer architecture refers to a temporal prediction model based on Transformer and incorporating causal constraints. It can employ an encoder-decoder structure, a decoder structure with causal attention masks, or a temporal Transformer structure. The encoder encodes state feature representations and environmental feature representations to obtain contextual representations; the decoder predicts future vehicle states step-by-step. The autoregressive method refers to using the predicted state of the previous time step, the control command of the current time step, and the contextual representation output by the encoder as input to predict the vehicle state of the current time step, and then using this predicted state for prediction in subsequent time steps.

[0074] Take the example of a speed bump or a series of potholes in front of a vehicle. In this scenario, the encoder first receives state feature representations and environmental feature representations. State feature representations can include current vehicle speed, vertical acceleration, pitch angle, roll angle, suspension height, and suspension travel. Environmental feature representations can include obstacle location, geometry, material hardness, and road surface adhesion conditions. The encoder encodes this information into a contextual representation. Subsequently, for a candidate joint control sequence, the decoder can progressively predict future states in an autoregressive manner. For example, in the first time step, the decoder can predict the vehicle speed, vertical acceleration, pitch angle, and suspension travel at the next moment based on the current state, contextual representation, and the target deceleration, suspension height adjustment, and damping force target value at that time step. In the second time step, the decoder can continue to input the predicted state from the previous time step and the control command for that time step to predict the vehicle state at a later time. Through progressive unfolding, the model can obtain the evolution results of the kinematic state, dynamic state, vehicle attitude, suspension state, and tire state of the candidate sequence in the next one to two seconds. Other candidate joint control sequences can also be predicted separately in the same way, thus obtaining multiple sets of future state evolution results for comparison.

[0075] This embodiment employs a causal Transformer architecture, which can express the temporal causal relationships between control inputs, road environment, and vehicle state. It not only enables multi-timestep extrapolation for each candidate joint control sequence but also improves the comparability of prediction results between different candidate sequences. Furthermore, the autoregressive approach can progressively accumulate future state changes, aiding in the prediction of continuous responses such as vehicle attitude, suspension state, and tire state, thereby providing a basis for selecting the objective joint control sequence in a multi-objective game equilibrium strategy.

[0076] The causal driving world model can be trained using methods such as supervised learning, self-supervised learning, reinforcement learning, or contrastive learning. In supervised learning, future states from real or simulated driving data can be used as labels, and the model is trained using mean squared error (MSE) or mean absolute error (MAE) loss. For example, a large amount of real or simulated driving data can be collected, including vehicle state feature representations, environmental feature representations, control command sequences, and corresponding real future state evolution results. The causal driving world model is trained using state feature representations, environmental feature representations, and control command sequences as inputs, and real future state evolution results as labels. The loss function can be either MSE or MAE. In self-supervised learning, the temporal relationship between consecutive frames can be used to predict the state of the next frame. For example, the temporal continuity in driving data can be used for self-supervised training. For instance, multiple consecutive frames of perception data and state data can be used as inputs, allowing the model to predict the state of the next frame, and self-supervised learning can be performed using the prediction error. In reinforcement learning, the world model can be jointly trained with a policy network as an environment model. For example, the causal driving world model can be jointly trained with a decision-making policy network as an environment model. The policy network outputs control commands, the causal driving world model predicts future states, and calculates reward signals based on these future states. Reinforcement learning algorithms (such as Proximal Policy Optimization (PPO) and Soft Actor Key Achievement (SAC)) are used to simultaneously optimize both the policy network and the causal driving world model. Contrastive learning training can improve generalization ability by giving similar driving scenarios similar representations. For example, contrastive learning methods can be used to learn state and environment representations, ensuring that similar driving scenarios have similar representations and different driving scenarios have different representations, thereby improving the generalization ability of the causal driving world model.

[0077] The causal driving world model predicts the future state evolution of a vehicle after executing candidate joint control sequences, including the vehicle's kinematic state, dynamic state, body posture, suspension state, and tire state. By predicting the future state evolution results corresponding to different candidate joint control sequences, the vehicle responses that different control schemes may cause can be compared before execution, thus providing a basis for subsequent game equilibrium decisions.

[0078] Step 308: Using a multi-objective game equilibrium strategy, evaluate the comprehensive impact of each candidate joint control sequence on multiple game agents related to driving quality based on the future state evolution results, so as to determine the target joint control sequence that satisfies the game equilibrium conditions.

[0079] This step comprehensively evaluates the future impact of multiple candidate joint control sequences through a multi-objective game equilibrium strategy, which can form a dynamic balance among multiple driving quality-related objectives and provide decision results for driving multiple execution units according to the objective joint control sequence.

[0080] The multi-objective game equilibrium strategy refers to a strategy that uses multiple objectives related to driving quality as game subjects and selects or solves them based on the future impact of candidate control sequences. This strategy can be implemented using methods such as Nash equilibrium solution, approximate Nash equilibrium solution, multi-head attention game solution networks, iterative optimization of value functions, or multi-objective collaborative optimization. The game subjects refer to the decision objects used to represent a certain type of driving quality objective. In an implementation more suitable for this scheme, the game subjects may include a speed efficiency side that pursues shorter travel time, a ride comfort side that pursues smaller vehicle vibration amplitude, and a stability and safety side that pursues smaller vehicle pitch and roll angles.

[0081] The comprehensive impact refers to the potential gains, losses, or constraints that a candidate joint control sequence may have on different game agents, which can be realized in the form of value functions, cost functions, scoring functions, risk indicators, or weighted evaluation results. The game equilibrium condition refers to the condition satisfied when multiple game agents reach a state of mutual restraint; for example, if the strategies of other game agents remain unchanged, it is difficult for any single game agent to further improve its own gains by unilaterally changing its own strategy. The target joint control sequence refers to the control sequence selected from multiple sets of candidate joint control sequences for subsequent execution.

[0082] The target joint control sequence output in this step can be input into step 310. Through this step, the system can select a control sequence that takes into account multiple driving quality-related objectives from multiple candidate control schemes, thereby helping to alleviate potential conflicts between traffic efficiency, ride comfort, and vehicle stability.

[0083] In one embodiment, the multiple game agents related to driving quality include at least two of the following (i.e., at least two of the following multiple stakeholders, hereinafter the same): a speed efficiency staker aiming for the shortest travel time, a ride comfort staker aiming for the minimum vehicle body vibration amplitude, a stability and safety staker aiming for the minimum vehicle body attitude angle, and a driving economy staker aiming for the lowest energy loss. The speed efficiency staker is a game agent aiming to shorten the time required to pass through the current road segment or obstacle, which can be achieved using a travel time estimation model, an average vehicle speed cost function, or a passage time penalty term. The ride comfort staker is a game agent aiming to reduce the vehicle body vibration amplitude, which can be achieved using a comfort cost function constructed from vertical acceleration, impact, suspension travel, or seat vibration indices. The stability and safety staker is a game agent aiming to reduce the vehicle body attitude angle and maintain driving stability margin, which can be achieved using pitch angle, roll angle, attitude angle change rate, tire load transfer, or instability margin indices. The driving economy is a game subject with the goal of reducing energy loss during vehicle passage. It can be implemented using a driving energy consumption, braking energy consumption, energy recovery benefits, suspension adjustment energy consumption, or comprehensive power consumption model.

[0084] It is understood that the game subjects described in this embodiment of the invention are mathematical decision-making objects abstracted within the control system for the convenience of multi-objective decision-making, rather than specific natural persons or organizations. Each of the aforementioned game subjects (i.e., each "party") corresponds to a type of driving quality objective and its benefit function or cost function. For example, the speed efficiency party corresponds to travel time, the ride comfort party corresponds to vehicle vibration, the stability and safety party corresponds to vehicle attitude angle and stability margin, and the driving economy party corresponds to energy loss. In the software, they can be represented as different evaluation modules, value functions, or optimization objectives, and do not possess consciousness, interest claims, or legal subject qualifications. The relationship between real entities such as drivers and car manufacturers and these game subjects is mainly reflected in the source and configuration of the objective functions. For example, car manufacturers can set the evaluation indicators, constraints, and default weights of each game subject through calibration, simulation, and engineering experience to ensure that the control results conform to the vehicle's positioning, safety requirements, and product style. Drivers can also influence the parameters of the game subjects through driving modes, comfort preferences, or travel rhythm preferences, such as increasing the influence of the speed efficiency party in sport mode and increasing the influence of the ride comfort party in comfort mode. Therefore, the main players in the game are the abstract decision-making units within the vehicle's driving control system, while drivers and automakers can be the designers, configurers, or beneficiaries of these decision objectives. The role of multi-objective game equilibrium strategies is to conduct quantitative comparisons and checks and balances among multiple abstract objectives, thereby outputting a joint control sequence that takes into account different driving quality requirements.

[0085] Each of the aforementioned game participants holds a corresponding value function. The specific form of the value function is merely an example; those skilled in the art can choose other forms of value functions according to actual needs, such as weighted summation or exponential forms. The value function of the speed efficiency side aims to minimize travel time. Specifically, the value function of the speed efficiency side can be defined as the time required for a vehicle to pass through a specific road segment (such as a segment containing special road conditions and / or obstacles). The smaller the value function value, the higher the travel efficiency. The value function of the speed efficiency side can be expressed as: Vspeed = Tpassage, where Tpassage is the time it takes for a vehicle to pass through the target road segment.

[0086] The value function of the ride comfort method aims to minimize the amplitude of vehicle body vibration. Specifically, the value function of the ride comfort method can be defined as the root mean square (RMS) or peak value of the vertical vibration acceleration of the vehicle body during the process of the vehicle passing through the target road segment. The smaller the value function, the better the ride comfort. The value function of the ride comfort method can be expressed as: Vcomfort = RMS(az), where az is the vertical vibration acceleration of the vehicle body.

[0087] The value function of the stability safety method aims to minimize the vehicle's attitude angle. Specifically, the value function of the stability safety method can be defined as the maximum or root mean square value of the vehicle's pitch and roll angles during the vehicle's passage through the target road segment. The smaller the value function value, the better the vehicle's stability. The value function of the stability safety method can be expressed as: Vstability = max(|θpitch|, |φroll|), where θpitch is the vehicle's pitch angle and φroll is the vehicle's roll angle.

[0088] Taking the example of a speed bump or a series of potholes in front of a vehicle, the speed efficiency side might prefer a higher passing speed and a shorter braking distance to reduce travel time; the ride comfort side might prefer a lower passing speed, adjusting the suspension height in advance, or using softer damping to reduce vertical acceleration and vehicle vibration; the stability and safety side might prefer to limit pitch angle, roll angle, and rate of attitude change to avoid excessive tire load transfer; and the driving economy side might prefer smooth braking, reducing unnecessary suspension adjustments, and increasing the proportion of regenerative braking. Different game actors can give different benefit or cost evaluations for the same candidate joint control sequence.

[0089] By designating the aforementioned objectives as game actors, travel time, comfort, stability, and economy can be transformed from weighted terms implicit in a single cost function into decision objects that can be evaluated individually and mutually constrained. In this way, the system can calculate the payoffs of each party based on the future state evolution of different candidate sequences, and then select a joint control sequence based on the objective through a game equilibrium strategy. Selecting at least two game actors for comprehensive game theory helps avoid a single objective dominating the control outcome; when speed efficiency, ride comfort, stability and safety, and driving economy are simultaneously included, it is possible to balance throughput efficiency, passenger experience, posture safety, and energy consumption under complex road conditions, making the control outcome more in line with the overall vehicle driving quality requirements.

[0090] In addition to the aforementioned game actors, other game actors related to driving quality can be set up according to actual needs. For example, a passability actor can be set up to assess the vehicle's ability to pass through potholes, mud, or slopes; a chassis wear actor can be set up to assess the wear and tear costs of suspension, tires, and brake components; and a driving intention actor can be set up to reflect the driver's or passenger's preference for travel pace. These game actors can all construct corresponding payoff functions based on future state evolution results and participate in solving multi-objective game equilibrium.

[0091] It is understandable that there may be conflicts between the objectives of the aforementioned multiple game actors. Therefore, a clear equilibrium solution method and judgment condition can be provided to stably select the objective sequence. In one embodiment, a multi-objective game equilibrium solution mechanism can be adopted. For example, the multi-objective game equilibrium strategy can adopt a Nash equilibrium solution strategy, a Nash negotiation solution solution strategy, a cooperative game payoff distribution strategy, or a multi-objective collaborative solution strategy based on iterative optimization of the value functions of the multiple game actors. Accordingly, the objective joint control sequence is the candidate joint control sequence output by the multi-objective game equilibrium strategy that makes the comprehensive payoff of the multiple game actors satisfy the preset equilibrium condition. Among them, the Nash equilibrium solution strategy is used to find candidate sequences in which it is difficult for any game actor to improve its own payoff by unilaterally changing its strategy, and can be implemented by optimal response iteration or ε-approximate Nash equilibrium judgment. The Nash negotiation solution solution strategy is used to solve the candidate sequence based on the payoff increment of each party relative to the negotiation breakdown point, and can be implemented by convex optimization. The cooperative game payoff distribution strategy is used to regard the multiple game actors as a cooperative alliance and distribute the overall payoff, and can be implemented by Shapley value or proportional distribution rule. The multi-objective collaborative solution strategy is used for iterative optimization based on the value function, and can be implemented using weighted sum optimization or Pareto search. The preset equilibrium conditions may include strict Nash equilibrium, ε-approximate Nash equilibrium, or multi-objective balance conditions, etc.

[0092] The methods for determining the game equilibrium conditions may include: When employing a Nash equilibrium strategy, for a set of candidate joint control sequences, if no player can unilaterally change their corresponding control strategy to increase their own value function value (i.e., reduce their own cost), then the candidate joint control sequence is deemed to satisfy the Nash equilibrium condition. Specifically, for a candidate sequence u, if for each player i, Vi(u) ≤ Vi(ui, u{-i}), where ui is any strategy of player i and u{-i} are the strategies of other players, then u satisfies the Nash equilibrium condition.

[0093] When employing an approximate Nash equilibrium strategy, a strict Nash equilibrium may be difficult to find or may not exist. Therefore, an approximate Nash equilibrium criterion can be used. If, for each player i, the improvement in the value function resulting from unilaterally changing the strategy does not exceed a preset threshold ε, then the candidate joint control sequence is determined to satisfy the ε-approximate Nash equilibrium condition. ≤ε.

[0094] When adopting a multi-objective balance strategy, if a set of candidate joint control sequences makes the difference between the value function values ​​of each game agent not exceed a preset threshold, or makes the weighted sum of the value function values ​​of each game agent the minimum, then the candidate joint control sequence is determined to satisfy the multi-objective balance condition.

[0095] Taking the appearance of a speed bump in front of a vehicle as an example, the system can calculate the value function values ​​of each player under different candidate sequences based on the future state evolution results. If the ε-approximate Nash equilibrium judgment is adopted, when the improvement in payoff brought by any player unilaterally changing its strategy does not exceed the threshold ε, the candidate sequence can be selected as the joint control sequence for the objective. It is evident that this embodiment provides an executable solution method and judgment conditions for game equilibrium strategies. It can not only handle objective conflicts between different players, but also select strict equilibrium, approximate equilibrium, or collaborative optimization methods according to the complexity of the scenario. Furthermore, screening target sequences through preset equilibrium conditions helps improve selection stability and multi-objective balancing capabilities.

[0096] Alternatively, the Pareto optimal solution selection scheme can be used, which first obtains the non-dominated candidate sequences and then selects the target sequence according to scenario preferences. This scheme can preserve multi-objective trade-off information and helps adapt to different driving needs.

[0097] While future state evolution results can reflect the potential impact of candidate sequences, the importance and safety boundaries of each game actor may differ under different road conditions and vehicle states. Directly comparing prediction results might include candidate sequences that do not meet physical constraints or safety requirements. To address this, a constraint-weighted evaluation mechanism based on evaluation parameters can be employed. In one embodiment, when evaluating the comprehensive impact of each group of candidate joint control sequences on multiple game actors related to driving quality based on the future state evolution results, evaluation parameters corresponding to the state characteristic representation and the environmental characteristic representation can be determined first. These evaluation parameters include constraints and / or weighting factors. Then, based on the future state evolution results, the comprehensive impact of each group of candidate joint control sequences on the multiple game actors under the condition of meeting the evaluation parameters is evaluated.

[0098] The evaluation parameters refer to parameters determined based on the current state characteristics and environmental characteristics, used to constrain or adjust the comprehensive evaluation process. These parameters can be implemented using parameter lookup tables, rule mapping, scenario classification models, or neural network inference. The constraints refer to the limitations that candidate joint control sequences or future state evolution results must satisfy. These may include physical constraints such as maximum acceleration, maximum braking deceleration, maximum steering angle, and suspension height range; safety constraints such as minimum safe distance and maximum roll angle threshold; and comfort constraints such as maximum vibration acceleration threshold. The weighting factors represent the importance of each game player in the comprehensive evaluation; a larger weighting factor indicates a higher proportion of the game player's value function in the comprehensive evaluation.

[0099] Taking a scenario where a significant speed bump appears ahead of the vehicle while the current speed is high, the system can determine evaluation parameters based on state and environmental characteristics. For example, constraints might include a maximum braking deceleration not exceeding 0.25g, a roll angle not exceeding a preset threshold, and suspension height within an adjustable range. Weighting factors could increase the proportion of comfort and stability / safety considerations while appropriately decreasing the proportion of speed / efficiency considerations. Subsequently, based on the future state evolution of each candidate sequence, the system first determines whether it meets the constraints, then calculates the value functions of each player for the constrained candidate sequences, and finally obtains the comprehensive impact according to the weighting factors.

[0100] The evaluation parameters used in this embodiment allow the comprehensive impact assessment to be adapted to the current vehicle state and road conditions. It not only eliminates candidate sequences that do not meet physical, safety, or comfort requirements through constraints, but also adjusts the importance of different game actors in the current scenario through weighting factors. This approach helps reduce invalid candidate sequences from entering the subsequent game-solving process, thereby improving the safety, relevance, and computational efficiency of the selection process.

[0101] Given that constraints and weighting factors may vary significantly under different road conditions and vehicle states, relying solely on manual, temporary settings may not be sufficient to balance efficiency and scenario adaptability. Therefore, two evaluation parameter determination mechanisms can be employed: parameter table lookup or large language model encoding. In one embodiment, when determining the evaluation parameters corresponding to the state feature representation and the environmental feature representation, the corresponding evaluation parameters can be directly queried from a preset evaluation parameter table. The preset evaluation parameter table refers to a table or database that pre-stores the mapping relationships between different driving scenarios and evaluation parameters. It can be implemented using lookup tables, rule bases, scenario index tables, or calibration parameter sets. The table can store constraints and weighting factors corresponding to scenarios such as highways, urban roads, off-road roads, speed bumps, and potholes.

[0102] Alternatively, a natural language description can be constructed based on the state feature representation and the environment feature representation, and a large language model can be invoked to encode the driving common sense suggestions corresponding to the natural language description into the evaluation parameters. The natural language description refers to the conversion of the state feature representation and the environment feature representation into a text-based scene expression, which can be achieved through template filling, feature decoding, scene summary generation, or prompt word construction. A large language model (LLM) is a large-scale language model capable of understanding natural language scene descriptions and outputting driving common sense suggestions. It can be implemented using in-vehicle deployment models, cloud models, models fine-tuned with driving data, or prompt engineering calls. The driving common sense suggestions refer to the control suggestions given by the model for the current scene, such as suggestions on speed, suspension stiffness, braking intensity, or traffic rhythm. Encoding driving common sense suggestions into evaluation parameters is the process of converting driving common sense suggestions into constraints and weight factors, which can be accomplished through keyword parsing, structured output, rule mapping, or semantic parsers.

[0103] Taking an example where there is a speed bump about 5 cm high 50 meters ahead of the vehicle, the current speed is 60 km / h, and the road surface is asphalt. If using a pre-defined evaluation parameter table query method, the system can first identify the current scenario as an urban road speed bump scenario based on state and environmental characteristics, and then query the corresponding evaluation parameters. For example, the query results may include parameters such as the maximum speed when passing the speed bump is 20 km / h, the maximum vertical acceleration threshold, a higher weight for comfort, and a lower weight for speed efficiency. If using a large language model encoding method, the system can first generate a natural language description, such as "The vehicle is currently traveling at 60 km / h, there is a speed bump 5 cm high 50 meters ahead, and the road surface is asphalt." The large language model can output driving common sense suggestions, such as "When passing the speed bump, you should slow down to below 20 km / h and adjust the suspension to a softer mode." Subsequently, the system can encode this suggestion into evaluation parameters, such as setting the maximum passing speed to 20 km / h, setting the suspension stiffness mode to softer, and increasing the weight factor corresponding to ride comfort.

[0104] As can be seen, this embodiment provides two feasible paths for determining evaluation parameters. The pre-defined evaluation parameter table lookup method not only improves parameter acquisition speed but also maintains the stability and interpretability of the evaluation criteria; the large language model encoding method can introduce driving common sense suggestions based on natural language scene descriptions, which helps handle complex or poorly pre-calibrated road conditions. Both methods can transform current state feature representations and environmental feature representations into constraints and weighting factors, which helps improve the matching degree between evaluation parameters and the current driving scenario.

[0105] Of course, besides the two methods mentioned above, a scenario classification model can directly output evaluation parameters. The trained classification or regression model predicts constraints and weighting factors based on state and environmental characteristics. This approach reduces manual table lookups and helps adapt to continuously changing driving scenarios. Alternatively, a historical case retrieval scheme can be used to match the current scenario with similar historical scenarios and reuse evaluation parameters from those similar scenarios. This scheme leverages existing driving experience and improves the stability of parameter determination. A fuzzy rule inference scheme can also be employed, inferring evaluation parameters based on inputs such as vehicle speed, obstacle height, and road surface adhesion conditions. This scheme can express progressively changing relationships under uncertain road conditions and helps smoothly adjust constraints and weighting factors.

[0106] Furthermore, when assessing the combined impact of each group of candidate joint control sequences on multiple game actors, a screening-then-ranking evaluation scheme can be adopted. This involves first eliminating candidate sequences that do not meet security or physical constraints based on future state evolution results, and then ranking them according to the value functions of each game actor. This scheme simplifies the evaluation process and helps ensure basic security boundaries.

[0107] Figure 4 This is a schematic diagram illustrating a collaborative control process based on multi-objective game equilibrium, as shown in an embodiment of the present invention. Figure 4 As shown, this process takes multiple sets of candidate joint control sequences and their corresponding future state evolution results as input, comprehensively evaluates each set of candidate sequences through a multi-objective game equilibrium strategy, and finally outputs a target joint control sequence that satisfies the game equilibrium conditions. The complete process can be divided into five parts: input layer, decision game core layer, prediction layer, game equilibrium solution layer, and output layer. The data flow and processing order between each part are indicated by arrows.

[0108] like Figure 4 As shown, the input to this process includes two parallel data sources: the current physical cognitive feature map F_t and the current vehicle state S_t, both of which can come from step 302 mentioned above. The physical cognitive feature map F_t is used to characterize the distribution of road condition physical attributes at various road surface locations in the current driving scenario; the current vehicle state S_t is used to characterize the vehicle's current motion state and chassis state. Both are input to the "Decision Game Core" module via downward arrows, serving as the initial basis for subsequent game decisions.

[0109] The core of the decision-making game is a holistic module comprising multiple functional modules. Internally, it initially sets up three game subjects side-by-side: speed efficiency (value function: shortest travel time) V1, ride comfort (value function: minimum vehicle vibration) V2, and stability and safety (value function: minimum attitude angle) V3. These three game subjects are arranged horizontally side-by-side, indicating that they participate in the decision-making process at the same level. Specifically, speed efficiency V1 optimizes for minimizing the travel time through the target road segment, ride comfort V2 optimizes for minimizing vehicle vibration amplitude, and stability and safety V3 optimizes for minimizing vehicle attitude angles (pitch and roll angles). The three game subjects evaluate candidate control schemes from three dimensions: travel efficiency, ride comfort, and vehicle stability.

[0110] The outputs of the three game actors all point downwards to "candidate joint control sequence U_candidate (t:t+N)", which represents the generation of multiple sets of candidate joint control sequences covering the next t to t+N control cycles, based on the current physical cognitive feature map F_t and the current vehicle state S_t, combined with the evaluation objectives of the three game actors. Each set of candidate joint control sequences includes control commands corresponding to multiple execution units of the vehicle, such as target deceleration, suspension height adjustment, damping force target value, and steering angle correction.

[0111] The candidate joint control sequence U_candidate (t:t+N) is input to the "driving world model (predictor)". This model adopts a causal Transformer architecture and uses an autoregressive approach to extrapolate the future state of each candidate joint control sequence. The output of the driving world model is used to predict the future state sequence S_pred (t+1:t+N) and the corresponding energy impact potential field evolution. That is, the vehicle state evolution results predicted by the driving world model for each candidate joint control sequence at times t+1 to t+N in the future, including state parameters such as vehicle position, velocity, acceleration, vehicle attitude angle, suspension displacement, and vehicle vibration acceleration, as well as the evolution of the corresponding energy impact potential field over time.

[0112] The predicted evolution is input into the "game equilibrium solution network," which uses a multi-head attention mechanism to evaluate the value functions of multiple candidate joint control sequences in parallel across the three player dimensions. The solution formula is Uopt = argmax Nash (V1,V2,V3), representing the selection of the target joint control sequence Uopt from multiple candidate joint control sequences that achieves Nash equilibrium (usually optimal / maximum overall payoff) for the three players through Nash equilibrium solution. Specifically, the game equilibrium solution network can calculate the value function values ​​of speed efficiency (V1), driving comfort (V2), and stability and safety (V3) for each candidate sequence based on the predicted future state sequence, and then filter out candidate sequences that satisfy the game equilibrium conditions using Nash equilibrium criteria (or approximate Nash equilibrium criteria, multi-objective balance criteria, etc.).

[0113] The game equilibrium solution network ultimately outputs the selected target joint control sequence U, which is the optimal control sequence under the current driving scenario (characterized by the current physical cognitive feature map F_t and the current vehicle state S_t). This target joint control sequence U contains control instructions corresponding to multiple execution units of the vehicle. In the subsequent step 310, the multiple execution units can be driven directly according to the control instructions in this sequence to achieve coordinated control of the vehicle.

[0114] Step 310: Drive the plurality of execution units according to the control instructions in the target joint control sequence to control the vehicle's movement.

[0115] This step drives multiple execution units according to the control instructions in the target joint control sequence, which can transform the results of the preceding prediction and game decision into the actual coordinated control actions of the vehicle, thereby achieving control of the vehicle driving process.

[0116] The control commands in the target joint control sequence refer to the sequential control commands acting on different execution units within the target sequence. These commands can be implemented in the form of target deceleration commands, target acceleration commands, suspension height adjustment commands, damping force adjustment commands, steering angle correction commands, braking force commands, or driving torque commands. Driving multiple execution units refers to the driving control system issuing control commands to corresponding actuators and changing the vehicle's driving state through these actuators. This can be achieved using electronic throttle, brake-by-wire, air springs, continuously variable damping shock absorbers, steering assist motors, or energy recovery devices.

[0117] In an implementation that better suits this solution, the cruise control actuators can adjust the vehicle speed based on the target deceleration or acceleration, the air springs can adjust the vehicle height based on the suspension height adjustment, the continuously variable damping shock absorbers can adjust the suspension damping based on the target damping force value, and the steering assist unit can perform steering angle corrections when needed. Since these control commands originate from the same target joint control sequence, the actions of multiple actuators can be coordinated around the same predicted result and the same decision objective.

[0118] After this step is executed, the vehicle's actual driving state can be collected again by sensors and used as the driving state data input in the next step 302. Thus, steps 302 to 310 can form a closed-loop control process, enabling the vehicle to continuously perform state perception, candidate generation, future inference, game theory decision-making, and collaborative execution under complex road conditions.

[0119] In one embodiment, the plurality of execution units may include at least two of the following: a cruise control unit, a power unit, a braking unit, a steering assist unit, a suspension adjustment unit, and an energy recovery unit.

[0120] The cruise control unit is an execution unit used to generate longitudinal speed control targets based on set vehicle speed, distance to the vehicle in front, relative speed, or passing target. It can coordinate the vehicle to maintain a set speed, follow the vehicle in front, or adjust its passing speed according to target speed, target acceleration, or target deceleration commands in the target joint control sequence. The power unit is an execution unit used to provide driving force to the vehicle, and may include a drive motor, engine, transmission system, or torque distribution device. In this solution, the power unit can adjust the drive torque, output speed, torque rise rate, or traction limit according to control commands, thereby affecting the vehicle's acceleration and passing ability. The braking unit is an execution unit used to decelerate the vehicle, and may include a hydraulic braking system, brake-by-wire system, or electronic brake control device. The braking unit can adjust braking pressure, braking deceleration, or front and rear axle braking force distribution according to control commands, thereby controlling the vehicle's deceleration process. The steering assist unit is an execution unit used to assist or correct vehicle steering, and may include an electric power steering device, active steering device, or steering motor. The steering assist unit can adjust the steering angle, steering torque, or path correction amount according to control commands, thereby helping the vehicle maintain the expected driving trajectory. The suspension adjustment unit is an actuator used to adjust vehicle height, suspension damping, or suspension stiffness. It may include air springs, continuously variable damping shock absorbers, active suspension, or height adjustment devices. The suspension adjustment unit can adjust vehicle posture, suspension travel, or vibration damping characteristics according to control commands, thereby improving the comfort and stability of the vehicle when traversing complex road surfaces. The energy recovery unit is an actuator used to recover kinetic energy during vehicle deceleration, downhill driving, or coasting. It may include a drive motor generator, regenerative braking system, or energy storage device. The energy recovery unit can provide regenerative braking torque according to control commands and convert a portion of the vehicle's kinetic energy into electrical energy for storage.

[0121] For example, when the cruise control unit is combined with the suspension adjustment unit, it can achieve coordinated control of longitudinal speed and vehicle attitude when the vehicle approaches a speed bump or pothole. For instance, the cruise control unit can reduce the vehicle speed to a suitable speed for overcoming the obstacle in advance, while the suspension adjustment unit can raise the vehicle body or adjust damping before the wheels contact the obstacle. Compared to simply reducing speed or simply adjusting the suspension, the combined effect of both can simultaneously reduce chassis impact and vehicle attitude changes, thus helping to balance ride comfort and driving stability. When the braking unit, energy recovery unit, and suspension adjustment unit are combined, they can achieve coordinated control of deceleration, energy utilization, and vehicle support when encountering obstacles ahead, going downhill, or on continuously undulating road surfaces. For example, the energy recovery unit can first provide some regenerative braking torque, the braking unit then supplements the friction braking force according to the target deceleration, and the suspension adjustment unit simultaneously adjusts damping or vehicle height. This allows for smooth deceleration while recovering some kinetic energy and reducing the impact of road excitation on the vehicle body. When the power unit, steering assist unit, and suspension adjustment unit are combined, they can achieve coordinated control of traction, path, and vehicle attitude in muddy, gravel, or low-traction curves. For example, the powertrain unit can limit drive torque to reduce the risk of wheel slippage, the steering assist unit can make minor path corrections, and the suspension adjustment unit can adjust vehicle height or damping to adapt to uneven road surfaces. This collaborative approach helps improve the vehicle's ability to navigate complex road conditions and its driving stability. When the cruise control unit, braking unit, and energy recovery unit are combined, they can achieve coordinated control of longitudinal speed in scenarios such as following a vehicle while decelerating or going downhill. For example, the cruise control unit can generate a target deceleration based on the distance to the vehicle in front and the relative speed, the energy recovery unit can prioritize regenerative braking, and the braking unit can supplement braking force when needed. This approach helps to make the deceleration process smoother while reducing energy consumption. When the powertrain unit and suspension adjustment unit are combined, they can achieve coordinated control of drive and vehicle attitude in scenarios such as starting on a slope, overcoming obstacles, or changing road surface adhesion conditions. For example, the powertrain unit can smoothly output drive torque, and the suspension adjustment unit can suppress vehicle pitch or adjust suspension support characteristics. This collaborative approach helps to reduce wheel slippage and vehicle attitude fluctuations, improving vehicle stability when traversing complex road sections.

[0122] It is evident that at least two of the multiple execution units can coordinate their actions under the same target joint control sequence. Since the control commands of each execution unit originate from the same prediction and game-theoretic decision-making results, different execution units can coordinate their timing around the same driving quality objective, thereby improving driving efficiency, ride comfort, vehicle stability, and energy economy under complex road conditions.

[0123] In the embodiments described in this invention, the vehicle cooperative control process can be directly applied to all road surfaces. That is, regardless of whether there are special road conditions and / or obstacles on the current road surface, the driving control system can continuously acquire driving status data and multi-source perception data, and execute control logic such as state feature representation, environmental feature representation, candidate joint control sequence generation, future state evolution prediction, multi-objective game equilibrium decision-making, and cooperative control of multiple execution units. In other words, the aforementioned steps 304-310 can be executed in all road surface scenarios.

[0124] Alternatively, considering that the control logic in steps 304-310 may consume significant vehicle resources, a conditional triggering mechanism can be adopted. This involves first determining whether special road conditions and / or obstacles exist ahead of the vehicle based on environmental characteristics; if special road conditions and / or obstacles are detected, then the generation of candidate joint control sequences and subsequent prediction, game theory, and execution processes are initiated; if no special road conditions and / or obstacles are detected, the original driving control state can be maintained. In other words, this scheme is applicable to both continuous cooperative control across the entire road segment and on-demand cooperative control under special road conditions.

[0125] In one embodiment, a conditional triggering mechanism based on special road conditions and / or obstacle detection can be employed, executing the processing logic of steps 304-310 only at appropriate times (i.e., when encountering special road conditions and / or obstacles). For example, when acquiring the driving state data and the multi-source perception data, the driving control system can be in automatic cruise mode (e.g., the ICC function of the driver assistance system is enabled). In this case, the presence of special road conditions and / or obstacles ahead of the vehicle can be identified based on the environmental feature characterization; then, if special road conditions and / or obstacles are identified, the generation of candidate sequences and subsequent steps are executed (i.e., after step 302, a decision is made based on the road condition identification result whether to execute step 304 and subsequent steps). Of course, if no special road conditions and / or obstacles are detected, the automatic cruise mode can remain unchanged. The automatic cruise mode refers to a driving mode in which the vehicle's longitudinal speed or following distance is automatically maintained by the driving control system, such as cruise control or adaptive cruise control. The special road conditions refer to road surface situations requiring additional coordinated control, such as muddy roads, waterlogged roads, icy roads, undulating roads, continuous slopes, and construction areas. Obstacles are objects on the road ahead that require additional coordinated control, such as speed bumps, potholes, gravel roads, slopes, protrusions, depressions, and foreign objects (such as fallen rocks and branches). Semantic segmentation, object detection, point cloud classification, road attribute recognition, and / or risk threshold judgment logic can be used to identify these special road conditions and obstacles. Maintaining the automatic cruise mode unchanged means not triggering candidate sequence generation and subsequent steps, and continuing to control the vehicle according to the current cruise target (e.g., continuing to control the vehicle according to ICC logic).

[0126] Taking an 80km / h automatic cruise as an example, when the road surface ahead is smooth and no special road conditions and / or obstacles are detected, the system can maintain the current cruise speed and following distance without initiating candidate sequence generation, world model prediction, or game-theoretic decision-making. When environmental characteristics indicate the presence of a speed bump approximately 10cm high or a deep pothole within 50 meters ahead, the system can determine the existence of special road conditions and / or obstacles and trigger the generation of multiple sets of candidate joint control sequences and subsequent steps. Subsequently, the system can select the target joint control sequence based on the future state evolution results and drive the cruise control unit, braking unit, suspension adjustment unit, or energy recovery unit to work together. After the vehicle passes through, the system can resume or maintain the automatic cruise mode.

[0127] This scheme can determine whether to initiate the cooperative control process based on the road conditions ahead. It not only helps reduce unnecessary calculations and driving interventions in normal cruise scenarios, but also promptly triggers candidate sequence generation, world model prediction, and game theory decision-making when special road conditions and / or obstacles appear. Furthermore, it helps maintain the continuity of the automatic cruise process, allowing the vehicle to maintain its original cruise state on regular road sections and enter targeted cooperative control on complex road sections.

[0128] Of course, in addition to real-time identification of special road conditions and / or obstacles, a navigation map pre-triggering scheme can also be adopted to determine in advance whether there are speed bumps, construction zones, or complex road sections ahead based on high-precision maps or navigation information. This scheme can reserve control preparation time in advance and help to smoothly trigger the collaborative control process.

[0129] Furthermore, during the process of traversing special road conditions or obstacles according to the target joint control sequence, the actual driving state of the vehicle may differ from the predicted result. Without feedback and judgment during execution, the system may struggle to determine whether the vehicle has escaped the influence of the special road conditions, or when to exit the special cooperative control and resume automatic cruise mode. To address this, a safe passage judgment and mode recovery mechanism based on driving state feedback can be employed. For example, during the process of controlling the vehicle's movement according to the target joint control sequence, the driving state feedback of the vehicle is detected; based on the driving state feedback, it is determined whether the vehicle has safely passed the special road conditions or obstacles; if it is determined that the vehicle has safely passed the special road conditions or obstacles, the automatic cruise mode is restored.

[0130] The driving state feedback refers to the set of information acquired during the execution of the target joint control sequence, reflecting the vehicle's actual driving response and road surface condition. This driving state feedback may include vehicle speed, longitudinal acceleration, lateral acceleration, vertical acceleration, vertical vibration acceleration, pitch angle, roll angle, yaw rate, suspension height, suspension travel, wheel speed, wheel slip ratio, braking pressure, drive torque, actual actuator response, and the vehicle's position relative to special road conditions or obstacles. It can be acquired through inertial measurement units (IMUs), wheel speed sensors, suspension height sensors, suspension travel sensors, cameras, lidar, millimeter-wave radar, positioning modules, CAN bus, in-vehicle Ethernet, or state estimators. Determining whether the vehicle has safely passed the special road condition or obstacle based on the driving state feedback refers to the process of determining whether the vehicle has left the area affected by the special road condition and returned to a stable driving state, based on the vehicle's actual driving response and the relative position of the obstacle. This determination can be achieved through rule-based judgment, threshold comparison, duration confirmation, state machine logic, multi-condition fusion judgment, or classification models. For example, the judgment conditions may include at least one of the following: obstacle disappearance, vehicle posture recovery, vibration index below a threshold, and stable driving state. Restoring the automatic cruise mode refers to the process of switching vehicle control from special road condition cooperative control mode back to automatic cruise control mode after determining that the vehicle has safely passed. This can be achieved through control permission switching, cruise target restoration, following distance restoration, smooth transition of target vehicle speed, or gradual change of actuator commands, to avoid significant speed fluctuations or vehicle posture disturbances during mode switching.

[0131] Taking the example of a vehicle approaching a speed bump in automatic cruise control mode. Assuming the vehicle is currently traveling at 80 km / h, and environmental characteristics indicate a speed bump approximately 10 cm high about 50 meters ahead, the system can first generate and execute a target joint control sequence. This sequence, coordinated by the cruise control unit, braking unit, suspension adjustment unit, and energy recovery unit, controls the vehicle to pass over the speed bump. During the process of controlling the vehicle according to the target joint control sequence, the system can continuously monitor driving status feedback. For example, the system can use multi-source sensing data to determine whether the speed bump is behind the vehicle or whether the vehicle has moved out of the speed bump's influence area; it can use the IMU and state estimator to detect whether the vehicle's pitch and roll angles have returned to normal ranges, such as a pitch angle less than 2° and a roll angle less than 3°; it can detect whether the root mean square value of the vehicle's vertical vibration acceleration is below a preset comfort threshold, such as less than 0.5 m / s²; and it can also detect whether the vehicle speed, longitudinal acceleration, lateral acceleration, and yaw rate have stabilized within normal ranges. If at least one or more of the above conditions are met within a preset duration, the system can determine that the vehicle has safely passed the speed bump. Subsequently, the system can resume automatic cruise control and gradually restore the vehicle speed to the original set speed or re-establish the following distance. If the driving status feedback indicates that the vehicle pitch angle, roll angle, vertical vibration acceleration, or yaw rate has not yet returned to the preset range, the system can continue to execute the current cooperative control, maintain a low vehicle speed, regenerate candidate joint control sequences, or wait for the vehicle status to stabilize before resuming automatic cruise control. For special road conditions such as consecutive potholes, consecutive speed bumps, or long stretches of gravel road, the system can also resume automatic cruise control only after the last obstacle has been passed and the driving status feedback meets the safe passage conditions.

[0132] As can be seen, this scheme can introduce driving status feedback judgment during the execution of the target joint control sequence. It can not only verify whether the vehicle has actually escaped the influence of special road conditions, but also avoid prematurely resuming the automatic cruise mode before the vehicle's posture or vibration has recovered. Furthermore, this scheme helps to promptly disengage the special cooperative control after confirming safe passage, reducing unnecessary control intervention, thereby balancing driving safety, passenger comfort, control continuity, and traffic efficiency.

[0133] Of course, besides judging safe passage based on driving status feedback, other schemes can be used for mode recovery. For example, a fixed-distance or fixed-time recovery scheme can be used, where the automatic cruise mode is resumed after the vehicle has passed a preset distance or time following a special road condition or obstacle. This scheme simplifies the judgment logic and helps maintain the stability of the mode switching process. Alternatively, a joint judgment scheme of perception re-check and status feedback can be used, first confirming that the special road condition or obstacle is behind the vehicle, and then combining vehicle attitude, vibration index, and driving stability to determine whether to resume automatic cruise mode. This scheme can improve the reliability of safe passage judgment and help reduce false or delayed recovery. A high-precision map or geofencing recovery scheme can also be used, resuming automatic cruise mode after the vehicle has exited the corresponding area of ​​a pre-marked speed bump, pothole, construction zone, or complex road section. This scheme can utilize prior map information to improve the predictability of the recovery timing and helps adapt to known special road sections. A risk scoring and replanning scheme can also be used, calculating the current passage risk based on driving status feedback, resuming automatic cruise mode when the risk is low, and regenerating candidate joint control sequences when the risk is high. This solution can adapt to changes in state during execution and helps improve control safety during passage through special road conditions.

[0134] Vehicles generate a large amount of data during passage through special road conditions or obstacles. This data can be used as empirical data for iterative training of the driving control system. However, given the limited computing resources on the vehicle side, it is difficult to directly train a large-scale driving control system based on this data. Furthermore, models trained in the cloud can be quite large, and directly deployed to the vehicle may not meet the requirements for deployment and real-time operation. Therefore, a mechanism combining cloud-based iterative training, lightweight processing, and OTA updates can be adopted. For example, empirical data generated during passage through the aforementioned special road conditions and / or obstacles can be collected, packaged, and uploaded to a model management server. This server then iteratively trains a copy of the driving control system deployed in the cloud simulation environment based on this empirical data and generates an OTA update package based on the iterated driving control system copy. The system then receives the OTA update package from the model management server and updates the locally deployed driving control system based on it. The OTA update package can be generated by lightweighting the iterated driving control system copy, and the parameter size of the updated driving control system is typically smaller than that of the iterated driving control system copy.

[0135] The empirical data may include process data and result data. The process data may include the aforementioned state feature representations, environmental feature representations, physical cognitive feature maps, energy impact potential fields, candidate joint control sequences, target joint control sequences, control commands of each execution unit, and execution states such as vehicle speed, acceleration, vehicle posture, and suspension status. The result data may include performance scores such as travel time, vibration indices, posture indices, and safe passage judgment results. The model management server is a cloud server (e.g., [missing information]) used to receive empirical data, manage training tasks, maintain model versions, perform simulation verification, and distribute upgrade packages. Figure 1 The model management server 13 shown is mentioned. The cloud simulation environment refers to a simulation platform deployed in the cloud for simulating vehicle driving processes under different driving scenarios. The driving control system replica refers to a large-scale driving control system deployed in the cloud simulation environment, which may include modules such as multimodal perception models, causal driving world models, game equilibrium decision models, and policy networks. Its parameter scale can be larger than that of the driving control system deployed locally on the vehicle. Iterative training is a process of continuously optimizing the driving control system replica based on empirical data. It may include data preprocessing, model training, simulation verification, and model evaluation, and can be implemented using supervised learning, reinforcement learning, or self-supervised learning. The OTA upgrade package is a software package delivered to the vehicle via a wireless network to update the vehicle's locally deployed driving control system. Lightweight processing is a method to reduce the model parameter scale or computational load, including but not limited to model pruning, model quantization, knowledge distillation, or network architecture search.

[0136] Taking a vehicle traversing a speed bump or a series of potholes as an example, the vehicle can collect experience data during the process. Process data can include state characteristic representations, environmental characteristic representations, physical cognitive characteristic maps, energy impact potential fields, multiple candidate joint control sequences, target joint control sequences, control commands from each execution unit, and execution states such as vehicle speed, vertical acceleration, pitch angle, roll angle, suspension height, and suspension travel. Result data can include travel time, vibration indices, attitude indices, and safe passage judgment results. The vehicle can upload this experience data to the model management server. The model management server can clean, denoise, standardize, and annotate the data to construct a training dataset and iteratively train a replica of the driving control system in a cloud simulation environment. For example, it can optimize the multimodal perception model's ability to identify the height and material of speed bumps, optimize the causal driving world model's ability to predict future state evolution, or optimize the game-theoretic equilibrium decision-making model's ability to balance comfort, stability, and efficiency. Subsequently, the cloud can verify the iterated replica of the driving control system in various simulation scenarios and evaluate indicators such as perception accuracy, prediction accuracy, decision optimality, and execution stability. After the evaluation is passed, the model management server can perform lightweight processing on the iterative copy of the driving control system. For example, it can generate a model suitable for vehicle deployment through model pruning, model quantization, knowledge distillation, or network architecture search, and then generate an OTA upgrade package and send it to the vehicle. After receiving the package, the vehicle can perform version and integrity verification, and update the locally deployed driving control system based on the OTA upgrade package.

[0137] This embodiment of the solution can form a closed-loop process of vehicle-side data acquisition, cloud-based iterative training, lightweight processing, and vehicle-side OTA updates. It not only enables continuous optimization of the driving control system using experience data from specific road conditions, but also allows for verification of iterative effects through a cloud simulation environment, reducing update risks. Furthermore, generating OTA upgrade packages through lightweight processing ensures that the updated local system parameters are smaller than the cloud system copy, thus facilitating the deployment of a higher-performance driving control system even with limited vehicle-side computing resources.

[0138] Furthermore, the collection of experience data is not limited to situations where the vehicle successfully passes through special road conditions or obstacles. Even when the vehicle fails to pass through special road conditions or obstacles—for example, when the vehicle body posture exceeds a preset range, vertical vibration index exceeds a comfort threshold, suspension travel approaches or reaches its limit, wheels exhibit significant slippage, the vehicle is forced to stop or detour, or the driver takes over control—the vehicle can still collect experience data generated during the passage process and upload this data to the model management server. This experience data can also include process data and result data. Process data can include state characteristic representations, environmental characteristic representations, physical cognitive characteristic maps, energy impact potential fields, candidate joint control sequences, target joint control sequences, control commands from each execution unit, and execution states such as vehicle speed, acceleration, body posture, and suspension status. Result data can include passage time, vibration index, posture index, safe passage judgment results, takeover records, or failure reason identifiers. The experience data from unsuccessful attempts can be used as negative samples or boundary samples in iterative training to optimize the multimodal perception model's ability to identify special road conditions, improve the causal driving world model's ability to predict future state evolution under extreme or critical conditions, and enhance the game equilibrium decision-making model's ability to balance comfort, stability, passability, and economy. Thus, successful experience data can be used to strengthen optimal control strategies, while unsuccessful experience data can be used to correct prediction biases and adjust evaluation parameters, contributing to improved adaptability and robustness of the driving control system under complex road conditions and critical conditions.

[0139] Of course, in addition to the aforementioned cloud-based iterative training and OTA updates, an on-vehicle incremental learning approach can also be adopted, where the vehicle locally fine-tunes certain modules of the driving control system based on limited experience data. This approach reduces data upload and communication overhead and helps to quickly adapt to the current individual state of the vehicle. Alternatively, a federated learning approach can be used, where multiple vehicles train model updates locally, upload the model updates to a model management server for aggregation, and then the server distributes the aggregated model updates. This approach leverages the experience of multiple vehicles to improve the model's generalization ability and helps reduce the privacy pressure caused by centralized uploading of raw driving data. A modular differential update approach can also be used, generating upgrade packages only for modules that have changed. This approach reduces the amount of OTA data and helps shorten the on-vehicle update process.

[0140] Figure 5 This is a flowchart illustrating another vehicle cooperative control method based on terrain perception, as shown in an embodiment of the present invention. The flowchart is... Figure 3 The illustrated method, in a specific embodiment, comprehensively covers the entire closed loop from system startup, continuous sensing, traffic condition triggering, collaborative control, feedback detection, and cloud-based iteration. For example... Figure 5As shown, the process includes steps 501-514.

[0141] Step 501: System initialization and self-test.

[0142] The process begins with "system power-on startup". After the system is powered on, initialization and self-test are performed first, including ICC (Intelligent Cruise Control) calibration, suspension setting to default mode, and large model preparation. This step ensures that all modules of the vehicle's driving control system are in normal working condition, providing a foundation for subsequent perception and control.

[0143] Step 502: Continuous perception and physical cognition of multimodal large models.

[0144] After initialization, the multimodal large model continuously performs environmental perception and physical cognition, outputting a physical cognition feature map F. This step corresponds to step 302 mentioned above, which determines the environmental feature representation of the current driving scenario based on the vehicle's multi-source perception data (LiDAR, camera, millimeter-wave radar, etc.) and generates a physical cognition feature map F to represent the distribution of physical attributes at various road surface locations in front of the vehicle. This step continues to run, constantly updating the physical cognition feature map.

[0145] Step 503: Determine whether any special road conditions / obstacles are detected in F.

[0146] The system determines whether there are special road conditions or obstacles ahead of the vehicle based on the physical cognitive feature map F. If the determination result is "no", the process arrow returns to step 502 to continue continuous perception and physical cognition; if the determination result is "yes", the subsequent collaborative control process is entered. This step corresponds to the aforementioned special road condition triggering mechanism, that is, in automatic cruise mode, candidate sequence generation and subsequent prediction, game and execution processes are only initiated when special road conditions or obstacles are detected; otherwise, the original cruise state is maintained.

[0147] Step 504: Accurate classification of road conditions.

[0148] After detecting special road conditions or obstacles, the system accurately classifies the road conditions, such as identifying them as speed bumps, potholes, gravel areas, muddy roads, undulating roads, and continuous slopes. The road condition classification results are used to generate more targeted candidate joint control sequences and to call large language models to obtain corresponding driving common sense suggestions.

[0149] Step 505: Generate multiple candidate joint control sequences U.

[0150] Based on the road condition classification results and the current vehicle status, the system generates multiple sets of candidate joint control sequences U. Each set of candidate sequences includes control commands corresponding to multiple execution units of the vehicle. This step corresponds to step 304 mentioned above. The candidate sequences can be generated by sampling, enumeration, optimization, policy network output, or rule generation.

[0151] Step 506: The common sense reasoning enhancement module calls the LLM to obtain driving common sense suggestions and constraints.

[0152] After generating candidate joint control sequences, the common sense reasoning enhancement module in the system calls LLM to obtain driving common sense suggestions and constraints. This step corresponds to the large language model encoding method in the aforementioned evaluation parameter determination method, that is, constructing a natural language description based on state feature representation and environmental feature representation, and calling LLM to encode the driving common sense suggestions into constraints and weight factors for subsequent game equilibrium solution.

[0153] Step 507: Jointly solve the world model deduction and game equilibrium.

[0154] The system invokes a causal driving world model to extrapolate the future states of multiple candidate joint control sequences, and then uses a multi-objective game equilibrium strategy to jointly solve the problem, calculating the target joint control sequence U. This step corresponds to steps 306 and 308 mentioned above, namely, firstly, the causal driving world model predicts the future state evolution results corresponding to each candidate sequence, and then the multi-objective game equilibrium strategy (such as Nash equilibrium solution, approximate Nash equilibrium solution, etc.) evaluates the comprehensive impact of each candidate sequence on the game subjects such as speed efficiency, driving comfort, and stability and safety based on the prediction results, and finally determines the target joint control sequence Uopt that satisfies the game equilibrium conditions.

[0155] Step 508: Send the control commands in Uopt to multiple execution units.

[0156] The system distributes each control command in the target joint control sequence Uopt to the corresponding execution unit, such as ICC (Intelligent Cruise Control), air spring, CDC (Continuously Variable Damping Damper), and steering. This step corresponds to step 310 above, achieving cross-domain coordinated control by simultaneously driving multiple execution units.

[0157] Step 509: All execution units execute in coordination.

[0158] Each execution unit coordinates its actions according to the received control commands, such as ICC adjusting vehicle speed, air springs adjusting vehicle height, CDC adjusting suspension damping, and the steering system correcting steering angle. Because these control commands originate from the same target joint control sequence, the actions of multiple execution units can be coordinated around the same prediction result and the same decision objective.

[0159] Step 510: Real-time status feedback detection.

[0160] During the collaborative execution of the execution units, the system collects real-time feedback on the vehicle's driving status, including parameters such as vehicle speed, attitude (pitch angle, roll angle), vibration (vertical acceleration), and suspension position. This step is used to monitor the actual state of the vehicle when passing through special road conditions or obstacles, providing a basis for subsequent safe passage judgments.

[0161] Step 511: Determine whether it is safe to pass through special road conditions.

[0162] The system determines whether a vehicle can safely pass through special road conditions or obstacles based on real-time status feedback. Judgment conditions may include at least one of the following: obstacle disappearance, vehicle posture recovery, vibration index below a threshold, and stable driving state. If the judgment result is "no," the process arrow returns to step 505, multiple candidate joint control sequences are regenerated, and subsequent processes are executed until the vehicle passes safely. If the judgment result is "yes," the process proceeds to step 512. Simultaneously, regardless of whether the judgment result is "yes" or "no," the process can proceed to step 513, where the experience data is uploaded to the cloud.

[0163] Step 512: Restore automatic cruise mode.

[0164] After determining that the vehicle has safely passed through the special road conditions or obstacles, the system can resume automatic cruise control, such as restoring the suspension to normal settings and the ICC to normal cruise control. Then, it returns to step 502 to continue continuous perception and physical recognition, waiting for the next trigger of special road conditions or obstacles.

[0165] Step 513: Package, verify, and upload the experience data to the model management server in the cloud.

[0166] Regardless of whether the vehicle passes safely, the system can package and verify the experience data generated during the vehicle's passage through special road conditions or obstacles, and upload it to the model management server in the cloud. This experience data can include process data (perception features, control commands, execution status, etc.) and result data (travel time, vibration indices, attitude indices, safe passage judgment results, etc.). Experience data can be collected and uploaded regardless of whether the vehicle successfully passes through the special road conditions; experience data from unsuccessful passages can be used as negative samples or boundary samples for iterative training.

[0167] Step 514: Cloud-based reinforcement learning evolution and OTA delivery to update vehicle-side model parameters.

[0168] Based on uploaded experience data, the cloud-based model management server iteratively trains a copy of the driving control system deployed in the cloud simulation environment (including reinforcement learning evolution and knowledge distillation), and generates an OTA upgrade package based on the iteratively trained system copy, which is then sent to the vehicle to update the vehicle-side model parameters. The updated vehicle-side model parameters are fed back to step 502, enabling the vehicle to use the updated model in subsequent perception and control, thus forming a complete closed-loop evolutionary process of vehicle-side data acquisition, cloud-based iterative training, lightweight processing, and vehicle-side OTA updates.

[0169] In summary, this complete process, through steps such as system initialization, continuous perception, road condition triggering, accurate classification, candidate generation, common sense enhancement, world model deduction and game equilibrium solution, multi-execution unit collaborative execution, real-time feedback detection, safe passage judgment, mode recovery, and cloud iteration and OTA update, constructs a full-link closed-loop collaborative control system covering perception, decision-making, execution, feedback and evolution, which can achieve intelligent, safe and comfortable vehicle driving control under all terrain conditions.

[0170] Figure 6 This is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Please refer to it. Figure 6 At the hardware level, the device includes a processor 601, a network interface 602, memory 603, non-volatile memory 604, and an internal bus 605, and may also include other hardware required for business operations. One or more embodiments of the present invention can be implemented in software, for example, the processor 601 reads the corresponding computer program from the non-volatile memory 604 into memory 603 and then runs it. Of course, in addition to software implementation, one or more embodiments of the present invention do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0171] Figure 7 This is a block diagram illustrating a terrain-aware vehicle cooperative control device according to an embodiment of the present invention. Please refer to... Figure 7 This device can be applied to, for example Figure 6 The device shown is used to implement the technical solution described in this invention. The device is applied to the aforementioned vehicle-mounted driving control system and includes: The feature determination module 701 is used to determine the state feature representation of the vehicle based on the vehicle's driving state data, and to determine the environmental feature representation of the current driving scenario based on the vehicle's multi-source perception data, wherein the environmental feature representation includes the road condition physical attributes under the current driving scenario. The sequence generation module 702 is used to generate multiple sets of candidate joint control sequences, each set of candidate joint control sequences including control instructions corresponding to multiple execution units of the vehicle; The state prediction module 703 is used to call the causal driving world model and predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representation and the environmental feature representation. The multi-objective game module 704 is used to evaluate the comprehensive impact of each candidate joint control sequence on multiple game subjects related to driving quality based on the future state evolution results through a multi-objective game equilibrium strategy, so as to determine the target joint control sequence that satisfies the game equilibrium condition. The collaborative control module 705 is used to drive the plurality of execution units respectively according to the control instructions in the target joint control sequence to control the vehicle's movement.

[0172] Optionally, the environmental feature representation includes a physical cognitive feature map, which is used to represent the distribution of physical attributes at various road surface locations in front of the vehicle.

[0173] Optionally, the physical cognitive feature map includes an energy impact potential field, which is used to characterize the energy distribution of the impact generated on the vehicle at a specific location on the road surface. The value of the energy impact potential field varies with the speed and / or attitude of the vehicle when it passes the specific location on the road surface.

[0174] Optionally, the feature determination module 701 is specifically used for one of the following: Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined, and the physical properties are input into a multimodal sensing model for inference to obtain the energy impact potential field output by the multimodal sensing model. Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined. The physical properties are then input into a multimodal sensing model for inference to obtain an initial potential field estimate. Finally, a physical calculation model is called to correct the initial potential field estimate based on the vehicle's dynamic parameters to obtain the energy impact potential field. Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined, and the physical properties are input into a physical calculation model. The physical calculation model then calculates the energy impact potential field based on the physical properties and the vehicle's dynamic parameters.

[0175] Optionally, the causal driving world model adopts a causal Transformer architecture to predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences in an autoregressive manner.

[0176] Optionally, the multiple game subjects related to driving quality include at least two of the following: The objectives are: speed efficiency with the shortest travel time; driving comfort with the lowest vehicle body vibration amplitude; stability and safety with the lowest vehicle body attitude angle; and driving economy with the lowest energy loss.

[0177] Optional, The multi-objective game equilibrium strategy includes Nash equilibrium solution strategy, Nash negotiation solution solution strategy, cooperative game benefit distribution strategy, or multi-objective collaborative solution strategy based on iterative optimization of the value functions of the multiple game subjects. The target joint control sequence is a candidate joint control sequence output by the multi-objective game equilibrium strategy that makes the comprehensive payoff of the multiple game subjects satisfy a preset equilibrium condition.

[0178] Optionally, the multi-objective game module 704 is specifically used for: Determine the evaluation parameters corresponding to the state characteristic representation and the environmental characteristic representation, wherein the evaluation parameters include constraints and / or weighting factors; Based on the future state evolution results, the comprehensive impact of each group of candidate joint control sequences on the multiple game agents is evaluated under the condition that the evaluation parameters are met.

[0179] Optionally, the multi-objective game module 704 is specifically used for: Query the evaluation parameters corresponding to the state characteristic representation and the environmental characteristic representation in the preset evaluation parameter table; or... A natural language description is constructed based on the state feature representation and the environmental feature representation, and a large language model is invoked to encode the driving common sense suggestions corresponding to the natural language description as the evaluation parameters.

[0180] Optionally, the plurality of execution units include at least two of the following: cruise control unit, power unit, braking unit, steering assist unit, suspension adjustment unit, and energy recovery unit.

[0181] Optionally, when acquiring the driving status data and the multi-source perception data, the driving control system is in automatic cruise mode, and the device further includes a condition triggering module 706, used for: Based on the environmental characteristics, identify whether there are special road conditions or obstacles in front of the vehicle; If the special road conditions or obstacles are detected, the generation of candidate sequences and subsequent steps are executed. If no special road conditions or obstacles are detected, the automatic cruise mode remains unchanged.

[0182] Optionally, a status feedback module 707 is also included, for: During the process of controlling the vehicle to drive according to the target joint control sequence, the driving status feedback of the vehicle is detected; Based on the driving status feedback, it is determined whether the vehicle can safely pass through the special road conditions or obstacles; If the vehicle is determined to have safely passed the special road conditions or obstacles, the automatic cruise mode is restored.

[0183] Optionally, an iterative training unit 708 is also included, for: Collect experience data generated during the process of passing through the special road conditions or obstacles, and upload the experience data to the model management server. The model management server will then iteratively train the driving control system copy deployed in the cloud simulation environment based on the experience data, and generate an over-the-air (OTA) upgrade package based on the iterated driving control system copy. The system receives the OTA upgrade package from the model management server and updates the driving control system deployed locally on the vehicle based on the OTA upgrade package.

[0184] Optionally, the OTA upgrade package is generated by lightweighting the iterative copy of the driving control system, and the parameter size of the updated driving control system is smaller than that of the iterative copy of the driving control system.

[0185] Accordingly, the present invention also provides a vehicle having a driving control system deployed thereon, the system being used for the methods described in any of the above embodiments.

[0186] Accordingly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0187] Accordingly, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any of the above embodiments.

[0188] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0189] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0190] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0191] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

Claims

1. A vehicle cooperative control method based on terrain perception, characterized in that, A driving control system applied to a vehicle includes: The vehicle's state characteristics are determined based on the vehicle's driving state data, and the environmental characteristics of the current driving scenario are determined based on the vehicle's multi-source perception data. The environmental characteristics include the physical attributes of the road conditions in the current driving scenario. Multiple sets of candidate joint control sequences are generated, and each set of candidate joint control sequences includes control instructions corresponding to multiple execution units of the vehicle. The causal driving world model is invoked to predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representation and the environmental feature representation. By employing a multi-objective game equilibrium strategy, the comprehensive impact of each candidate joint control sequence on multiple game agents related to driving quality is evaluated based on the future state evolution results, so as to determine the target joint control sequence that satisfies the game equilibrium condition. The plurality of execution units are driven according to the control instructions in the target joint control sequence to control the vehicle's movement.

2. The method according to claim 1, characterized in that, The environmental feature representation includes a physical cognitive feature map, which is used to represent the distribution of physical attributes at various road surface locations in front of the vehicle.

3. The method according to claim 2, characterized in that, The physical cognitive feature map includes an energy impact potential field, which is used to characterize the energy distribution of the impact generated on the vehicle at a specific location on the road surface. The value of the energy impact potential field varies with the speed and / or attitude of the vehicle when it passes the specific location on the road surface.

4. The method according to claim 3, characterized in that, Determining the energy impact potential field based on the multi-source sensing data of the vehicle includes one of the following: Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined, and the physical properties are input into a multimodal sensing model for inference to obtain the energy impact potential field output by the multimodal sensing model. Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined. The physical properties are then input into a multimodal sensing model for inference to obtain an initial potential field estimate. Finally, a physical calculation model is called to correct the initial potential field estimate based on the vehicle's dynamic parameters to obtain the energy impact potential field. Based on the multi-source sensing data, the physical properties of a specific location on the road surface are determined, and the physical properties are input into a physical calculation model. The physical calculation model then calculates the energy impact potential field based on the physical properties and the vehicle's dynamic parameters.

5. The method according to claim 1, characterized in that, The causal driving world model adopts a causal Transformer architecture and uses an autoregressive approach to predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences.

6. The method according to claim 1, characterized in that, The multiple game actors related to driving quality include at least two of the following: The objectives are: speed efficiency with the shortest travel time; driving comfort with the lowest vehicle body vibration amplitude; stability and safety with the lowest vehicle body attitude angle; and driving economy with the lowest energy loss.

7. The method according to claim 1, characterized in that, The multi-objective game equilibrium strategy includes Nash equilibrium solution strategy, Nash negotiation solution solution strategy, cooperative game benefit distribution strategy, or multi-objective collaborative solution strategy based on iterative optimization of the value functions of the multiple game subjects. The target joint control sequence is a candidate joint control sequence output by the multi-objective game equilibrium strategy that makes the comprehensive payoff of the multiple game subjects satisfy a preset equilibrium condition.

8. The method according to claim 1, characterized in that, The evaluation of the combined impact of each candidate joint control sequence on multiple game actors related to driving quality based on the future state evolution results includes: Determine the evaluation parameters corresponding to the state characteristic representation and the environmental characteristic representation, wherein the evaluation parameters include constraints and / or weighting factors; Based on the future state evolution results, the comprehensive impact of each group of candidate joint control sequences on the multiple game agents is evaluated under the condition that the evaluation parameters are met.

9. The method according to claim 8, characterized in that, The determination of the evaluation parameters corresponding to the state characteristic representation and the environmental characteristic representation includes: Query the evaluation parameters corresponding to the state characteristic representation and the environmental characteristic representation in the preset evaluation parameter table; or... A natural language description is constructed based on the state feature representation and the environmental feature representation, and a large language model is invoked to encode the driving common sense suggestions corresponding to the natural language description as the evaluation parameters.

10. The method according to claim 1, characterized in that, The plurality of execution units include at least two of the following: cruise control unit, power unit, braking unit, steering assist unit, suspension adjustment unit, and energy recovery unit.

11. The method according to claim 1, characterized in that, When acquiring the driving status data and the multi-source perception data, the driving control system is in automatic cruise mode, and the method further includes: Based on the environmental characteristics, identify whether there are special road conditions or obstacles in front of the vehicle; If the special road conditions or obstacles are detected, the generation of candidate sequences and subsequent steps are executed. If no special road conditions or obstacles are detected, the automatic cruise mode remains unchanged.

12. The method according to claim 11, characterized in that, Also includes: During the process of controlling the vehicle to drive according to the target joint control sequence, the driving status feedback of the vehicle is detected; Based on the driving status feedback, it is determined whether the vehicle can safely pass through the special road conditions or obstacles; If the vehicle is determined to have safely passed the special road conditions or obstacles, the automatic cruise mode is restored.

13. The method according to claim 11 or 12, characterized in that, Also includes: Collect experience data generated during the process of passing through the special road conditions or obstacles, and upload the experience data to the model management server. The model management server will then iteratively train the driving control system copy deployed in the cloud simulation environment based on the experience data, and generate an over-the-air (OTA) upgrade package based on the iterated driving control system copy. The system receives the OTA upgrade package from the model management server and updates the driving control system deployed locally on the vehicle based on the OTA upgrade package.

14. The method according to claim 13, characterized in that, The OTA upgrade package is generated by lightweighting the iterative copy of the driving control system, and the parameter size of the updated driving control system is smaller than that of the iterative copy of the driving control system.

15. A vehicle driving control system, characterized in that, include: The feature determination unit is used to determine the state feature representation of the vehicle based on the vehicle's driving state data, and to determine the environmental feature representation of the current driving scenario based on the vehicle's multi-source perception data, wherein the environmental feature representation includes the road condition physical attributes under the current driving scenario. A sequence generation unit is used to generate multiple sets of candidate joint control sequences, each set of candidate joint control sequences including control instructions corresponding to multiple execution units of the vehicle; The state prediction unit is used to invoke the causal driving world model and predict the future state evolution results corresponding to the multiple sets of candidate joint control sequences based on the state feature representation and the environmental feature representation. A multi-objective game unit is used to evaluate the comprehensive impact of each candidate joint control sequence on multiple game subjects related to driving quality based on the future state evolution results through a multi-objective game equilibrium strategy, so as to determine the target joint control sequence that satisfies the game equilibrium condition. The collaborative control unit is used to drive the plurality of execution units respectively according to the control instructions in the target joint control sequence, so as to control the vehicle driving.

16. A vehicle, characterized in that, It is equipped with a driving control system for implementing the method as described in any one of claims 1-14.

17. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method as described in any one of claims 1-14.

18. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the method as described in any one of claims 1-14.