Off-road automatic driving motion control method and device based on visual-force coupling hierarchical control

Abstract

This invention provides an off-road autonomous driving motion control method based on vision-force coupling hierarchical control, comprising: a slow-frequency perception decision layer that transforms the global state factors of the off-road environment into geometric factors, semantic factors, and force factors, and determines the spatial look-ahead distance based on the current vehicle speed, outputting the look-ahead geometric intent and brake caliper pre-charge signal; a mid-frequency strategy cerebellum that performs route matching and assigns control weights to different terrains; and a fast-frequency force-feedback execution layer that, based on the control weights, performs nonlinear model predictive control optimization in each control cycle, incorporates the braking execution delay into the state transition prediction equation, and uses a closed-loop adaptive Smith predictor to reconstruct the state in the look-ahead, obtaining optimized control commands through rolling optimization, and issuing them to the vehicle drive mechanism and braking mechanism to achieve closed-loop motion control of the vehicle. This invention also provides an off-road autonomous driving motion control device, storage medium, and electronic device based on vision-force coupling hierarchical control. Therefore, this invention can improve the stability and traction capability of off-road autonomous driving motion control.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving motion control and intelligent chassis control technology, and in particular to an off-road autonomous driving motion control method, device, storage medium and electronic device based on vision-force coupling hierarchical control. Background Technology

[0002] The wilderness environment is characterized by varied terrain (such as straight roads, steep slopes, and sharp bends), diverse materials (such as sand, mud, and rocks), and highly nonlinear dynamics (high risk of slippage and rollover). In the current field of autonomous driving and intelligent chassis control, there are two main technical solutions for motion control of all-terrain vehicles under complex road conditions: large-model end-to-end semantic planning systems and traditional chassis dynamics feedback control with constant time delay compensation.

[0003] Large-scale model end-to-end semantic planning systems (such as the VLM-MPC coupled architecture described in relevant public literature) mainly utilize a Vision-Language Model (VLM) or Vision-Language-Action (VLA) deployed on a Graphics Processing Unit (GPU) to perform end-to-end semantic reasoning on the road ahead through multimodal inputs (such as camera images and LiDAR point clouds). The principle is as follows: the large model identifies obstacles, road boundaries, and qualitative features of the terrain (such as "smooth road surface" or "bumpy road surface") in the environment, and outputs a sequence of target waypoints or a static cost map at the macro-semantic level. Subsequently, the underlying traditional Model Predictive Control (MPC) algorithm tracks these reference waypoints with a low execution frequency to achieve closed-loop adjustment of vehicle speed and steering.

[0004] Traditional chassis dynamics feedback control schemes with constant time delay compensation (such as patent CN113778074B, "Feedback Correction Method for Model Predictive Control of Autonomous Vehicles") mainly rely on classical feedback control algorithms deployed on the vehicle's central processing unit (CPU). To address the physical response lag of chassis actuators such as hydraulic braking systems (e.g., a hydraulic build-up dead zone of 100ms to 300ms), these schemes establish a linear or nominally nonlinear mechanistic model of the vehicle's state space containing a fixed time delay term, and employ a traditional Smith predictor for open-loop forward integration. This method uses high-frequency sensors to acquire measured values ​​of the vehicle's current state, calculates the deviation between the measured values ​​and the first-order static residual of the mechanistic model at the current moment, and directly feeds this first-order residual back to the next rolling cycle of the MPC for linear superposition correction, thereby eliminating tracking deviations caused by constant road friction variations or braking delays.

[0005] In conclusion, the existing technology obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the Invention

[0006] To address the aforementioned shortcomings, the present invention aims to provide an off-road autonomous driving motion control method, device, storage medium, and electronic device based on vision-force coupling hierarchical control, which can improve the stability and traction of off-road autonomous driving motion control.

[0007] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0008] In a first aspect, embodiments of the present invention provide an off-road autonomous driving motion control method based on vision-force coupling hierarchical control, comprising the following steps:

[0009] The slow-frequency perception step, based on the factorized Markov decision process, factorizes the global state of the off-road environment into geometric factors, semantic factors and force factors, and determines the spatial look-ahead distance according to the current vehicle speed, and outputs the look-ahead geometric intention and brake caliper precharge signal at the spatial look-ahead distance.

[0010] In the intermediate frequency routing step, the intermediate frequency strategy cerebellum receives the geometric factors, the semantic factors, and the force factors, and performs route matching and assigns control weights for different terrains in a predetermined expert policy library.

[0011] In the fast-frequency execution step, the fast-frequency force sensing execution layer performs nonlinear model predictive control optimization in each control cycle according to the control weight. The pure time delay step corresponding to the braking execution delay is explicitly written into the state transition prediction equation, and the state in the control cycle is reconstructed by a closed-loop adaptive Smith predictor. The optimized control command is obtained by rolling optimization.

[0012] The output step involves sending the optimized control command to the vehicle drive mechanism and braking mechanism to achieve closed-loop motion control of the vehicle.

[0013] According to the method of the present invention, in the slow-frequency sensing step, the global state is factorized by a dynamic Bayesian network, and its joint state transition probability satisfies the following conditional independence equation:

[0014]

[0015] in, Let S be the global state at the next moment, S be the global state at the current moment, and u be the control input; Geometric factor For semantic factors, For the i-th force perception factor, The force feedback factor is used as the input for macroscopic motion control. It includes at least the instantaneous friction coefficient between the wheel and the ground, the slip rate of the four wheels and the deviation of the yaw rate, and is updated in real time by the force feedback.

[0016] According to the method of the present invention, in the slow-frequency sensing step, the spatial look-ahead distance satisfies the calculation formula:

[0017]

[0018] The current vehicle speed, The total system delay is the sum of the large model inference delay and the hydraulic braking execution delay.

[0019] The brake caliper precharge signal is sent in advance before the vehicle arrives at the position corresponding to the spatial forward distance, pre-filling the empty stroke of the hydraulic line and compressing the effective pressure build-up of the brake caliper to within a predetermined time.

[0020] According to the method of the present invention, in the fast-frequency execution step, the state transition prediction equation is in a discretized nonlinear form, specifically:

[0021]

[0022] in, Let be the vehicle state vector at the k-th control step. The motor drive torque at step k is... The vehicle braking request is issued d steps in advance, where d is the pure lag step number, which is equal to the ratio of the hydraulic braking execution delay to the control cycle; the motor drive torque acts on the vehicle dynamic response in real time, while the braking force is determined by the control command issued d steps in advance.

[0023] According to the method of the present invention, in the fast execution step, the process of the closed-loop adaptive Smith predictor performing look-ahead reconstruction of the state specifically includes:

[0024] Calculate the delay window residual of the current control step, where the delay window residual is the difference between the actual vehicle state measured by the sensor and the predicted value of the mechanism model for the current moment before the pure delay step.

[0025] By using a time-varying adaptive gain matrix that is dynamically adjusted according to terrain semantics, the residual of the delay window is exponentially decayed in a multi-step forward extrapolation to obtain the error compensation amount for each step in the prediction time domain.

[0026] The error compensation amount is superimposed on the basic prediction value of the mechanism model to obtain the reconstructed prediction state, which is then input into the nonlinear model prediction control loop to participate in rolling optimization.

[0027] According to the method of the present invention, in the fast-frequency execution step, the target cost function of the nonlinear model predictive control includes a high-weighted quadratic penalty term for the rate of change of braking force, and the weight of the penalty term is significantly greater than the cost weight of the motor torque; by severely penalizing the step-by-step change of braking force, the temporal smoothness of the braking command is constrained, so that during the lag window period when hydraulic braking is not effective, the transient dynamic behavior of the vehicle is preferentially adjusted by the motor torque.

[0028] According to the method of the present invention, after the fast execution step and before the execution output step, the method further includes:

[0029] Feedforward feedback closed-loop step: The forward geometric intention and the brake caliper precharge signal are transmitted in advance as feedforward quantities to the fast frequency execution step, and the real-time status feedback of the fast frequency execution step is used to update the control weight of the intermediate frequency routing step and the force perception factor of the slow frequency sensing step.

[0030] In a second aspect, embodiments of the present invention provide an off-road autonomous driving motion control device based on vision-force coupling hierarchical control constructed according to any one of the methods described above, the device comprising:

[0031] The slow-frequency perception module is used to factorize the global state of the off-road environment into geometric factors, semantic factors and force factors based on the factorized Markov decision process. It also determines the spatial look-ahead distance based on the current vehicle speed and outputs the look-ahead geometric intention and brake caliper precharge signal at the spatial look-ahead distance.

[0032] The intermediate frequency routing module is used to receive the geometric factors, the semantic factors, and the force factors in the intermediate frequency policy cerebellum, and to perform route matching and assign control weights for different terrains in a predetermined expert policy library.

[0033] The fast-frequency execution module is used by the fast-frequency force sensing execution layer to perform nonlinear model predictive control optimization in each control cycle according to the control weight. The pure time delay step number corresponding to the braking execution delay is explicitly written into the state transition prediction equation, and the state in the control cycle is reconstructed by a closed-loop adaptive Smith predictor. The optimized control command is obtained by rolling optimization.

[0034] The execution output module is used to send the optimized control commands to the vehicle drive mechanism and braking mechanism to realize closed-loop control of vehicle motion.

[0035] Thirdly, embodiments of the present invention provide a storage medium for storing a computer program for performing any of the methods described herein.

[0036] Fourthly, embodiments of the present invention provide an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement any of the methods described above.

[0037] This invention relates to an off-road autonomous driving motion control technology based on vision-force coupled hierarchical control, comprising: a slow-frequency perception decision layer that transforms the global state factors of the off-road environment into geometric factors, semantic factors, and force factors, and determines the spatial look-ahead distance based on the current vehicle speed, outputting the look-ahead geometric intent and brake caliper pre-charge signal; a mid-frequency strategy cerebellum that performs route matching and assigns control weights to different terrains; and a fast-frequency force-feedback execution layer that, based on the control weights, performs nonlinear model predictive control optimization in each control cycle, incorporates braking execution delay into the state transition prediction equation, and uses a closed-loop adaptive Smith predictor to reconstruct the state in the look-ahead, obtaining optimized control commands through rolling optimization, and issuing them to the vehicle drive mechanism and braking mechanism to achieve closed-loop motion control of the vehicle. This invention also provides an off-road autonomous driving motion control device, storage medium, and electronic device based on vision-force coupled hierarchical control. Through this, the invention achieves beyond-line-of-sight feedforward inference through a large model (data-driven), and implements delay compensation and dynamic reshaping through low-level NMPC (mechanism-driven), thereby improving the stability and traction capability of off-road autonomous driving motion control. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the off-road autonomous driving motion control method based on vision-force coupling hierarchical control provided in Embodiment 1 of the present invention.

[0039] Figure 2 This is the overall system architecture diagram provided in Embodiment 2 of the present invention;

[0040] Figure 3 This is a schematic diagram of the spatiotemporal look-ahead alignment principle provided in Embodiment 3 of the present invention;

[0041] Figure 4 This is a dynamic force balance diagram of (single motor + open differential) under unilateral slippage provided in Embodiment 4 of the present invention;

[0042] Figure 5 This is a block diagram of a low-level closed-loop Smith predictor with error feedback correction provided in Embodiment 5 of the present invention.

[0043] Figure 6 The data flow diagram of low-level 200 Hz NMPC rolling optimization provided in Embodiment 6 of the present invention (including Smith predictor and delay stage).

[0044] Figure 7 This is the torque redistribution timing diagram under the differential lock simulation primitive provided in Embodiment 7 of the present invention;

[0045] Figure 8 This is the main flowchart of the vision-force coupling hierarchical control provided in Embodiment 8 of the present invention.

[0046] Figure 9 This is a schematic diagram of the off-road autonomous driving motion control device based on vision-force coupling hierarchical control provided in Embodiment 9 of the present invention;

[0047] Figure 10 This is a schematic diagram of the structure of the electronic device provided in Embodiment 10 of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0049] It should be noted that references to "an embodiment," "embodiment," "example embodiment," etc., in this specification refer to the described embodiment including specific features, structures, or characteristics, but not every embodiment must include these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge of those skilled in the art.

[0050] Furthermore, certain terms are used in the specification and subsequent claims to refer to specific components or parts. Those skilled in the art will understand that manufacturers may use different names or terms to refer to the same component or part. This specification and subsequent claims do not distinguish components or parts by differences in name, but rather by differences in function. The terms "comprising" and "including" used throughout the specification and subsequent claims are open-ended and should be interpreted as "including but not limited to." Additionally, the term "connection" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connections made through other means.

[0051] The following description, in conjunction with the accompanying drawings, details the off-road autonomous driving motion control method based on vision-force coupling hierarchical control provided by the present invention through specific embodiments and application scenarios.

[0052] After in-depth research, this invention discovered that the two existing technologies mentioned above suffer from spatial and temporal dispersion and divergence problems when facing diverse, extreme, and highly real-time off-road autonomous driving motion control, as well as the divergence and collapse problems of traditional static error correction under heterogeneous conditions. Under extreme off-road conditions, due to the extended inference time of the large VLM / VLA model (e.g., over 100ms) and its inability to perceive the true mechanical properties of the ground, when the large model outputs an obstacle avoidance or deceleration trajectory, the vehicle has already physically passed that feature point. At this point, the underlying MPC's forced tracking inevitably leads to a severe misalignment between the control signal and the real physical space, causing severe overshoot, oscillation, or even direct loss of control and rollover of the chassis. Simultaneously, the prediction equations of traditional MPC and static Smith predictors rely on tire models (such as the Magic Formula) and fixed road adhesion coefficients. However, the extremely heterogeneous nature of off-road terrain (random alternation of sand, mud, and rocks, with combinations of steep slopes and sharp bends) results in high modeling uncertainty in the mechanistic model. Under this condition, using only the "first-order static residual at the current moment" for superposition correction will not only fail to decouple the error on the forward time axis, but will also amplify exponentially during the braking delay vacuum period, ultimately causing the low-level nonlinear model predictive control (NMPC) optimizer to diverge, and the vehicle to get stuck in a slippage dead loop or become stuck.

[0053] This invention deconstructs the failure mechanism of the "vision + chassis" joint closed loop, revealing that the fundamental flaw in existing technologies lies in the failure to hierarchically coordinate high-frequency (200Hz+) force feedback and slow-frequency (10Hz~20Hz) large-model visual look-ahead within a unified time axis and state space. This results in the two operating independently within the control system, leading to severe system dispersion. To address this issue, this invention proposes a hierarchical control method and system with vision-force coupling. Through a large model (data-driven), beyond-line-of-sight feedforward inference is achieved, while low-level NMPC (mechanism-driven) performs delay compensation and dynamic reshaping, thereby improving the stability and traction of off-road autonomous driving motion control.

[0054] Figure 1 This is a flowchart illustrating the off-road autonomous driving motion control method based on vision-force coupling hierarchical control provided in Embodiment 1 of the present invention. The method includes the following steps:

[0055] Step S101, slow frequency perception step: The slow frequency perception decision layer based on the factorized Markov decision process factorizes the global state of the off-road environment into geometric factors, semantic factors and force factors, and determines the spatial look-ahead distance according to the current vehicle speed, and outputs the look-ahead geometric intention and brake caliper precharge signal at the spatial look-ahead distance.

[0056] Preferably, the global state is factorized using a dynamic Bayesian network, and its joint state transition probabilities satisfy the following conditional independence equation:

[0057]

[0058] in, Let S be the global state at the next time step, S be the global state at the current time step, and u be the control input. Geometric factor For semantic factors, For the i-th force perception factor, This serves as the input for macroscopic motion control. The force feedback factors include at least the instantaneous coefficient of friction between the wheels and the ground, the slip ratio of the four wheels, and the deviation of the yaw rate, which are updated in real time by the force feedback.

[0059] Preferably, the spatial look-ahead distance satisfies the calculation formula:

[0060]

[0061] The current vehicle speed, The total system delay is the sum of the large model inference delay and the hydraulic braking execution delay.

[0062] The brake caliper precharge signal is sent in advance before the vehicle arrives at the corresponding position in the spatial forward distance, pre-filling the empty stroke of the hydraulic line, and compressing the effective pressure build-up of the brake caliper to within the predetermined time.

[0063] Step S102, intermediate frequency routing step: The intermediate frequency strategy cerebellum receives geometric factors, semantic factors and force factors, and performs route matching and assigns control weights for different terrains in a predetermined expert policy library.

[0064] Step S103, fast frequency execution step: The fast frequency force sensing execution layer performs nonlinear model predictive control optimization in each control cycle according to the control weight. The pure time delay step corresponding to the braking execution delay is explicitly written into the state transition prediction equation. The state in the control cycle is reconstructed by the closed-loop adaptive Smith predictor, and the optimized control command is obtained by rolling optimization.

[0065] Preferably, the state transition prediction equation is in discretized nonlinear form, specifically:

[0066]

[0067] in, Let be the vehicle state vector at the k-th control step. The motor drive torque at step k is... The braking request for the entire vehicle is issued d steps in advance, where d is the pure lag step number, equal to the ratio of the hydraulic braking execution delay to the control cycle. The motor drive torque acts instantly on the vehicle's dynamic response, while the braking force is determined by the control command issued d steps in advance.

[0068] Preferably, the process of the closed-loop adaptive Smith predictor performing look-ahead reconstruction of the state specifically includes:

[0069] Calculate the delay window residual for the current control step. The delay window residual is the difference between the actual vehicle state measured by the sensor and the predicted value of the mechanism model for the current moment before the pure delay step.

[0070] By using a time-varying adaptive gain matrix that dynamically adjusts according to terrain semantics, the residual of the delay window is exponentially decayed through multi-step forward extrapolation to obtain the error compensation amount for each step in the prediction time domain.

[0071] The error compensation is superimposed on the basic prediction value of the mechanism model to obtain the reconstructed prediction state, which is then input into the nonlinear model predictive control loop to participate in rolling optimization.

[0072] Preferably, the objective cost function of the nonlinear model predictive control includes a high-weighted quadratic penalty term for the rate of change of braking force, with the weight of the penalty term significantly greater than the cost weight of the motor torque. By heavily penalizing the step-by-step change in braking force, the temporal smoothness of the braking command is constrained, ensuring that during the lag window when hydraulic braking is not in effect, the transient dynamics of the vehicle are preferentially adjusted through motor torque.

[0073] Step S104: Execute the output step, send the optimized control command to the vehicle drive mechanism and braking mechanism to realize the closed-loop control of vehicle motion.

[0074] Preferably, after the fast execution step and before the output step, the method further includes:

[0075] Feedforward feedback closed-loop steps: The forward geometric intent and brake caliper precharge signal are transmitted as feedforward quantities to the fast frequency execution step, and the real-time status feedback of the fast frequency execution step is used to update the control weight of the intermediate frequency routing step and the force perception factor of the slow frequency sensing step.

[0076] This invention discloses an off-road autonomous driving motion control system and method based on Vision-Force Coupled Hierarchical Control (VFCHC). The system is deployed on an automotive-grade chip and is divided into a three-layer asynchronous multi-frequency architecture: a slow-frequency perception and decision layer (10Hz~20Hz, using a large visual-language model (VLM) or a large visual-language-action model (VLA) on the GPU) factorizes the off-road scene into "geometric factors". semantic factors Force factor "Three types of subsets and output" The spatial distance look-ahead geometry and the corresponding brake caliper hydraulic pre-fill signal; the mid-frequency policy cerebellum (50 Hz~100 Hz, expert policy library SMP on the CPU) controls the sticky routing output for different terrains; the fast-frequency force perception execution (200 Hz+, explicit delay-compensated NMPC optimizer on the real-time kernel) explicitly writes the pure hysteresis of hydraulic braking into the prediction equation, and adopts "time-varying adaptive gain with delay window residual". The closed-loop Smith predictor, employing "multi-step feedforward error attenuation prediction," performs state look-ahead reconstruction. For the extremely limited and low-cost chassis of a "single motor + open differential," this invention further proposes a control primitive that uses "the vehicle's main brake caliper + the motor's excess positive torque" to reverse-force the open differential, simulating differential lock-up. This solves the traditional problem of all driving force being transferred to the slipping wheel by the open differential during unilateral slippage in off-road driving, achieving safe closed-loop vehicle control in highly heterogeneous off-road scenarios with low-cost hardware.

[0077] Keywords: Vision-force coupled hierarchical control; off-road autonomous driving; Factored MDP; Delay-compensated Smith predictor; Nonlinear Model Predictive Control (NMPC); Single-motor open differential; Differential lock simulation; Brake caliper hydraulic prefilling.

[0078] Key terms and abbreviations Abbreviations / Terms Full English name Chinese meaning and specific definition in the context of this invention VFCHC Vision-Force CoupledHierarchical Control Vision-force coupled hierarchical control; the three-layer asynchronous multi-frequency control architecture of "large model visual look-ahead + force feedback" proposed in this invention. VLM / VLA Vision-Language Model / Vision-Language-ActionModel Visual-language model / Visual-language-action model; running on the GPU of the in-vehicle intelligent chip, outputting semantic factors. With forward-looking intentions. Primitive Primitive / ControlPrimitive Primitives / Control Primitives; in the context of this invention, a primitive refers to a named, parameterizable, and indivisible sub-control law in NMPC or SMP that can be called by name by higher layers. Examples: differential lock simulation primitive, steep slope climbing primitive, pre-fill braking primitive. Factored MDP Factored Markov DecisionProcess Factorized Markov Decision Process; This invention explicitly decomposes the global state of the off-road into Three types of sub-factors. DBN Dynamic Bayesian Network Dynamic Bayesian Networks (DNBs) are used to decouple the state transition probabilities of Factored MDPs based on conditional independence. / / Geometric / Semantic / Physical-Force Factor Geometric factors (slope, roll, obstacle height, pit depth) / Semantic factors (terrain type) / Force factors (instantaneous friction coefficient, four-wheel slip rate, yaw rate deviation). NMPC Nonlinear ModelPredictive Control Nonlinear model predictive control; this invention performs rolling optimization under a 200 Hz+ period, N=40 prediction time domain, and Δt=5 ms discrete step length. SmithPredictor Smith Predictor (closed-loop adaptive form) Smith predictor; this invention employs "time-varying adaptive gain with delayed window residuals". The closed-loop adaptive form of "+ multi-step feedforward error attenuation prediction" is used. Pre-fill Brake Caliper HydraulicPre-fill Hydraulic pre-filling of brake calipers: The hydraulic system is pre-filled ≤ 150 ms in advance after the visual forward signal arrives and before the vehicle reaches the heterogeneous terrain. SMP Skill / Policy MacroPrimitive (Expert PolicyLibrary) Mid-frequency expert strategy library; routing control weights based on terrain type (obstacle crossing / drift / smooth). / / GPU inference / Actuatorlatency / Total systemlatency Large model inference delay (≈100 ms) / hydraulic brake execution delay (≈150 ms) / total system delay (≈250 ms). d Delay Step Count Pure time delay steps; d = / = 150 / 5 = 30 steps, explicitly enter the NMPC prediction equation. Brake Clip Force Single-sided brake caliper clamping force; this invention uses asymmetric... Reverse "hold" open the differential to simulate a differential lock. Brake Force Total braking force Wheel Slip Ratio Wheel slip ratio, defined as This invention uses it as One of the continuous physical factors. Local Estimated Tire-RoadFriction Coefficient Instantaneous local estimation of friction coefficient; obtained from real-time observations at 200 Hz+ feedback, entering... Update.

[0079] The purpose of this invention is to address the problem that existing technologies fail to achieve hierarchical collaborative design of fast-frequency force feedback and slow-frequency large-model visual look-ahead within a unified time axis and state space. This invention proposes a hierarchical control method and system with vision-force coupling. Through a large model (data-driven), beyond-line-of-sight feedforward inference is achieved, while low-level NMPC (mechanism-driven) performs delay compensation and dynamic reshaping, thereby improving the stability and obstacle-avoidance capability of off-road autonomous driving motion control.

[0080] I. System Overall Architecture and Spatiotemporal Alignment Features

[0081] This invention proposes a Vision-Force Coupled Hierarchical Control (VFCHC) system, characterized by employing an asynchronous multi-frequency architecture:

[0082] Slow-frequency perception brain (high-level, 10Hz-20Hz): Running on the GPU of an automotive intelligent chip, its end-to-end model inference latency is... Using visual / LiDAR input, the off-road environment is decoupled into geometric factors and semantic-physical factors based on Factored Markov Decision Process (FactoredMDP).

[0083] Mid-frequency strategy cerebellum (middle layer, 50Hz-100Hz): Runs on the CPU of the in-vehicle intelligent chip. It allocates control weights based on hybrid expert policy (SMP) and sticky routing mechanism.

[0084] High-frequency physical execution (low-level, 200Hz+): Runs in the real-time kernel of the onboard intelligent chip. Constructs a nonlinear model predictive control (NMPC) optimizer with explicit actuator delay compensation, whose control cycle is... .

[0085] The present invention is characterized by explicitly defining the total physical delay of the system:

[0086] High-rise large model based on current vehicle speed It focuses its visual forward vision on the space in front of the vehicle. The system feeds forward to the ground at the location to predict the terrain type and expected slip ratio range, and triggers a pre-fill signal for the chassis calipers 150ms in advance to eliminate hydraulic free travel.

[0087] II. Definition of State Space and Spatiotemporal Transition Network in High-Level Large Model Factored MDP

[0088] The slow-frequency perception brain described in this invention will perceive the global state of the off-road environment. Explicit decoupling factorization is converted to geometric factor semantic factors With force perception factor The set of conditionally independent factors.

[0089] (1) Definition of state factor set

[0090] Geometric factors : These represent the instantaneous longitudinal slope, lateral tilt angle, height of the obstacle ahead, and depth of the pit, estimated by LiDAR (Light Detection and Ranging) and vision, respectively.

[0091] semantic factors : Represents discrete terrain semantic tags, whose value range is defined as .

[0092] Force Factor : These represent the instantaneous maximum adhesion coefficient of the wheel-to-ground contact in the low-level reverse thrust, the real-time slip ratio of the four wheels, and the yaw rate deviation, respectively.

[0093] (2) Causal Conditional Transition Graph (Dynamic Bayesian Network)

[0094] Using a dynamic Bayesian network to factorize the state transition probabilities, the joint state transition probabilities satisfy the following conditional independence equation:

[0095] The physical constraint of this equation lies in the geometric factor at the next moment. Depends only on the current geometric configuration and the vehicle's macroscopic motion input; semantic factors at the next moment. It possesses spatial continuity and is only affected by the current position and current semantics; while the force perception factor at the next moment... It is highly coupled to the current terrain geometry, terrain semantics, the current slip state of the wheel, and the instantaneous actuator input of the chassis. The high-level large model calculates the future within each slow-frequency cycle using the aforementioned factorization model. Joint conditional probability distribution within the time limit.

[0096] III. Low-level NMPC Delay Compensation State Equation

[0097] Suppose the system at the current time The state vector is (These represent longitudinal vehicle speed, lateral vehicle speed, yaw rate, and four-wheel angular velocity, respectively).

[0098] The control input vector is ,in This is the motor drive torque vector. The total braking force request is for the entire vehicle.

[0099] Given the existence of hydraulic braking systems The pure time delay has a corresponding control step lag term as follows: step.

[0100] The discretized nonlinear state transition look-ahead prediction equation constructed in this invention is as follows:

[0101]

[0102] The physical meaning of this equation is: at the current moment drive motor torque Immediately generate a dynamic response in the next moment (time delay) (approximately 0); while the mechanical braking force acting on the vehicle body at the current moment depends on... Vehicle braking request issued 150ms prior to the step .

[0103] IV. NMPC Optimization Objective Function Design

[0104] To offset braking delay and protect the bus from control dispersion, the NMPC solver performs prediction in the time domain during each control cycle. (set up The time-domain rolling optimization is performed within a 200ms look-ahead period. Its objective function is... Defined as:

[0105]

[0106]

[0107]

[0108]

[0109] Braking rate of change penalty Set a higher weighting coefficient Through severe punishment This forces the system to output a smooth, deterministic global deceleration trend in the time domain.

[0110] Motor drive matrix Set a smaller weight so that When the hydraulic brake has not yet taken effect (within the 150ms blind zone), the system prioritizes the use of the extremely fast-responding negative torque of the motor to eliminate high frequency and quickly counteract transient slippage.

[0111] V. Adaptive Error Correction Mechanism for Low-Level Smith Predictors

[0112] Due to the heterogeneity of off-road terrain, the mechanism prediction model Modeling uncertainties and time-varying ground disturbances are inevitable. This invention proposes a closed-loop Smith predictor mechanism with forward integration by a state observer and adaptive compensation for multi-step errors.

[0113] (1) Calculation of residuals in the delay window

[0114] In the current control step Define the chassis absolute modeling residual vector for the current step. for:

[0115]

[0116] in, This represents the actual vehicle body posture and force perception state measured by the sensors in the current step. For the system in The unbiased prediction of the current moment is made using the mechanistic model.

[0117] (2) Time-varying adaptive filtering and multi-step forward error prediction

[0118] This invention constructs a time-varying adaptive gain matrix For future prediction within the time domain ( For each step of the predicted state, the forward error is extrapolated:

[0119]

[0120] In the formula, This is the time-domain decay factor. Based on the current semantic factors Dynamic adjustment.

[0121] (3) Reconstruction and decoupling of Smith prediction states

[0122] During rolling optimization, the predicted state vector in the state constraint equations of the low-level NMPC solver is explicitly reconstructed as follows:

[0123]

[0124] in, It is the full state vector obtained by direct forward integration based on the current driving torque assumption from the 200Hz real-time mechanism model without considering the 150ms lag.

[0125] (4) Decoupling observer mechanism for known caliper disturbance: This mechanism particularly emphasizes the "known disturbance decoupling" processing of the underlying force feedback observer: due to the single-sided caliper force issued by the differential lock simulation primitive It is a command quantity known to the controller itself, rather than an unknown external disturbance. Therefore, EKF operates at 200 Hz based on the wheel-end dynamics equation. Inverse ground adhesion coefficient When, the known terms With known terms (Motor torque is distributed via differential) The residual force is the ground reaction torque to be estimated, which is then explicitly subtracted from the measured wheel speed change rate. Thus we obtain This "known disturbance decoupling" design ensures that even at the moment of differential lock simulation primitive intervention (when a single caliper experiences a reverse resistance torque of hundreds of N·m), the observer's estimate of ground adhesion remains uncontaminated by the self-applied caliper force, preventing the logical closed-loop conflict of "being unable to distinguish whether the resistance comes from the ground or from the caliper's self-clamping." This mechanism is a standard design practice in ABS / EBD / EDL / TCS industrial controllers for nearly 20 years. This invention adopts and propagates this semantic feedback channel to the larger model within the Smith predictor + NMPC state look-ahead framework.

[0126] The technical effects of this invention include:

[0127] I. Deterministic High-Frequency Physical Safety: The asynchronous multi-frequency mechanism mathematically decouples the large model inference (100ms) from the underlying control. If the large model experiences occasional stuttering, the lower-level MPC can still maintain the vehicle's attitude at 200Hz+ based on discrete nonlinear state equations, providing a safety baseline.

[0128] II. Elimination of Dual Delay of Model and Actuator: By introducing [a certain method / mechanism] into the optimization objective function The high-weight penalty forcibly constrains the frequent jitter of the 150ms lag actuator. Combined with caliper pre-charge, multi-step forward error prediction and state look-ahead equation, the control dispersion and overshoot risks caused by response lag and environmental modeling uncertainty are eliminated from the control mechanism.

[0129] III. Reshaping the basic mechanism of the restricted chassis: This invention can not only work under the ideal hardware of four-wheel distributed independent drive, but also enable the traditional chassis architecture of "single motor + open differential + 150ms braking delay" which has commercial cost-effectiveness but high physical limitations to have the ability to control the vehicle and get out of trouble in extreme heterogeneous off-road conditions through "simulated differential lock primitives".

[0130] Example 1

[0131] Single motor + open differential topology

[0132] This embodiment (and the entire text) strictly distinguishes the following three types of physical quantities:

[0133] (i) Clamping force (Unit: N, the normal clamping force of the piston pushing the brake pads against the brake disc, which is the direct control output of the ESP HCU proportional valve, and is the industry standard symbol).

[0134] (ii) Caliper reverse braking torque (Unit: N·m, obtained from the clamping force through geometric conversion of the friction disc) ,in The friction coefficient of the friction plate is . m is the effective working radius of the brake disc, 2 is the number of pistons in a single caliper, and a typical conversion factor. m);

[0135] (iii) Ground longitudinal braking force (Unit: N, the reaction force of braking torque transmitted from the tire to the contact patch on the ground) ,in (This refers to the wheel's rolling radius).

[0136] In this embodiment and in subsequent embodiments involving values ​​such as 200 N·m, the reverse braking torque of the caliper is consistently referred to. (Dimensional N·m), and its corresponding clamping force kN (typical single caliper pressure in passenger vehicles, compliant with the 0~10 MPa output range of ESP HCU proportional valve); The following text is reserved. Symbols with units marked in kN still refer to clamping force and do not require conversion.

[0137] Actuator constraints: The entire vehicle is driven by a single range-extended motor via an open differential; the braking system is controlled by an automotive-grade ESP hydraulic control unit (HCU) with 4 independent proportional valves to achieve independent pressure adjustment for each caliper (i.e., industry-standard ABS / EBD / EDL hardware base, complying with the mandatory configuration requirements of GB 21670 / FMVSS 126 for passenger car ESC), with a physical delay of approximately 150 ms for hydraulic pressure build-up in a single caliper. The longitudinal braking interface exposed by the upper-level NMPC meets the scalar deceleration requirements of the entire vehicle. The HCU's built-in EBD distribution law distributes pressure to all four wheel ends; the differential lock simulation primitive of this invention injects an additional unilateral pressure command on top of this distribution law. (Applies only to the slipping caliper, not the high-adhesion side). Control input degenerates into a scalar pair. Additional single-sided caliper pressure It will be issued separately from the original language.

[0138] Timing Constraints: Activation of the differential lock simulation primitive requires simultaneous fulfillment of two timing prerequisites; neither can be omitted. Prerequisite one addresses the spatial dimension (making unilateral clamping possible), while prerequisite two addresses the temporal dimension (ensuring unilateral clamping and motor torque take effect synchronously). Prerequisite 1: Caliper hydraulic pre-fill is complete—the high-level VLM / VLA large model triggers a pre-fill signal 100-150ms beforehand based on the ground material recognition result at the visual look-ahead focus area D, pre-pressurizing the hydraulic chamber of the slipping caliper to 0.2-0.5 MPa (only eliminating hydraulic free stroke, not yet generating significant clamping force), compressing the equivalent delay from "command issuance" to "actual clamping force establishment" of the unilateral caliper from ~150 ms to ~5 ms. Prerequisite 2: Motor torque The injection must use a smooth S-shaped slope obtained from NMPC, derived from the cost function. The medium-weighted smoothing penalty term is enforced (typical slope upper limit 8000 N·m / s), eliminating the risk of "motor pulse torque + caliper not yet clamping" being injected into the slip wheel during the window period when the pre-fill is not yet completed and the caliper force has not been established, causing the slip ratio to diverge instantaneously.

[0139] Primitive activation and pre-fill signal initiation are determined by three non-exclusive criteria, with routing selected by the SMP middle-layer policy layer, and dynamic switching is allowed:

[0140] Scenario 1 (High-level semantic feedforward, VLM / VLA dominant): The VLM / VLA large model determines that there is a high risk of slippage at the location based on the ground material recognition results (such as ice, wet mud, lateral ditches, water accumulation and snow cover) at the forward visual focus area D of the vehicle. The SMP directly triggers the pre-fill signal to the corresponding side caliper, completing the hydraulic pressure build-up 100~150 ms in advance. This scenario is a typical beyond-line-of-sight feedforward strategy, which relies on the large model semantics to perceive the unknown working conditions ahead in advance.

[0141] Scenario 2 (NMPC Model Prediction Dominant): NMPC jointly predicts future scenarios based on terrain semantic labels (geographic information) issued by a large model and the vehicle's current kinematic / dynamic state within a 200 Hz rolling time domain. Slip ratio of a wheel end at a certain moment within a step Will cross the upper limit The NMPC issues the corresponding side caliper pre-fill command to the ESP HCU in advance; this is a mechanism-driven model prediction strategy, independent of the real-time output reliability of the large model.

[0142] Scenario 3 (Prediction + Real-time Correction Closed Loop, an extension and enhancement of the first two scenarios): Based on the pre-triggering of Scenario 1 or Scenario 2, the low-level force feedback observer continuously monitors the actual slip ratio and wheel speed at the wheel end: if the vehicle does not slip after reaching the predicted position ( Persistently below Da Frame), SMP immediately releases the pre-fill pressure (pressure in the caliper hydraulic chamber is released to 0 MPa) to avoid ineffective parasitic braking; if slippage is confirmed ( (If the threshold spikes), then NMPC injects clamping force into the smooth ramp according to this control law. This activates the complete differential lock simulation primitive. This scenario, through a prediction-verification-correction closed loop, retains the low-latency advantage of the look-ahead strategy while avoiding unnecessary braking losses caused by false alarms in large models.

[0143] Simulated differential lock primitive control law: When the force feedback observer (wheel speed + IMU via EKF back-calculates the ground adhesion coefficient at 200 Hz) ̂) Tire slip ratio captured on the slipping side (left, L) When the left wheel speed abnormally spikes (power loss occurs), the lower-level NMPC activates the differential lock simulation primitive. Within a look-ahead prediction window (d=30 steps, i.e., 150 ms), the NMPC jointly solves the coordinated timing of the three execution channels: ① Through the ESP HCU single-channel proportional valve, an additional clamping force command is issued only to the slipping (left) caliper. (The right wheel caliper on the high-adhesion side remains idle.) = 0), the reverse resistance torque is established by this single-sided caliper.

[0144] ② Align with the caliper pre-fill completion time, command the single motor to output the target drive torque with a smooth ramp-increase in NMPC. (||Δ) || Medium-weighted smoothing constraint (non-step pulse) to ensure equal torque constraints on both sides of the open differential. = = / 2 Stable establishment within 5~10 ms; ③ Upper-level scalar braking interface It only handles the normal longitudinal deceleration requirements of the entire vehicle and does not participate in the torque redistribution of the differential lock simulation primitive. The drag torque that the slipping left wheel end can withstand is equal to the upper limit of ground friction. Reaction torque of unilateral caliper The sum; with the caliper completely idle, the right wheel end, thanks to its high adhesion coefficient , will be obtained through synchronization via open differential The entire amount is converted into net propulsion force on the ground.

[0145] Physical laws of open differentials: = = / 2; Torque balance on slip side: Right wheel net thrust: .

[0146] Torque balance numerical example (typical off-road condition, used to demonstrate the physical feasibility of open differential + single-side caliper + single motor): taking the vertical load of the tire. = 6 kN, rolling radius = 0.35 m, the left wheel on the slipping side is on the ice. The right wheel on the high-adhesion side is located on hard soil. Case A is "Primitives not activated (original open differential)": Left wheel ground bearing capacity = 0.05 × 6000 × 0.35 ≈ 105 N·m, from... = physical laws Upper limit ≤ 210 N·m, right wheel net thrust = 105 N·m, left wheel →1. The program spins in place. Case B is "activating this primitive." = 200 N·m (single side): The total resistance torque that the left wheel can withstand = ground 105 + caliper 200 = 305 N·m, obtained from the physical laws of open differentials. = = 305 N·m; Right wheel = 305 N·m is entirely used for ground propulsion (far below the right wheel's upper limit of 0.70×6000×0.35 ≈ 1470 N·m, and the right caliper consumes no torque when idle), net propulsion = 305 N·m; 105 N·m of the left wheel is consumed by ground friction, and 200 N·m is absorbed as heat by the left caliper. The effective propulsion force of the whole vehicle ≈ (305 + 105) / 0.35 ≈ 1170 N, the total torque required by the motor = 2 × 305 = 610 N·m (injected by NMPC in an S-shaped slope over tens of milliseconds, rather than a literal pulse, to avoid impact on the half-shaft / differential planetary gears).

[0147] Therefore, the right wheel on the non-slip side receives the synchronous load from the open differential when the caliper is completely idle. Under full torque conditions, thanks to its high adhesion To generate effective net propulsion on the ground = / The equivalent "electronic differential lock" function is implemented on the drive end to complete the escape from trouble—this is the complete closed-loop logic of the differential lock simulation primitive described in this invention. Throughout the entire process: the right wheel caliper on the high-adhesion side remains idle (…). = 0), no parasitic braking reaction torque is generated; the slipping left wheel caliper only bears the "artificially constructed reverse resistance to lift". The "constraint upper limit" function does not participate in vehicle deceleration; the whole vehicle scalar deceleration interface Decoupled from this primitive, it only handles the routine longitudinal deceleration requirements. The primitive's start / stop criteria are determined by the observer. Real-time cross-threshold triggering, NMPC in ||Δ || and ||Δ Under the combined constraint of high-weight smoothing penalties, the low-frequency hunting surge pattern of "slippage-stagnation-release-slippage again" is naturally avoided.

[0148]

Example 2

[0149] Dual motor (single motor on the front axle + single motor on the rear axle) + open differential topology

[0150] Actuator constraints: The front axle and the rear axle are each driven by an independent motor. Each axle still has an open differential (i.e., there is a differential between the left and right wheels within the axle). (Constrained by physical laws such as torque); the braking system uses the automotive-grade ESP HCU 4-channel independent pressure regulating hardware described in Example 1 (capable of independently applying pressure to each caliper), with a single caliper hydraulic pressure build-up delay of approximately 150 ms. A range-extended powertrain (range extender / engine + generator + high-voltage battery) supplies power to the dual motors. The control input is expanded to a scalar quaternion. (in (for the target power generation of the range extender), add 4 single-wheel caliper pressure commands. The primitives in the inner-axis sublayer of this embodiment are issued separately.

[0151] Two-tier control architecture (addressing the critical issue of single-wheel slippage in open differentials within the shaft): In this embodiment, torque control is achieved by NMPC jointly solving two levels under the same objective function; neither level can be omitted. The first level, "inter-shaft vector distribution," handles the "whole-shaft overall slippage / center of gravity shift" condition, where NMPC dynamically redistributes the total motor torque between the front and rear axles. The second layer, "Internal Differential Lock Simulation," directly inherits and reuses the "Single-sided Caliper Pre-fill + Single-sided" configuration described in Example 1. The primitive "+ motor smooth ramp increase" is extended from "single motor + single open differential" to "dual motor + front and rear open differentials" for a total of 4 wheel positions (FL / FR / RL / RR). The two levels are solved simultaneously in the 200 Hz rolling time domain of NMPC: the first level's dominant torque redistribution occurs when the observer only identifies "uneven axle load"; the second level's redistribution occurs when the observer simultaneously identifies "a certain single-angle wheel..." "At that time, the second layer of instantaneous intervention applied to the single-angle caliper." This architecture explicitly opposes the extreme assumption that "getting out of trouble in cross-axle / single-wheel suspension conditions can be achieved solely through inter-axle vector distribution of the motor shaft"—motor torque redistribution can only suppress "overall shaft slippage," and cannot create unilateral adhesion out of thin air under the constraint of an open differential within the shaft; single-wheel slippage within the shaft must be provided by the primitives of Example 1 (Pre-fill + unilateral EDL) to "artificially create unilateral reverse resistance to lift." The hard solution mechanism with "constraint upper limit".

[0152] State quantity acquisition division of labor and large model role boundaries (clarifying high / low layer signal sources to avoid contamination of chassis high-frequency control by the 100 ms delay of the VLM / VLA large model): This embodiment strictly separates two types of signal sources. The first type is "real-time feedback quantity" (high frequency, ≥ 200 Hz, calculated directly by complementary filtering / EKF from the onboard 6-axis IMU + 4 wheel speed sensors + suspension displacement sensor): longitudinal slope Lateral tilt Dynamic vertical loads on front and rear axles 4-wheel real-time slip ratio Ground adhesion coefficient estimation These quantities directly enter the NMPC's state equations and constraint sets, without going through any GPU large model chain. The second type, "look-ahead semantic quantities" (low frequency, 10 Hz, output from VLM / VLA large model inference via camera + LiDAR, end-to-end latency approximately 100 ms): future... Terrain semantic labels at distance (e.g., Loose_Rock / Wet_Mud / Cross_Axle_Rut), future road surface material transition boundary location, and terrain macro-mode labels (e.g., mud climbing, gravel off-roading, snow obstacle avoidance). The second type of quantity is only used for: (a) triggering SMP expert policy switching, thereby adjusting the NMPC cost function weights. With upper limit of slip ratio constraint (b) Trigger the caliper pre-fill signal 100-150ms in advance. Type II quantities never enter the NMPC real-time feedback loop; even if phantoms or token jitter occur in the large model output, the NMPC's 200 Hz underlying closed loop can still independently maintain vehicle attitude stability based on Type I quantities, providing a safety baseline.

[0153] Inter-axle torque vector distribution law: Real-time calculation of dynamic vertical loads on the front / rear axles by a low-level 200 Hz observer. and (Longitudinal acceleration calculated by IMU) ,slope Overall vehicle yaw angle Based on quasi-static allocation back-calculation (period 5 ms), the middle-layer SMP activates the "center of gravity transfer expert" when the terrain pattern identified by the large model is "muddy slope / gravel climbing," outputting the NMPC cost weights. With the upper and lower limits of inter-shaft torque ( NMPC performs continuous optimization of the front and rear axle torques under the following hard constraints, rather than any discrete rule switching or instantaneous step increase: (i) Motor peak torque saturation constraint (As given in the motor specifications, typical 420 N·m / single motor); (ii) Motor peak power saturation constraint Similarly, rear; (iii) Bus transient power constraint (See the next item "Bus Energy Transient Balance" for details); (iv) Overall Axis Slip Ratio Constraints (v) Smoothness constraint and Entering the cost function eliminates step transitions. Under the typical operating condition of "rearward shift of vehicle center of gravity, lighter front axle, and overall axle slippage," the natural result of NMPC's solution is: within the limits of saturation and power constraints, transfer as much of the distributable torque released from the front axle as possible to the rear axle (typically up to ~1.5~1.8 times the original ratio, depending on...). and The real-time margin is dynamically determined, and there is no discrete rule switching of "forced hard-write 180%"; at the same time, SMP will "in-axis The "single wheel slippage" criterion is pushed down to the second-level "intra-axle differential lock simulation primitive" processing. Mechanical braking (150 ms lag) is not "reduced in dependence," but rather decelerated longitudinally from the whole vehicle. Repositioned as an in-axle single-wheel limited-slip differential The latter, which is compressed to an equivalent delay of ~5 ms by the Pre-fill timing described in Example 1 (see the next item "Inter-axle differential lock simulation primitive call"), is an irreplaceable hard-break mechanism in this example under extreme road conditions (cross-axle, single wheel suspension, severe uneven adhesion within the axle).

[0154] Bus transient energy balance constraint (addressing the transient energy mismatch in the range-extended powertrain where the range extender's power reduction is slower than the motor's redistribution): The range-extended dual-motor architecture must explicitly model the transient energy balance of the high-voltage bus. Otherwise, the electrical energy released by the front axle motor during emergency torque reduction will have nowhere to be absorbed when the rear axle motor's absorption margin is insufficient, leading to bus overvoltage triggering the vehicle's emergency protection, or conversely, forcing the range extender (internal combustion engine + generator) to momentarily cut off fuel, causing severe speed fluctuations or even stalling. This embodiment explicitly adds a transient power balance equation to the NMPC constraint set: , in Provides power buffering for battery charge and discharge (millisecond-level response, subject to current SOC and temperature limits). (Two-way hard constraint) Power generation of the range extender (response time constant) (ms, much slower than the ~5 ms timescale of motor redistribution). NMPC uses this to set the target power of the range extender. As a "slowly variable control variable," it is smoothly distributed according to the average energy demand within the predicted time domain, allowing the battery to absorb the millisecond-level transient difference completely. When the battery SOC approaches its upper limit (> 85%), the NMPC automatically tightens the upper limit of the transient gradient for inter-axis torque redistribution. This constraint suppresses peak demand on the battery's absorption side, avoiding the energy trap of "front axle releasing too quickly / rear axle unable to handle it / battery fully charged / range extender unable to keep up." It ensures that the inter-axle torque vector distribution will not cause the range extender to enter abnormal operating conditions under any transient conditions.

[0155] In-axle differential lock simulation primitive call conditions (push-down and reuse of primitives in dual-motor architecture in Example 1): When the low-level force feedback observer detects "severe uneven adhesion of left and right wheels / single wheel suspended / single wheel stuck" on a certain axle at 200 Hz and meets the trigger criterion (single wheel corner)... Furthermore, the wheel speed on that side experienced an abnormal surge, and the entire axle... If the problem is still within an acceptable range (indicating the issue is inconsistency within the shaft rather than overall shaft slippage), the NMPC will invoke the "differential lock simulation primitive" described in Example 1 on that shaft: (Step a) 100-150 ms in advance, the VLM / VLA large model triggers a pre-fill signal for the single-angle caliper based on terrain semantics; (Step b) After the trigger criterion is met, the ESP HCU single-channel proportional valve issues a reverse clamping force command only to the single-angle caliper. (The caliper on the other side of the same axle remains) (Step c) The shaft motor ( or Under the NMPC smooth ramp constraint, the corresponding target torque is synchronously injected, utilizing the iron law of open differentials. This releases the thrust from the high-adhesion side. The synergistic effect of the two-level control is reflected in the following: the first level, "inter-axle vector distribution," determines the total torque allocation for each axle; the second level, "intra-axle differential lock simulation," determines whether the torque within each axle can be effectively applied. Both are jointly solved by NMPC under a unified objective function, eliminating inter-level competition or timing conflicts. This embodiment thus covers all typical off-road extrication conditions, from uneven axle load (where the inter-axle solution is sufficient) to severe uneven adhesion on the left and right sides of a single axle / cross-axle / single wheel suspension (where inter-axle + intra-axle cascading is necessary).

[0156]

Example 3

[0157] Distributed independent drive (four-wheel wheel-side / hub motor) topology

[0158] Actuator constraints: Each of the four wheels is driven by a separate wheel-side / hub motor (peak torque per motor). Peak power of a single motor Current loop bandwidth Torque response corresponding to Hz The braking system uses the automotive-grade ESP HCU 4-channel independent proportional valve described in Example 1 (with independent pressure adjustment capability for each caliper, and a single caliper hydraulic pressure build-up delay of approximately 150 ms). The control input is extended to a "full-on state" scalar seven-element group. It should be noted that the unsprung mass of this topology is significantly increased due to the hub / wheel-side motor (typically). (kg / wheel), the resulting trade-off in suspension dynamics and ride comfort is weighed by the overall vehicle design, and this invention does not compensate for this unsprung mass trade-off at the control layer.

[0159] Four-wheel four-quadrant force perception closed-loop primitive: Under this ideal topology, when a single wheel is suspended and slipping, the observer... Within milliseconds, the disappearance of wheel-end resistance is detected (determined jointly by the sudden increase in wheel speed and the zeroing of the wheel-end reaction torque calculated by the force feedback EKF). The NMPC then commands the single-wheel motor to enter the fourth quadrant electric braking state (back EMF negative torque, output controlled by the NMPC). (Smoothing cost constraints to avoid half-shaft torsional vibration excitation), actively synchronize the wheel speed to The project aims to mitigate the equivalent slip ratio impact at the moment of re-landing, rather than to "lock the slip ratio in the air" (when suspended). Physically (Undefined). Under normal off-road extrication conditions, this topology can complete the extrication without issuing a 150 ms delayed hydraulic request to the braking system; it only works in scenarios involving continuous high-power electric braking (such as long downhill slopes + battery SOC nearing its limit, with reverse charging margin). When there is insufficient torque (e.g., mechanical braking intervenes as an auxiliary safety measure). When the intermediate route reaches the "high-speed drift obstacle avoidance expert", the system uses the torque difference on the coaxial side (e.g., and )exist Within milliseconds, a direct yaw moment (DYC) is applied to the vehicle to actively coordinate with the steering. When solving for four motors in a coordinated manner, NMPC explicitly adheres to: (i) single-motor saturation constraints. (ii) Single motor power saturation (iii) Transient power constraint of high-voltage bus (Consistent with Example 2, "Bus Energy Transient Balance"), to prevent situations where "4 motors simultaneously demand peak power, causing a sudden drop in bus voltage" or "battery reverse charging peak exceeds..." "Instability mode, etc. Vehicle mechanical braking." This transformed it into a purely security defense line, rather than a conventional execution channel.

[0160]

Example 4

[0161] Single motor + open differential, force-sensing control for low-speed climbing on steep slopes (gradient ≥ 25°).

[0162] Actuator constraints: This embodiment is the same as Embodiment 1—the entire vehicle has one central drive motor driven via an open differential, with electro-hydraulic brake calipers distributed to each of the four wheels (front, rear, left, and right). The overall vehicle weight... kg, center of gravity height m, wheelbase m, peak torque of the motor N·m. Total system delay ms.

[0163] Operating condition description: The vehicle is... Climb slowly for a while at m / s Steep slope, the first 6 m is compacted gravel ( The last 3 m were then converted to compacted hard soil. Physical feasibility verification: Gravity component along the inclined plane N; Maximum thrust that the road surface can withstand N N, with a margin of about 7% used to overcome rolling resistance and disturbances, so this working condition is physically achievable (even without additional anti-slip measures, the vehicle will not "stop before slipping"). If traditional throttle-brake open-loop control is used, the vehicle will get stuck in gravel sections due to instantaneous slip rate overshoot, and wheel-ground impact will occur in hard soil sections due to sudden changes in drive.

[0164] Implementation process of vision-force coupling hierarchical control:

[0165] Step 1 (10 Hz low-frequency sensing, t=0): VLM / VLA large model self- ms Start inference (input) (Camera + LiDAR image at ms time), in Output to front of the vehicle Semantic look-ahead prediction at the location. Among them m is a safety margin used to avoid the near-field blind spot of the front-facing camera (typical blind spot radius). m); in m / s, Under the condition of s, take m, actual forecast distance At distance m, ground material identification is primarily driven by LiDAR (reliable depth information at long distances), with assistance from the front-facing camera (semantic texture classification). This step outputs: (a) Semantic factors. (a) The transition boundary is located 2.375 m in front of the vehicle; (b) Approximately before the vehicle reaches the transition boundary. Within the margin of s, a Pre-fill signal is sent to the left and right brake calipers of the front axle (hydraulic target 0.6 MPa, stroke pre-fill 8 mm), delaying the effective hydraulic pressure build-up of the caliper on that side from ms compression to ms.

[0166] Step 2 (100 Hz IF strategy, t=20 ms): The SMP expert policy library is routed to a hybrid weighted combination of "low-attached loose climbing experts" and "hard-attached steady-state transition experts." This hybrid weighting is not pre-hardcoded but is determined by the SMP based on (a) the large model. semantic tags Output confidence (b) The remaining distance between the current parking space and the transition boundary. (c) Current measured overall shaft slip ratio The difference from the nominal upper limit is calculated online; its typical value under this operating condition is... SMP thus outputs the NMPC cost function weight combination. With upper limit of slip ratio .

[0167] Step 3 (200 Hz+ high-frequency force sensing execution, t = 25 ms~250 ms rolling): NMPC in the prediction time domain In-step state trajectory With control trajectory , Perform continuous optimization solutions with explicit constraints. and N·m, add to the cost function and Smoothing penalty term; and at the same time because Step (150 ms) pure time delay is written into the prediction equation The solution obtained by NMPC The trajectory naturally exhibits a pattern of "maintaining a certain position within approximately 25 steps before the transition boundary". After the transition boundary, it gradually rises to within approximately 25 steps. The continuous smooth shape (without any discrete rule switching or artificial shaping) corresponds to the solution. Linear unloading begins 6 steps before the transition, ensuring that the residual braking pressure at the start of the hard soil section is exactly zero, thus avoiding wheel-ground impact.

[0168] Step 4 (200 Hz+ force feedback, entire process): The Smith predictor uses... Reflecting actual measurements instantaneous changes ( (Based on the back-digging of 4 wheel speeds + IMU via EKF at 200 Hz), the following is given: Multi-step forward-looking revision, and according to Reconstruct the predicted state of NMPC.

[0169] Implementation effect: During the climbing process, the maximum instantaneous slip ratio of the four wheels was constrained by NMPC. Within (traditional MPC will appear under this condition) The wheels spin freely; the impact of the material transition without wheels (the peak of longitudinal acceleration from the moment the wheels spin freely); m / s² decreased (m / s²); total time was reduced by 12% compared to traditional MPC, and no anti-trap rollback was triggered.

[0170] Example 5

[0171] Single motor + open differential, 35 km / h off-road high-speed emergency collision avoidance (VLM / VLA forward-looking + pre-fill braking + differential lock simulation simultaneous operation)

[0172] Actuator constraints: Same as in Example 1.

[0173] Operating condition description: The vehicle is... km / h Driving straight on a dirt road at a speed of m / s, a fallen tree suddenly appears about 8.5 m ahead (about 0.5 m high, blocking about 80% of the driving surface). You need to... Complete three actions within seconds: maximum deceleration, 0.8 m right-side obstacle avoidance, and suppression of open differential torque leakage. Total system delay. The ms (ms) account for 28% of the action window. Traditional MPC will overshoot due to the 150 ms hydraulic lag, triggering ESC emergency intervention and causing the vehicle to lose control and fishtail. This embodiment uses a three-piece set of VLM / VLA visual look-ahead (pre-filling 100~150 ms in advance) + NMPC smooth ramp + differential lock simulation primitive (reusing embodiment one) to simultaneously absorb the total delay, completing the right turn acceleration and deceleration without triggering ESC emergency intervention.

[0174] Implementation process of vision-force coupling hierarchical control:

[0175] Step 1 (10 Hz low frequency, t=0): VLM / VLA self MS starts inference, in Output to front of the vehicle Semantic look-ahead in the range of m: geometric factor semantic factors The planned trajectory was simultaneously switched to a combination of "0.8 m right turn + simultaneous full braking." This was based on the vehicle dynamics principle that "a right turn requires greater braking force on the right side than on the left to generate a rightward yaw moment" (yaw moment). , (Based on wheelbase), immediately send a Pre-fill signal to both front right and rear right brake calipers (target hydraulic pressure 1.4 MPa, stroke pre-fill 12mm), delaying the effective hydraulic pressure build-up of these two channels from... ms compression to ms; Left front / left rear calipers remain idle (because the left side requires less deceleration force to maintain yaw direction during right-hand maneuvers).

[0176] Step 2 (100 Hz intermediate frequency, t=20 ms): SMP switches to "High-speed emergency collision avoidance expert" and outputs NMPC cost. Emphasis on lateral stability (weight) (Raised from a nominal value of 1.0 to 8.0), and the permissible longitudinal deceleration was increased. from m / s² relaxed to m / s².

[0177] Step 3 (200 Hz+ low frequency, lasting from t=25 ms): NMPC solves for the frequency at... Target braking force of the four calipers at milliseconds They are respectively kN (the right side is much larger than the left side to generate a rightward yaw moment) (In conjunction with active steering assist for right-hand maneuvering). Due to With pure time delay, the solver directly writes this set of target force commands into the solution. (Its role in the prediction equation) Step 1), that is, "immediately issue" the chassis execution channel. Using the 12 mm pre-fill stroke already filled in the right front / right rear hydraulic lines from Step 1, the two main force-bearing calipers... Pressure can be built up to the target in milliseconds, with the equivalent hydraulic hysteresis decreasing from 150 milliseconds to [the desired value]. ms (negligible relative to the 900 ms action window). The two non-primary force-bearing calipers on the left front / left rear retain their original 150 ms delay due to the lack of pre-fill, but because the target forces are small (0.4 kN, 0.6 kN), their hysteresis contributes negligible to the overall yaw moment.

[0178] Step 4 (Differential lock simulation, starting from t=50 ms): During deceleration, the vehicle load shifts forward, and the left front wheel (about to pass over the fallen tree debris) It jumped instantaneously to 0.42. Detected by NMPC. Activate the differential lock simulation primitive (same as the torque balance mechanism added in Implementation Example 1): Apply an additional single-sided reverse clamping force to the left front wheel caliper. kN, the corresponding caliper reverse braking torque N·m; (Right front wheel caliper) (Keep idle in drive direction); simultaneously command the single motor in NMPC Injecting target driving torque with an S-shaped ramp under medium-weighted smoothing constraints (satisfies saturation hard constraint) N·m, there is no "transient exceeding 15%" excess situation). According to the ironclad rule of open differentials. :Pick N·m, then N·m; the net ground thrust of the left front wheel is N (single wheel), the entire 200 N·m of the right front wheel is converted into net propulsion force on the ground. N (single wheel), total net thrust of both front drive wheels N. This positive thrust maintains longitudinal control of the vehicle during the right turn, and works in conjunction with the yaw moment of the right main brake caliper to complete the obstacle maneuver.

[0179] Implementation Results: Under the combined conditions of "right-hand maneuver + deceleration," the vehicle completes a 0.8 m rightward lateral movement and clears the fallen tree, without the rigid requirement to come to a complete stop within 8.5 m (obstacle avoidance logic takes precedence over braking). If the right-hand maneuver is disabled and only a pure braking test is performed as a reference, the braking distance of this invention under the same hydraulic system and pre-fill conditions is reduced from 12.3 m in the traditional MPC to 6.8 m (a reduction of 44.7%); yaw stability margin peak from Reduce to (Comparison under the same obstacle avoidance amplitude); the entire process had no ESC emergency intervention or ABS lock-up oscillation, and the differential lock simulation primitive intervention did not cause considerable longitudinal jerking (forward acceleration peak). ).

[0180] Figure 2 This is the overall system architecture diagram provided in Embodiment 2 of the present invention, showing the asynchronous flow of the three layers of slow frequency, medium frequency, and fast frequency.

[0181] Figure 2 is a system overall architecture diagram provided in Embodiment 2 of the present invention, showing the asynchronous flow of the slow frequency, medium frequency, and fast frequency layers. The figure illustrates the three-layer asynchronous multi-frequency architecture of the Vision-Force Coupled Layered Control (VFCHC) system, from top to bottom as follows:

[0182] 1. Slow-frequency perception brain (high-level, 10Hz-20Hz, in-vehicle intelligent chip GPU)

[0183] • Visual / LiDAR multimodal input: End-to-end model inference latency τ_gpu=100ms, utilizing cameras and LiDAR to collect off-road environment information.

[0184] • Factored MDP model: Based on factorized Markov decision process, the off-road environment is decoupled into geometric factors, semantic factors, and force-physical factors.

[0185] • VLM / VLA large model beyond-line-of-sight feedforward inference: outputs look-ahead geometric intent and brake caliper precharge signal

[0186] 2. Mid-frequency strategy cerebellum (middle layer, 50Hz-100Hz, in-vehicle intelligent chip CPU)

[0187] • SMP Hybrid Expert Strategy Library: Includes various expert strategies such as escape, drift, and stabilization.

[0188] • Sticky routing mechanism: Adaptive control weight allocation, performing route matching and assigning control weights for different terrains.

[0189] 3. High-frequency physical execution (low-level, 200Hz+, real-time kernel of automotive intelligent chip).

[0190] • Wheel-to-ground force feedback observer: 200Hz transient slip and force observation, real-time update of force feedback factors.

[0191] • Explicit Delay Compensation NMPC: Smith predictor state reconstruction, which incorporates braking execution delay into the state transition prediction equation.

[0192] • Chassis actuator: Asymmetric decoupling control law, outputting optimized control commands.

[0193] 4. Chassis physical actuators: Single / multi-motor drive and vehicle braking to achieve closed-loop control of vehicle motion.

[0194] The cross-frequency control signal path includes: [Signal 1] Feedforward deceleration / caliper precharge cross-frequency control signal path (slow frequency → fast frequency); [Signal 2] Expert routing weight (intermediate frequency → fast frequency).

[0195] Figure 3 This is a schematic diagram illustrating the spatiotemporal look-ahead alignment principle provided in Embodiment 3 of the present invention, showing the relationship between geometric look-ahead distance and time delay. The diagram demonstrates the spatiotemporal alignment mechanism between the large model's visual look-ahead and the chassis's physical execution, comprising three levels:

[0196] 1. Visual Feedforward Perception Layer / VLM Perception (10Hz, in-vehicle intelligent chip GPU).

[0197] • An autonomous off-road vehicle is moving forward at a speed of V.

[0198] • Large model visual forward look-ahead (VLM) with inference latency τ_gpu / VLMlatency≈100ms.

[0199] • Look-Ahead Distance D is the spatial distance in front of the large model when it is focused.

[0200] 2. Dynamic Look-Ahead Distance (SMP strategy layer).

[0201] • Core formula: D = V·τ_total + D_safety.

[0202] • Total system latency: τ_total = τ_gpu + τ_actuator + τ_brake_prefill ≈ 250ms (large model inference latency + hydraulic brake execution latency).

[0203] • Safety margin: D_safety≥2m, used to avoid the near-field blind spot of the camera.

[0204] The forward focus area is determined by both the driving speed and the total system latency.

[0205] 3. Hazard Zone.

[0206] • Pre-fill trigger target area: low adhesion, heterogeneous terrain.

[0207] • Extreme Conditions: Low-Adhesion / Heterogeneous Terrain.

[0208] Spatiotemporal alignment principle: The system sends a caliper pre-charge signal in advance before the vehicle arrives at the corresponding position of the forward distance, pre-fills the empty stroke of the hydraulic line, and compresses the effective pressure build-up delay of the brake caliper from ~150ms to ~5ms, which is equivalent to achieving near-zero hysteresis vehicle stability control.

[0209] (1) Total physical delay of the system (Large Model Reasoning) ms) (Single caliper hydraulic pressure build-up) ms) ms.

[0210] (2) At the current moment VLM / VLA is based on ms to Image inference output for the front of the car Forward forecasting of terrain semantics / material transitions.

[0211] (3) The system in The preceding text appears to be a fragmented and incomplete sentence, possibly due to a formatting error or incomplete source material. A more accurate translation would require the full context and complete sentences. MPa (only eliminating hydraulic free stroke, without generating significant clamping force), reducing the equivalent delay from "command issuance" to "actual clamping force establishment" of the target caliper. ms compressed to ms.

[0212] (4) When the vehicle is When the vehicle enters this low-adhesion / heterogeneous region, the caliper has already pre-filled, and at this point, issuing the actual braking force command only requires... A millisecond equivalent delay is sufficient to establish the system, effectively achieving near-zero hysteresis vehicle stability control (strictly speaking). (ms, which is negligible relative to the action window).

[0213] Figure 4 This is the dynamic force balance diagram of (single motor + open differential) under unilateral slippage provided in Embodiment 4 of the present invention. Figure 4 The mechanical principles of the differential lock simulation primitives are demonstrated, comprising four levels:

[0214] 1. Explanation of the reaction force mechanism of mechanical bridging.

[0215] The left front wheel (slipping side, μ_L=0.05 ice / mud) falls into the low-traction zone, where the ground can withstand a maximum torque of μ_L・F_z・r_w≈105N・m, which is far lower than the net propulsion torque of the right wheel expected by NMPC.

[0216] • NMPC issues reverse braking torque τ_clip_L (typically 200 N·m, corresponding to clamping force F_clip_L≈1.9 kN) only to the single caliper on the slipping side (front left); the caliper on the high-adhesion side (front right) is idle throughout the entire range, and τ_clip_R=0.

[0217] Under the smoothing weight constraint of ||ΔT_motor||², NMPC commands a single motor to inject the target driving torque T_motor* (≤T_motor_max=420N・m) in an S-shaped ramp increment (non-step pulse).

[0218] • Open differential ironclad rule: T_L = T_R = T_motor * / 2; the left wheel's T_L is jointly borne by (ground 105 + caliper 95) N·m, while the right wheel's T_R = 305 N·m is netly converted into ground propulsion.

[0219] • Caliper pre-fill (VLM / VLA triggered 100-150ms in advance) reduces the effective delay of a single caliper from 150ms to ~5ms, precisely aligning with the motor S-curve timing and eliminating the gap between "motor torque increased / caliper not effective".

[0220] 2. Motor Control Layer.

[0221] • Single range-extended motor + open differential, T_motor*≤T_motor_max=420N・m

[0222] • NMPC real-time optimization, S-shaped ramp torque injection, ||ΔT_motor||² smooth weight constraint.

[0223] 3. Drive Wheels.

[0224] • The ironclad rule of open differentials: T_L=T_R=T_motor* / 2, with a single motor driving the left and right wheels.

[0225] • Left front wheel (slipping side): μ_L=0.05, slip ratio λ_FL≥0.3, τ_clip_L=200N・m (F_clip_L≈1.9kN) is applied only on this side.

[0226] • Right front wheel (high adhesion side): μ_R=0.7, full net thrust, caliper idle for the entire range τ_clip_R=0 (F_clip_R=0).

[0227] 4. Torque balance result / ForceBalanceOutput.

[0228] • Torque distribution on the left wheel (slipping side): T_L = T_motor* / 2 = 200 N·m (evenly distributed in open differential); of which 105 N·m is consumed by ground friction (adhesion limit, ground propulsion), and 95 N·m is absorbed by the remaining heat of the caliper τ_clip_L (the remaining anti-slip ratio λ continues to diverge); the ground friction component propulsion force F_b_ground_L = τ_ground_L / r_w = 105 / 0.35 = 300 N.

[0229] • Right wheel (high adhesion side) net propulsion: T_R=T_motor* / 2=200N・m (net full propulsion); F_b_ground_R=T_R / r_w=200 / 0.35≈571N (deduct caliper 0, full ground contact, no braking loss).

[0230] The total thrust from left and right is 300 + 571 = 871 N, which is sufficient to get out of trouble.

[0231] Figure 5 is a block diagram of the low-level closed-loop Smith predictor with error feedback correction provided in Embodiment 5 of the present invention. This figure illustrates the closed-loop Smith predictor mechanism with forward integration by the state observer and multi-step error adaptive compensation, comprising four levels:

[0232] 1. State Acquisition (time k) at the current moment.

[0233] • Sensor measured value x(k): The actual state observation at the current moment, which is collected by a 200Hz sensor to measure the vehicle's actual attitude and force state.

[0234] • Output of the uncorrected mechanistic model 30 steps ago: x̂_uncorr(k): The current step error alignment benchmark, which is the unbiased prediction of the system at time k-30 using the mechanistic model for the current time.

[0235] • Error e(k) calculation: e(k)=x(k)-x̂_uncorr(k), which is the delay window residual, which is the difference between the actual vehicle state measured by the sensor and the predicted value of the pure hysteresis forward mechanism model for the current moment.

[0236] 2. Adaptive Error Correction Layer

[0237] • Time-varying adaptive filter updater: online weight iterative update, constructing a time-varying adaptive gain matrix K_adapt(k), which is dynamically adjusted according to the current semantic terrain type.

[0238] • Multi-step error compensation module: δx(k+j) forward error prediction measurement generation, which performs forward error extrapolation on the prediction state at each step in the future prediction time domain (j=1,2,…,N).

[0239] • Core formula: δx(k+j|k)=K_adapt(k)·e(k)·e^(-α·j·Δt) .

[0240] Where α is the time-domain decay factor, the error compensation amount for each step in the prediction time domain is obtained by exponential decay multi-step forward extrapolation.

[0241] 3. Smith-Corrected ForecastState.

[0242] • Multi-step prediction errors are superimposed and corrected online to eliminate the mismatch bias of the mechanism model and feed forward into the prediction status of each step of NMPC.

[0243] • Reconstruction formula: x̂_smith(k+j)=x̂_uncorr(k+j)+δx(k+j) .

[0244] Mechanism model base prediction value ⊕ adaptive error compensation amount, online superposition correction

[0245] • The error compensation is superimposed on the basic prediction value of the mechanism model to obtain the reconstructed prediction state.

[0246] 4. NMPC Rolling Optimizer.

[0247] • Accept the lookahead state prediction of x̂_smith(k+j), and perform lookahead control solution for the current frame j=0,1,…,N steps.

[0248] • The reconstructed predicted state input nonlinear model predictive control loop participates in rolling optimization.

[0249] Figure 6 is a data flow diagram of the low-level 200 Hz NMPC rolling optimization provided in Embodiment 6 of the present invention (including the Smith predictor and delay stage). This figure illustrates the complete data flow closed loop of 200 Hz real-time sensing, Smith residual, time-varying adaptive gain, multi-step feedforward, NMPC SQP / IPOPT, and 30-step pure time delay feedback, comprising three levels:

[0250] 1. Sensing Acquisition and Adaptive Error Calculation

[0251] • 200Hz real-time loop, online Smith residual calculation, and time-varying gain K_adapt(k) iteration.

[0252] • 200Hz sensor acquisition: High-frequency signal acquisition such as wheel speed, IMU, and hydraulic pressure.

[0253] • State observer x̂(k): Wheel speed + IMU are fused online via EKF to estimate vehicle state and ground adhesion coefficient μ̂.

[0254] Smith residual e(k): e(k) = x(k) - x̂_m(k|kd), calculates the delay window residual of the current control step.

[0255] • Time-varying K_adapt(k) adaptive gain matrix: online iterative update, eliminating delay mismatch, and dynamically adjusting according to terrain semantics.

[0256] 2. Online NMPC Optimization.

[0257] • Upper-layer 100ms VLM / VLA look-ahead intention D, multi-step feedforward correction δx, SQP / IPOPT online solution.

[0258] • Upper-layer VLM / VLA look-ahead intent D: Provides look-ahead geometric intent and caliper pre-fill signal.

[0259] • NMPC Online SQP / IPOPT Solver:

[0260] Objective function: min J

[0261] Constraint: stx(k+1) = f + g・F_b(kd)

[0262] Prediction time domain N=40 steps, distance from walk Δt=5ms

[0263] The pure time delay step d corresponding to the braking execution delay is explicitly written into the state transition prediction equation.

[0264] • Multi-step feedforward correction δx(k+j): Forward error prediction measurement, which adds the error compensation amount to the predicted value of the mechanism model.

[0265] 3. Control Output & Actuation.

[0266] •u*(k+N) rolling output, motor drive + hydraulic braking, 30-step pure time delay z^-d feedback to Smith residual calculation.

[0267] • u(k+N) rolling optimized output sequence*: control instructions for various actuators such as T_motor(k) and F_b(k).

[0268] • Motor drive + hydraulic brake actuator: Sends optimized control commands to the vehicle drive mechanism and braking mechanism.

[0269] • Includes 30 steps of pure time delay z^-d → feedback to Smith residual calculation, forming a complete closed loop.

[0270] Figure 7This is a timing diagram of torque redistribution under the differential lock simulation primitive provided in Embodiment 7 of the present invention. The diagram shows the timing relationship of each key signal during the activation process of the differential lock simulation primitive, with the time axis from t=-150ms to t=250ms:

[0271] Key timeline nodes: t=-150ms→t=0→t=5ms→t=25ms→t=50ms→t=150ms→t=250ms.

[0272] 1. VLM / VLA large model frame output.

[0273] • The semantics are in place at time t=0, marking the transition boundary or obstacle ahead.

[0274] The large model outputs forward forecasts based on image inference from t=-100ms to t=0.

[0275] 2. Pre-fill calipers.

[0276] • Triggering is performed 150ms in advance, and the hydraulic chamber is pre-filled with 0.2-0.5MPa to eliminate empty stroke.

[0277] • Reduce the effective delay of a single caliper from 150ms to ~5ms.

[0278] • The pre-filling stage only eliminates hydraulic free stroke and does not generate significant clamping force.

[0279] 3. λ_FL cross-threshold detection

[0280] At time t≈50ms, the slip ratio of the left front wheel was detected as λ_FL=0.42>λ_max=0.18.

[0281] The force feedback observer detected that the slip ratio of the slipping tire on the slipping side exceeded the threshold, and the left wheel speed spiked abnormally.

[0282] 4. Motor torque T_motor.

[0283] • NMPC smooth S-shaped ramp (weighted smoothing in ||ΔT||²) injection, starting to rise from t≈50ms.

[0284] ·T_motor_max=420N・m, take ~95%T_motor_max≈400N・m.

[0285] • The saturation hard constraint is met throughout the process, with no excess of "peak value exceeding 15%".

[0286] • Eliminate mechanical shock to the half-shaft / differential.

[0287] 5. Front left caliper F_clip_L

[0288] The pressure build-up begins at approximately 50ms and reaches the target value of 0.9kN at approximately 150ms.

[0289] • Only reverse clamping of the single caliper on the slipping side (front left).

[0290] • The corresponding caliper reverse braking torque τ_clip_L≈94N・m.

[0291] 6. Right front caliper F_clip_R

[0292] • The right-side caliper is idle throughout (F_clip_R=0) and does not participate in primitives.

[0293] • The high-adhesion side caliper remains idle, without generating any parasitic braking reaction torque.

[0294] 7. Actual traction force on the right side, F_propulsion.

[0295] • Net propulsion per wheel ≈ T_R / r_w ≈ 571N.

[0296] • The combined torque of the two front wheels is approximately 1.14 kN, which is enough to get them out of trouble.

[0297] • Positive thrust maintains longitudinal control of the vehicle, and works in conjunction with yaw moment to complete obstacle avoidance or escape from trouble.

[0298] Key timing characteristics: Pre-fill is triggered by VLM / VLA 150ms in advance, compressing the equivalent delay of caliper command → actual clamping force from 150ms to ~5ms; slippage is detected at t=50ms, and the actual caliper pressure build-up + motor torque are synchronized at t=150ms, with a total equivalent lag of ~5ms (negligible relative to the action window).

[0299] (1) Pre-fill is provided by VLM / VLA in The milliseconds (ms) advance trigger reduces the effective delay between caliper command and actual clamping force from 150ms to [a smaller value]. ms.

[0300] (2) MS detected slippage. The actual pressure build-up of the MS caliper and the synchronous motor torque are synchronized, resulting in a total equivalent lag. ms (negligible relative to the action window).

[0301] (3) Motor torque throughout the entire range Saturated hard constraints prevent "transient exceedance of 15%" and eliminate mechanical shock to the half-shaft / differential.

[0302] (4) NMPC and High-weight smoothing penalty naturally avoids the low-frequency surge pattern of "slippage → stalling → release → slippage again".

[0303] Figure 8 This is the main flowchart of the vision-force coupling hierarchical control method provided in Embodiment 8 of the present invention. The flowchart shows the complete main flow of the off-road autonomous driving motion control method based on vision-force coupling hierarchical control, which includes the following core steps:

[0304] Step S101: Slow Frequency Sensing Step

[0305] • The slow-frequency perception decision layer based on Factored Markov Decision Process (FDP) factorizes the global state of the off-road environment into geometric factors, semantic factors, and force factors.

[0306] • Determine the spatial look-ahead distance D = v_x・τ_total based on the current vehicle speed.

[0307] • The forward geometry intent and brake caliper pre-fill signal at the forward look-ahead distance of the output space.

[0308] The global state is factorized using a dynamic Bayesian network to satisfy the conditional independence equation.

[0309] Step S102: Intermediate Frequency Routing Steps

[0310] • Mid-frequency strategy: The cerebellum receives geometric factors, semantic factors, and force factors.

[0311] • Perform route matching for different terrains using a pre-defined expert strategy library (SMP).

[0312] • Assign control weights and adaptively adjust the weights of each expert strategy through a sticky routing mechanism.

[0313] Step S103: Fast execution steps

[0314] • The fast-frequency force sensing execution layer performs nonlinear model predictive control (NMPC) optimization on a rolling basis in each control cycle according to the control weight.

[0315] • The pure time delay step d corresponding to the braking execution delay is explicitly written into the state transition prediction equation: x(k+1)=f(x(k),T_m(k))+g(x(k))・F_b(kd).

[0316] • A closed-loop adaptive Smith predictor is used to reconstruct the state within the control cycle.

[0317] o Calculate the delay window residual e(k).

[0318] o performs exponential decay multi-step forward extrapolation using the time-varying adaptive gain matrix K_adapt(k).

[0319] The predicted state is reconstructed by superimposing the error compensation amount.

[0320] • Scrolling optimization yields optimization control instructions.

[0321] The NMPC objective cost function includes a highly weighted quadratic penalty term for the rate of change of braking force, which constrains the temporal smoothness of the braking command.

[0322] Step S104: Execute the output step

[0323] • Optimize control commands and send them to the vehicle's drive and braking mechanisms.

[0324] • Achieve closed-loop control of vehicle motion.

[0325] Feedforward feedback closed-loop steps (optional enhancement).

[0326] • The forward geometry intent and brake caliper precharge signal are transmitted as feedforward quantities to the high-frequency execution layer in advance.

[0327] • The real-time status feedback of the fast frequency execution layer is used to update the control weights of the intermediate frequency routing layer and the force perception factors of the slow frequency sensing layer.

[0328] • Form a complete vision-force coupled closed-loop control architecture.

[0329] It should be noted that the off-road autonomous driving motion control method based on vision-force coupling hierarchical control provided in this embodiment of the invention can be executed by an electronic device, a apparatus, or a control module within that apparatus for executing the method. This embodiment of the invention uses an apparatus executing the method as an example to illustrate the off-road autonomous driving motion control apparatus based on vision-force coupling hierarchical control provided in this embodiment of the invention.

[0330] Figure 9 This is a schematic diagram of the off-road autonomous driving motion control device based on vision-force coupling hierarchical control provided in Embodiment 9 of the present invention. The device 100 includes a slow-frequency perception module 10, a medium-frequency routing module 20, a fast-frequency execution module 30, and an execution output module 40, wherein:

[0331] The slow-frequency perception module 10 is used to factorize the global state of the off-road environment into geometric factors, semantic factors and force factors based on the factorized Markov decision process, and determine the spatial look-ahead distance according to the current vehicle speed, and output the look-ahead geometric intention and brake caliper precharge signal at the spatial look-ahead distance.

[0332] Preferably, the global state is factorized using a dynamic Bayesian network, and its joint state transition probabilities satisfy the following conditional independence equation:

[0333]

[0334] in, Let S be the global state at the next time step, S be the global state at the current time step, and u be the control input. Geometric factor For semantic factors, For the i-th force perception factor, This serves as the input for macroscopic motion control. The force feedback factors include at least the instantaneous coefficient of friction between the wheels and the ground, the slip ratio of the four wheels, and the deviation of the yaw rate, which are updated in real time by the force feedback.

[0335] Preferably, the spatial look-ahead distance satisfies the calculation formula:

[0336]

[0337] The current vehicle speed, The total system delay is the sum of the large model inference delay and the hydraulic braking execution delay.

[0338] The brake caliper precharge signal is sent in advance before the vehicle arrives at the corresponding position in the spatial forward distance, pre-filling the empty stroke of the hydraulic line, and compressing the effective pressure build-up of the brake caliper to within the predetermined time.

[0339] The intermediate frequency routing module 20 is used to receive geometric factors, semantic factors and force factors in the intermediate frequency strategy cerebellum, and to perform route matching and assign control weights for different terrains in a predetermined expert strategy library.

[0340] The fast-frequency execution module 30 is used by the fast-frequency force sensing execution layer to perform nonlinear model predictive control optimization in each control cycle according to the control weight, explicitly write the pure time delay step number corresponding to the braking execution delay into the state transition prediction equation, and use the closed-loop adaptive form of the Smith predictor to perform look-ahead reconstruction of the state in the control cycle, and obtain the optimized control command through rolling optimization.

[0341] Preferably, the state transition prediction equation is in discretized nonlinear form, specifically:

[0342]

[0343] in, Let be the vehicle state vector at the k-th control step. The motor drive torque at step k is... The braking request for the entire vehicle is issued d steps in advance, where d is the pure lag step number, equal to the ratio of the hydraulic braking execution delay to the control cycle. The motor drive torque acts instantly on the vehicle's dynamic response, while the braking force is determined by the control command issued d steps in advance.

[0344] Preferably, the process of the closed-loop adaptive Smith predictor performing look-ahead reconstruction of the state specifically includes:

[0345] Calculate the delay window residual for the current control step. The delay window residual is the difference between the actual vehicle state measured by the sensor and the predicted value of the mechanism model for the current moment before the pure delay step.

[0346] By using a time-varying adaptive gain matrix that dynamically adjusts according to terrain semantics, the residual of the delay window is exponentially decayed through multi-step forward extrapolation to obtain the error compensation amount for each step in the prediction time domain.

[0347] The error compensation is superimposed on the basic prediction value of the mechanism model to obtain the reconstructed prediction state, which is then input into the nonlinear model predictive control loop to participate in rolling optimization.

[0348] Preferably, the objective cost function of the nonlinear model predictive control includes a high-weighted quadratic penalty term for the rate of change of braking force, with the weight of the penalty term significantly greater than the cost weight of the motor torque. By heavily penalizing the step-by-step change in braking force, the temporal smoothness of the braking command is constrained, ensuring that during the lag window when hydraulic braking is not in effect, the transient dynamics of the vehicle are preferentially adjusted through motor torque.

[0349] The execution output module 40 is used to send optimized control commands to the vehicle drive mechanism and braking mechanism to realize closed-loop control of vehicle motion.

[0350] Preferably, the off-road autonomous driving motion control device 100 based on vision-force coupling hierarchical control further includes a feedforward feedback closed-loop module, which, after the fast-frequency execution module 30 executes and before the execution output module 40 executes, transmits the look-ahead geometric intent and brake caliper precharge signal as feedforward quantities to the fast-frequency execution module 30 in advance, and uses the real-time status feedback of the fast-frequency execution module 30 to update the control weights of the intermediate-frequency routing module 20 and the force perception factor of the slow-frequency perception module 10.

[0351] The off-road autonomous driving motion control device based on vision-force coupling hierarchical control provided in this invention can achieve... Figures 1-8 The various processes implemented in the embodiment of the off-road autonomous driving motion control method based on vision-force coupling hierarchical control shown are not described in detail here to avoid repetition.

[0352] The present invention also provides a storage medium for storing, for example, Figures 1-8The computer program for any of the vision-force coupled hierarchical control-based off-road autonomous driving motion control methods described above. For example, computer program instructions, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operation of the computer, achieving the same technical effect; to avoid repetition, these will not be elaborated further here. The program instructions for invoking the methods of the present invention may be stored in a fixed or removable storage medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in the storage medium of a computer device operating according to the program instructions.

[0353] According to one embodiment of the present invention, the present invention also provides such a Figure 10 The illustrated electronic device 400 may optionally include a storage medium 200 for storing a computer program and a processor 300 for executing the computer program. When the computer program is executed by the processor 300, it implements any of the aforementioned off-road autonomous driving motion control methods based on vision-coupling hierarchical control, triggering the electronic device 400 to execute methods and / or technical solutions based on the foregoing embodiments, achieving the same technical effect. To avoid repetition, these will not be elaborated further here. It should be noted that the electronic devices in this embodiment include mobile electronic devices and non-mobile electronic devices. For example, mobile electronic devices may be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, super mobile personal computers, netbooks, or personal digital assistants, etc., while non-mobile electronic devices may be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This embodiment does not specifically limit the scope of the invention.

[0354] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including associated data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of the present invention can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.

[0355] This invention can be implemented on a computer as a computer-based method, or in dedicated hardware, or a combination of both. Executable code or portions thereof for the method according to the invention can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Optionally, the computer program product includes non-transitory program code components stored on a computer-readable medium so as to execute the method according to the invention when the program product is executed on a computer.

[0356] In an optional embodiment, the computer program includes computer program code components adapted to perform all steps of the method according to the invention when the computer program is run on a computer. Optionally, the computer program is embodied on a computer-readable medium.

[0357] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0358] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A motion control method for off-road automated driving based on vision-force coupling hierarchical control, characterized in that, Includes the following steps: The slow-frequency perception step, based on the factorized Markov decision process, factorizes the global state of the off-road environment into geometric factors, semantic factors and force factors, and determines the spatial look-ahead distance according to the current vehicle speed, and outputs the look-ahead geometric intention and brake caliper precharge signal at the spatial look-ahead distance. In the intermediate frequency routing step, the intermediate frequency strategy cerebellum receives the geometric factors, the semantic factors, and the force factors, and performs route matching and assigns control weights for different terrains in a predetermined expert policy library. In the fast-frequency execution step, the fast-frequency force sensing execution layer performs nonlinear model predictive control optimization in each control cycle according to the control weight. The pure time delay step corresponding to the braking execution delay is explicitly written into the state transition prediction equation, and the state in the control cycle is reconstructed by a closed-loop adaptive Smith predictor. The optimized control command is obtained by rolling optimization. The output step involves sending the optimized control command to the vehicle drive mechanism and braking mechanism to achieve closed-loop motion control of the vehicle.

2. The method of claim 1, wherein, In the slow-frequency sensing step, the global state is factorized using a dynamic Bayesian network, and its joint state transition probability satisfies the following conditional independence equation: in, Let S be the global state at the next moment, S be the global state at the current moment, and u be the control input; Geometric factor For semantic factors, For the i-th force perception factor, The force feedback factor is used as the input for macroscopic motion control. It includes at least the instantaneous friction coefficient between the wheel and the ground, the slip rate of the four wheels and the deviation of the yaw rate, and is updated in real time by the force feedback.

3. The method of claim 1, wherein, In the slow-frequency sensing step, the spatial look-ahead distance satisfies the calculation formula: is the current vehicle speed, is the total system delay, which is the sum of the large model inference delay and the hydraulic braking execution delay; The brake caliper precharge signal is sent in advance before the vehicle arrives at the position corresponding to the spatial forward distance, pre-filling the empty stroke of the hydraulic line and compressing the effective pressure build-up of the brake caliper to within a predetermined time.

4. The method of claim 1, wherein, In the fast-frequency execution step, the state transition prediction equation is in discretized nonlinear form, specifically: in, Let be the vehicle state vector at the k-th control step. The motor drive torque at step k is... The vehicle braking request is issued d steps in advance, where d is the pure lag step number, which is equal to the ratio of the hydraulic braking execution delay to the control cycle; the motor drive torque acts on the vehicle dynamic response in real time, while the braking force is determined by the control command issued d steps in advance.

5. The method of claim 1, wherein, In the fast execution step, the process of the closed-loop adaptive Smith predictor performing look-ahead reconstruction of the state specifically includes: Calculate the delay window residual of the current control step, where the delay window residual is the difference between the actual vehicle state measured by the sensor and the predicted value of the pure delay step mechanism model for the current moment. By using a time-varying adaptive gain matrix that is dynamically adjusted according to terrain semantics, the residual of the delay window is exponentially decayed in a multi-step forward extrapolation to obtain the error compensation amount for each step in the prediction time domain. The error compensation amount is superimposed on the basic prediction value of the mechanism model to obtain the reconstructed prediction state, which is then input into the nonlinear model prediction control loop to participate in rolling optimization.

6. The method of claim 1, wherein, In the fast-frequency execution step, the target cost function of the nonlinear model predictive control includes a high-weighted quadratic penalty term for the rate of change of braking force. The weight of the penalty term is significantly greater than the cost weight of the motor torque. By severely penalizing the step-by-step change of braking force, the temporal smoothness of the braking command is constrained, so that during the lag window when hydraulic braking is not effective, the transient dynamic behavior of the vehicle is preferentially adjusted by the motor torque.

7. The method of claim 1, wherein, After the fast execution step and before the execution output step, the following steps are also included: Feedforward feedback closed-loop step: The forward geometric intention and the brake caliper precharge signal are transmitted in advance as feedforward quantities to the fast frequency execution step, and the real-time status feedback of the fast frequency execution step is used to update the control weight of the intermediate frequency routing step and the force perception factor of the slow frequency sensing step.

8. A cross-country automatic driving motion control device based on visual-coupled hierarchical control constructed based on the method of any one of claims 1 to 7, characterized by, The device includes: The slow-frequency perception module is used to factorize the global state of the off-road environment into geometric factors, semantic factors and force factors based on the factorized Markov decision process. It also determines the spatial look-ahead distance based on the current vehicle speed and outputs the look-ahead geometric intention and brake caliper precharge signal at the spatial look-ahead distance. The intermediate frequency routing module is used to receive the geometric factors, the semantic factors, and the force factors in the intermediate frequency policy cerebellum, and to perform route matching and assign control weights for different terrains in a predetermined expert policy library. The fast-frequency execution module is used by the fast-frequency force sensing execution layer to perform nonlinear model predictive control optimization in each control cycle according to the control weight. The pure time delay step number corresponding to the braking execution delay is explicitly written into the state transition prediction equation, and the state in the control cycle is reconstructed by a closed-loop adaptive Smith predictor. The optimized control command is obtained by rolling optimization. The execution output module is used to send the optimized control commands to the vehicle drive mechanism and braking mechanism to realize closed-loop control of vehicle motion.

9. A storage medium, characterized by Used to store a computer program for performing the method according to any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images