Path planning method and system fusing multi-modal terrain perception and vehicle dynamics

Abstract

The application belongs to the technical field of path planning, and discloses a path planning method and system fusing multi-modal terrain perception and vehicle dynamics, which comprises the following steps: acquiring and fusing multi-modal geographic data to generate terrain physical representation, calculating terrain complexity based on the terrain physical representation; predicting the overturning index and stability margin of the vehicle based on the terrain physical representation, and generating a passability cost function; taking the passability cost function as a constraint condition, selecting a motion primitive according to the terrain complexity to perform trajectory fitting, and generating a vehicle path; obtaining a vehicle state by executing the vehicle path, correcting the passability cost function by using the vehicle state, and optimizing the vehicle path; constructing a loss function, training a path planning model, and performing path planning by using the trained model. The application realizes safe and self-adaptive local path planning under complex unstructured terrain by establishing a two-way coupling mechanism of terrain and vehicle, and significantly improves the passability and safety of the system in a dynamic environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of path planning, in particular to a path planning method and system fusing multi-modal terrain perception and vehicle dynamics. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Traditional local path planning algorithms mostly rely on geometric passability models for path planning, which is better adapted to hard road surfaces, but ignores key mechanical parameters such as ground stiffness, adhesion coefficient, and sinking risk in complex terrain environments, resulting in serious deviations in evaluating the "real passability" of the planned path. For example, in soft mud or steep terrain, a path generated based solely on geometric obstacle avoidance principles may cause vehicle sinking or skidding due to insufficient consideration of ground bearing capacity.

[0004] Existing algorithms generally do not consider vehicle dynamics constraints, and key parameters such as climbing limit, roll stability threshold, and tire slip rate are not included in the path optimization objective system, which makes the planned path prone to triggering rollover, skidding, or chassis locking during actual execution in complex environments, seriously threatening driving safety. At the same time, the fixed-order motion model algorithm path generation mechanism is rigid and difficult to adapt to the high curvature changes and continuous perturbation characteristics of complex terrain, resulting in poor trajectory smoothness and frequent control command mutations, which further exacerbates vehicle jolting, increases actuator wear, and shortens its service life.

[0005] In particular, existing traditional methods lack environmental adaptability, and the perception-planning-control process is disconnected, unable to build a dynamic closed-loop system from terrain perception to vehicle dynamics response and real-time path update, and unable to optimize planning strategies by accumulating historical experience, which causes the system to respond lag when facing dynamic obstacles or sudden terrain changes, unable to make timely and effective adjustments, and frequently resulting in planning failures in new scenarios. SUMMARY

[0006] To solve the above problems, the present application proposes a path planning method and system fusing multi-modal terrain perception and vehicle dynamics, which establishes a bidirectional coupling mechanism between terrain and vehicle, combines variable-order motion primitive optimization and architecture, realizes safe and adaptive local path planning in complex unstructured terrain, and significantly improves the passability and safety of the system in dynamic environments.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a path planning method fusing multi-modal terrain perception and vehicle dynamics, comprising the following steps:

[0009] Obtaining multi-modal geographic data, fusing the multi-modal geographic data, generating terrain physical representation, calculating terrain complexity based on the terrain physical representation;

[0010] Inputting the terrain physical representation into a vehicle dynamics model, predicting the overturning index and stability margin of the vehicle, and generating a passability cost function based on the overturning index and stability margin;

[0011] Taking the passability cost function as a constraint condition, selecting motion primitives of different orders according to the terrain complexity to perform trajectory fitting, and generating a vehicle path;

[0012] Executing the vehicle path to obtain the vehicle state, correcting the passability cost function using the vehicle state, and optimizing the vehicle path;

[0013] Constructing a hybrid loss function to train the path planning model, and using the trained model to perform path planning.

[0014] As an optional implementation, the path planning method further comprises uploading the trained local model parameters to a central server, and the central server aggregates the model parameters to form global shared parameters.

[0015] As an optional implementation, the multi-modal geographic data is obtained by:

[0016] The camera provides texture entropy and infrared reflectivity, the laser radar reconstructs the terrain elevation curvature through three-dimensional point cloud coordinates, the point cloud density is used to deduce the subsidence probability, the soil dielectric constant is obtained by soil penetration detection of millimeter wave radar, and the ground stiffness coefficient is dynamically corrected combined with the Bayesian subsidence probability model and the multi-spectral vegetation analysis is used to cover and water distribution to quantify the adhesion coefficient.

[0017] As an optional implementation, the overturning index is calculated based on the terrain physical representation:

[0018] ;

[0019] Wherein, is the terrain curvature, is the ground stiffness, is the vehicle speed.

[0020] As an optional implementation, the stability margin is calculated based on the terrain physical representation:

[0021] ;

[0022] Wherein, is the roll moment, For pitching moment, This is the critical threshold for roll torque.

[0023] As an alternative implementation method, the traversal cost function is:

[0024] ;

[0025] in, For ground stiffness, The minimum safety stiffness threshold, , , For weight parameters, As the risk index for overturning, For terrain gradient, It is the Heaviside step function.

[0026] Secondly, the present invention provides a path planning system that integrates multimodal terrain perception and vehicle dynamics, comprising:

[0027] The first module is configured to: acquire multimodal geographic data, fuse multimodal geographic data, generate a terrain physical representation, and calculate terrain complexity based on the terrain physical representation;

[0028] The second module is configured to: input the terrain physical representation into the vehicle dynamics model, predict the vehicle's rollover index and stability margin, and generate a drivability cost function based on the rollover index and stability margin.

[0029] The third module is configured to: use the traversability cost function as a constraint, select different orders of motion primitives according to the terrain complexity to perform trajectory fitting, and generate vehicle paths;

[0030] The fourth module is configured to: execute vehicle routing, obtain vehicle status, use vehicle status to modify the trafficability cost function, and optimize vehicle routing;

[0031] The fifth module is configured to: construct a hybrid loss function, train the path planning model, and use the trained model for path planning.

[0032] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0033] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0034] In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of the first aspect.

[0035] Compared with the prior art, the present application has the following beneficial effects:

[0036] The present application provides a path planning method fusing multi-modal terrain perception and vehicle dynamics, which fuses multi-modal terrain perception, vehicle dynamics constraints and machine learning strategies, breaks through the limitations of traditional path planning geometric models, and establishes a closed-loop link of terrain dynamics attributes→vehicle dynamic response→path feasibility evaluation→path optimization generation. Through the establishment of a bidirectional coupling mechanism of terrain dynamics attributes and vehicle dynamic response, combined with variable-order motion primitive optimization and U-shaped segmentation learning architecture, safe and self-adaptive local path planning under complex unstructured terrain is realized. The key problems such as lack of terrain physical property modeling, neglect of vehicle dynamics constraints and rigid path generation in traditional methods are solved, and the traffic capacity and safety of the system in dynamic environment are significantly improved. It is suitable for complex natural environments such as mountains, jungles, soft ground and trench topography, and can perceive terrain features, adapt to vehicle driving ability, and dynamically generate safe and feasible local paths.

[0037] The advantages of the additional aspects of the present application will be partially given in the following description, partially become obvious from the following description, or be known by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which form a part of the present application, are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and together with the description serve to explain the present application. Embodiments of the present application and its description are used to explain the present application and do not constitute an improper limitation of the present application.

[0039] Figure 1 The decision architecture and technical elements of the present application;

[0040] Figure 2 The framework diagram of the path planning method fusing multi-modal terrain perception and vehicle dynamics of the present application;

[0041] Figure 3 The framework diagram of multi-modal geographic data fusion of the present application;

[0042] Figure 4 The flowchart of adaptive path generation driven by mechanical constraints of the present application;

[0043] Figure 5 The flowchart of data collaborative optimization of the present application;

[0044] Figure 6 The flowchart of U-shaped segmentation learning evolution layer in multi-vehicle path planning. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0047] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but includes other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0049] Example 1

[0050] like Figure 2 As shown, this embodiment provides a path planning method that integrates multimodal terrain perception and vehicle dynamics, including the following steps:

[0051] Acquire multimodal geographic data, fuse multimodal geographic data, generate terrain physical representation, and calculate terrain complexity based on terrain physical representation;

[0052] The terrain physical representation is input into the vehicle dynamics model to predict the vehicle's rollover index and stability margin, and a drivability cost function is generated based on the rollover index and stability margin.

[0053] Using the traversability cost function as a constraint, different orders of motion primitives are selected according to the terrain complexity to perform trajectory fitting and generate vehicle paths.

[0054] Execute vehicle routing, obtain vehicle status, and use vehicle status to modify the trafficability cost function to optimize vehicle routing;

[0055] Construct a hybrid loss function, train the path planning model, and use the trained model for path planning.

[0056] The specific solution of the present invention is as follows:

[0057] The path planning method of fusion multi-modal terrain perception and vehicle dynamics of the application is a new type of multimodal-mechanics coupled local path planning algorithm (MMC-LPPA), a bidirectional coupling mechanism of terrain vehicle, a variable order motion primitive optimization model, a U-shaped segmentation learning-mechanics model hybrid architecture are proposed, a closed loop link of "terrain mechanical properties-vehicle dynamic response-path feasibility evaluation-path optimization generation" is established, and the adaptability of the planning system to complex natural environment and the driving safety are significantly improved.

[0058] The bidirectional coupling mechanism of terrain vehicle realizes real-time perception of the physical characteristic parameters of the terrain through multi-modal sensors, including ground stiffness, adhesion coefficient, subsidence probability, soil moisture content and surface roughness. After quantizing these terrain physical response, the vehicle dynamics model is input, a nonlinear response function is constructed, the stability margin and overturning index of the vehicle at a specific terrain point are dynamically predicted, and they are embedded into the path cost function in the form of a unified path passability score, forming a closed-loop decision chain of "terrain mechanical properties-vehicle dynamics response-path feasibility". This breaks the limitation of traditional path planning algorithms that only judge the passability based on elevation, obstacle, slope and other geometric features, realizes the quantitative mapping of terrain state to path risk, and supports path re-planning when the terrain changes or the vehicle state boundary is approached, thereby significantly improving the path reliability and operation safety of the system in slope, soft soil, trench and sudden topography environments.

[0059] The variable order motion primitive optimization model is a variable order polynomial motion primitive optimization unit based on mechanical constraint driving. The optimization unit dynamically selects the order of the trajectory expression according to the local terrain complexity index, uses a three-order trajectory in low complexity areas to improve calculation efficiency, uses a five-order curve in medium disturbance areas to ensure smooth transition, and uses a seven-order trajectory in high-risk complex terrain to meet the continuous acceleration requirement, thereby realizing the structural adaptability and dynamic adjustability of path generation. At the same time, vehicle stability constraints are added in the trajectory optimization process, and a joint optimization objective is formed by combining trajectory smoothness, mechanical consumption cost and path safety function, which is solved in real time by quasi-Newton gradient optimization, ensuring the optimal balance between smoothness, controllability and mechanical stability of the path. The problems of poor smoothness, sudden turning and discontinuous control of fixed order trajectory structure in complex terrain in traditional path planning are solved.

[0060] The U-shaped segmentation learning-mechanical model hybrid architecture is a hybrid path planning architecture that fuses U-shaped segmentation learning and vehicle mechanical models. The architecture is based on a multi-vehicle collaborative distributed training strategy, and each vehicle continuously optimizes the local strategy network through actual driving experience, and regularly aggregates the strategy gradient through the central server to form a globally shared strategy. Compared with traditional centralized training or single-vehicle training methods, U-shaped segmentation learning in the path planning scene connects end-side perception, center-level multi-source fusion, learned planning capability, and end-side dynamic constraints and rapid and safe control into a trainable, experience-driven closed-loop system, which can improve adaptability in complex unstructured terrain and dynamic obstacles, and also takes into account real-time performance and privacy / bandwidth constraints.

[0061] In the decision-making process, the model introduces a "mechanical model safety layer" as a barrier for strategy execution, that is, before the RL strategy outputs the path / action, a round of feasibility screening is performed through the vehicle response model. If an action or path segment violates the dynamic stability or exceeds the risk of sinking, etc., the system automatically rejects the strategy action and falls back to the alternative path or triggers local re-planning. This hybrid mechanism combines the flexibility of data-driven with the certainty of physical models, improving the adaptability of the strategy in uncharted terrain, while ensuring the safety and robustness of path planning, solving the problems of frequent environmental changes, difficult experience transfer, and poor adaptability to new terrain in complex terrain environments.

[0062] The local path planning includes a four-layer closed-loop algorithm architecture of "perception, coupling, decision-making, and evolution". Through the collaborative interaction of the multi-modal sensing layer, the terrain vehicle coupling layer, the variable trajectory optimization layer, and the U-shaped segmentation learning evolution layer, safe and reliable local path planning is achieved in complex terrain. The decision-making architecture and technical elements of the application are as shown in Figure 1 .

[0063] Specifically, in the multi-modal perception layer, heterogeneous sensors such as laser radar, multi-spectral camera, millimeter wave radar, and IMU generate spatiotemporally aligned multi-source data streams through hardware synchronization, generating a terrain physical parameter tensor. The texture entropy quantifies the macroscopic roughness of the ground surface, the dielectric constant derived from the millimeter wave radar deduces the ground stiffness , the laser radar point cloud density characterizes the terrain curvature sinking probability and water content. These parameters together form a multi-scale description to represent the micro-pore structure of the terrain soil. The three achieve all-weather robust perception through a defect compensation mechanism. The terrain semantic features are extracted through a physical inversion subsystem and an improved YOLOv8seg-DLKA model, and a cross-modal feature tensor is constructed: .

[0064] In the terrain-vehicle coupling layer, this tensor is input into a multi-degree-of-freedom dynamic response model to calculate the overturning risk index in real time. With six-dimensional stability margin Both are used together as vehicle stability criteria and are integrated into a weighted cost function. This generates the trafficability cost function:

[0065] ;

[0066] This function employs a dynamic weight adjustment mechanism: using the Heaviside step function. A weighted amplification is applied to areas exceeding the overturn limit, forcing path detours. When the overturning index... When the safety threshold is exceeded, The output is 1 if the value is 1, otherwise it is 0. Output replanning commands in response to terrain change events.

[0067] In the variable-order trajectory optimization layer, based on terrain complexity The trajectory order is dynamically switched. The L-BFGS method is used to solve the following hybrid loss function, achieving a balance between safety and efficiency through real-time baseline updates and adaptive risk weighting.

[0068] ;

[0069] The meaning of each parameter will be explained in detail later.

[0070] In the U-shaped segmentation learning evolution layer, a double-buffered experience pool is used to store differentiated samples, and physical barrier verification is combined to achieve safety policy screening. A distributed-centralized hybrid learning framework is constructed to achieve multi-vehicle collaborative experience sharing.

[0071] ;

[0072] in, yes The global model parameters at any given time are responsible for storing terrain and traffic knowledge shared by all vehicles; It is a U-shaped segmentation learning rate to control the speed of knowledge fusion; It is the number of vehicles that are effectively participating; This is the local model update amount; It is a security verification function; It is the regularization intensity coefficient; It is a gradient calibration operator.

[0073] In the U-shaped segmentation learning evolution layer, the diversified samples generated in the whole process of perception-planning-control are stored and resampled by double-buffered experience pool, and the uploaded update quantity is safety screened by combining physical barrier checking mechanism, so as to ensure the stability and safety of the distributed-centralized hybrid framework in multi-vehicle cooperative path planning.

[0074] The global strategy update is completed under the constraint of Krum algorithm, forming a closed-loop learning link of "perception-coupling-optimization-execution-experience feedback-model evolution". The architecture significantly improves the planning adaptability and safety in complex unstructured environments through the cross-layer data bus and physical-learning coupling mechanism.

[0075] Based on this technology layer, a new type of multimodal coupled local path planning algorithm (Multimodal-Mechanics Coupled Local Path Planning, MMC-LPPA) is proposed and designed, which adopts a phased processing mode, and its algorithm framework is as shown in Figure 2 .

[0076] The algorithm flow is as follows:

[0077] The MMC_LPPA(sensor_data, vehicle_state) function is constructed to realize the process of multimodal perception, mechanics coupling and path optimization, which is used to generate a safe trajectory. Among them, the parameter sensor_data contains multi-modal sensor data such as lidar, camera, IMU, etc.; vehicle_state contains the current state information of the vehicle such as speed, acceleration, load, etc. The return value trajectory is the generated safe trajectory, and the format is spatial coordinates and time sequence [x(t), y(t), z(t)].

[0078] Stage 1: Multimodal Perception Fusion, function T = MultimodalFusion(sensor_data). The input value is multi-source sensor data, and the output value is the terrain mechanics tensor T = [G(x,y), μ(x,y), P_sink(x,y), ].

[0079] Stage 2: Mechanics constraint generation, function constraints = DynamicCoupling(T, vehicle_state). The input values are terrain tensor T and vehicle state, and the output value is dynamic constraint set {Cost_couple, threshold κ}.

[0080] Phase 3: Adaptive path optimization, function is trajectory = AdaptiveOptimizer(T, constraints). The input values are the terrain tensor T and dynamic constraints, and the output is the safe trajectory (spatial coordinates + time series) solved by the variable optimization unit and loss function.

[0081] Further, a multi-modal terrain perception and vehicle mechanical performance coupling decision framework is proposed, which includes a multi-modal terrain perception system and a terrain-vehicle bidirectional coupling mechanism.

[0082] The multi-modal terrain perception system constitutes the first module of the MMC-LPPA, which aligns the data streams of heterogeneous sensors such as lidar, multispectral camera, and millimeter wave radar in space and time, and fuses the physical inversion and semantic recognition capabilities to generate a terrain parameter tensor , providing unified environmental representation for terrain-vehicle coupling decision. The specific process is shown in Figure 3 .

[0083] First, the camera provides texture entropy and infrared reflectivity. The lidar reconstructs the terrain elevation curvature using three-dimensional point cloud coordinates , and uses point cloud density to deduce the subsidence probability . The millimeter wave radar soil penetration detection obtains the soil dielectric constant , and combines the Bayesian subsidence probability model to dynamically correct the ground stiffness coefficient and the multispectral vegetation analysis to quantify the adhesion coefficient of water distribution , and integrates the obstacle probability field , the calculation formula is:

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] ；

[0089] The above parameters are fused to generate a terrain physical parameter tensor parameter , which provides full-dimensional environmental representation for bidirectional coupling decision and calculates the terrain complexity through implicit parameter inversion, the calculation formula is:

[0090] ;

[0091] ;

[0092] The multimodal terrain perception system is controlled by two subsystems: a physical inversion subsystem and a semantic recognition subsystem. The physical inversion subsystem integrates multi-source physical signals and quantifies mechanical parameters through the inversion model; the semantic recognition subsystem analyzes visual semantic features to supplement the environmental cognition blind spots of the physical model.

[0093] The physical inversion subsystem retrieves a subsidence model by fusing camera and radar signals. The multispectral camera sensor utilizes texture entropy... Quantifying surface particle roughness allows for precise capture of visual details, but it has limitations in foggy and dusty environments due to its inability to penetrate obstacles; radar sensors, on the other hand, utilize point cloud density... Dielectric constant While capable of analyzing soil porosity and moisture content and possessing all-weather detection capabilities, it lacks color semantic recognition. This invention achieves robust all-weather perception through a defect compensation mechanism: in optically obscured scenarios, the radar's penetration capability compensates for camera failure; in semantic perception scenarios such as mud identification, the color resolution capability of the multispectral camera fills the semantic gap in the radar. This heterogeneous data fusion creates a cross-modal emergent effect that single-source sensors cannot achieve, jointly constructing a multi-dimensional understanding of surface physical properties. By generating inversion information from multispectral images, using millimeter-wave radar dielectric constant information, and combining lidar point cloud density changes, this invention establishes a Maxwell viscoelastic surface model to predict ground subsidence behavior under unit pressure. This model comprehensively considers key factors such as soil elastic modulus, moisture content, and contact pressure; the prediction formula is as follows:

[0094] ;

[0095] In the formula, To predict the amount of subsidence; The surface shear strength parameter characterizes the soil's resistance to shear deformation, and is obtained through lidar point cloud density. Distribution variance inversion; The soil plasticity coefficient reflects the plastic deformation characteristics of the soil and is obtained by analyzing the infrared reflectance data from a multispectral camera. The effective contact width of the tire is calculated based on the spatial distribution characteristics of the lidar point cloud, and directly affects the ground pressure distribution. This refers to the local grounding voltage of the vehicle. For soil moisture content, the normalized water index inversion in multispectral bands is used to quantify key indicators of soil plasticity and bearing capacity.

[0096] The model output, used as the "subsidence risk factor" in the path traversability scoring function, is converted into subsidence probability. The form input subsequent passability cost function, ensure path planning dynamic response to ground mechanics change.

[0097] The semantic recognition subsystem, aiming at the identification difficulties of unstructured natural terrain such as ambiguous feature boundary, large scale span, irregular shape, and easy loss of small targets, introduces deformable large kernel attention mechanism (DLKA) on the basis of traditional semantic segmentation model, significantly improves the semantic feature modeling capability for complex ground. In addition, through the normalized wasserstein distance loss function (NWD) in addition to the traditional IoU loss, the small target detection problem is solved, and the distance difference between the predicted box And the real box In the feature space distribution, so as to improve the small target detection accuracy. The loss function form is as follows:

[0098] ;

[0099] Among them, Indicates the two-dimensional wasserstein distance, that is, the optimal transmission distance square, The calibration parameter, the measured gully small target detection F1-score is improved to 0.87, which ensures the reliability of the subsequent obstacle probability field In the terrain parameter tensor.

[0100] The improved model can realize high-precision segmentation of key features such as stones, mud, and gullies, and output structured semantic masks and convert them into adhesion coefficients , obstacle probability field , as the core input of terrain complexity .

[0101] The specific algorithm steps of multimodal perception fusion are as follows:

[0102] S1: Construct the function MultimodalFusion(lidar, camera, radar), which receives lidar data, camera data and millimeter wave radar data as input.

[0103] S2: Use the subfunction = LaplacianFilter(lidar) to process the lidar point cloud data, and use the completion algorithm. Use Laplace filter to process the lidar data, calculate the curvature Of the terrain;

[0104] S3: Utilize the sub-function Psink = SinkProbability(lidar, radar) to combine the data of lidar and millimeter wave radar, and obtain the sink probability Psink of the terrain through the sink probability Bayesian estimation algorithm.

[0105] S4: Utilize the sub-function G = StiffnessInverse(radar, ε_r) for millimeter wave object interpretation, which adds an inversion algorithm. Based on the millimeter wave radar data and the relative dielectric constant ε_r, the stiffness G of the terrain is calculated using the stiffness inversion algorithm (formula ).

[0106] S: Utilize the sub-function μ = FrictionCompensation(camera, radar) for multispectral semantic compensation. Adopt a reinforcement fusion mechanism, combine the data of the camera and the millimeter wave radar, and calculate the adhesion coefficient μ of the terrain through the adhesion coefficient cross-modal compensation algorithm.

[0107] S6: The calculated terrain stiffness G, adhesion coefficient μ, sink probability Psink, and terrain curvature are constructed into a spatiotemporally aligned tensor and returned.

[0108] The terrain-vehicle bidirectional coupling mechanism constitutes the second module of the MMC-LPPA. This mechanism establishes a closed-loop interactive system between terrain physical properties and vehicle dynamics response, realizes bidirectional dynamic coupling through the "terrain to vehicle" positive action chain and the "vehicle to terrain" feedback chain, and supports real-time optimization of the passability cost function.

[0109] The system realizes real-time evaluation and control of vehicle stability through multi-dimensional physical modeling. The terrain tensor generated by the above multi-modal perception closes the terrain to vehicle positive action chain in two dimensions, realizing the mapping of terrain physical properties to vehicle dynamics response.

[0110] In terms of real-time stability evaluation, the rollover index is calculated in real time based on the terrain parameter tensor:

[0111] ;

[0112] where the terrain curvature determines the roll moment, the ground stiffness reflects the response characteristics of the suspension system, and the vehicle speed directly affects the centrifugal effect.

[0113] In terms of multi-degree-of-freedom stability cone analysis, a six-degree-of-freedom dynamic stability margin model is constructed:

[0114] ;

[0115] where the roll moment is coupled by lateral acceleration and the height of the center of mass. The pitch moment depends on the longitudinal acceleration and the slope, the risk of sinking is precisely quantified by the composite calculation of the ground contact area and the probability of sinking, forming a complete evaluation closed loop from micro parameters to macro stability.

[0116] Both are used as vehicle stability criteria and are combined into a weighted cost function. The passage cost function is generated:

[0117] ;

[0118] This function adjusts the weight dynamically: when , the Heaviside step function is used to amplify the weight in the overturning overrun area, forcing the path to bypass; when , the output re-planning instruction. In response to sudden terrain changes.

[0119] The feedback chain of the vehicle to the terrain is mainly based on the coupling decision mechanism, which dynamically corrects the terrain parameter evaluation error through real-time vehicle response data, solves the perception and decision mismatch problem caused by sudden environmental changes, and realizes the reverse regulation of vehicle dynamics state to terrain risk assessment correction.

[0120] The input layer of the feedback chain mainly includes the slip ratio and the vertical stability margin . The slip ratio is calculated by the wheel speed sensor and the IMU, which quantifies the relative deviation of the driving wheel circumferential speed and the vehicle body center of mass speed, and sets as the adhesion failure warning threshold. The six-axis inertial navigation combined with the suspension displacement sensor dynamically calculates the vertical stability margin , which monitors the sinking trend of the vehicle body in real time, and triggers the soil bearing capacity risk warning when .

[0121] When the tire slip ratio is overrun, the real-time compensation mechanism of the adhesion coefficient is triggered:

[0122] ;

[0123] where, , indicates that the shear stress of the tire tread exceeds the Coulomb friction limit. 0.2 is the road self-adaptive damping coefficient, reflecting the attenuation effect of slip energy on adhesion.

[0124] When the vertical stability is close to failure, the sinking risk emergency weighting mechanism is triggered:

[0125] ;

[0126] wherein, The vehicle breaks through the static bearing capacity limit. The risk probability gradient needs to be correspondingly adjusted and corrected.

[0127] After correction, the terrain model is re-evaluated as:

[0128] ;

[0129] ;

[0130] Further, the present application proposes a mechanical constraint driven adaptive path generation algorithm. After completing the real-time injection of the passability cost function output by the coupling layer, the system dynamically selects the mathematical expression form of the motion primitive according to the terrain complexity, designs a variable order optimization unit, quantifies the abstract mechanical risk into an executable trajectory control point sequence through an adaptive order selection strategy, and develops a multi-objective optimization solver to realize the dynamic trade-off between safety and motion smoothness. This part of the algorithm constitutes the third module of the MMC-LPPA, the mechanical constraint real-time embedding algorithm, and the process is shown in Figure 4 .

[0131] The specific algorithm steps are as follows:

[0132] S1: Construct the function DynamicCoupling(T, state), which realizes the dynamic coupling function in the mechanical performance coupling, receives the terrain tensor T and the vehicle state state as input, performs real-time calculation of mechanical risk and dynamic weight adjustment, and finally returns the coupling cost and stability threshold.

[0133] S2: Real-time calculation of mechanical risk, reinforcement algorithm closed loop. Get the curvature and stiffness G from the terrain tensor T, get the speed v from the vehicle state state, calculate the rollover risk index κ, κ = 1 / (1 + exp(-(0.5 * + 0.3 * (G ** -1) + 0.2 * (v ** -1)))).

[0134] S3: Use the six-degree-of-freedom stability cone algorithm, and get the terrain tensor T and the vehicle state state to calculate the stability DSM = StabilityConeModel(T, state).

[0135] S4: Dynamic weight adjustment algorithm (new decision logic). Check if the calculated overturning index κ is greater than 0.35, if κ is greater than 0.35, multiply the weight parameter β by 1.5, realize the risk-driven weight amplification.

[0136] S5: Return the coupling cost Cost_couple(α,β,γ)+DSM_threshold for subsequent path optimization solution.

[0137] Through the verification of the two mechanical constraints, the vehicle dynamics boundary is directly or indirectly embedded into the path generation process, which improves the adaptive ability of the vehicle in the dynamic environment and optimizes the safety and motion smoothness in driving.

[0138] The variable order optimization unit is a path optimization method proposed by the application, which combines a variable order polynomial trajectory structure and a mechanical response driven loss function. The method improves the physical adaptability and execution stability of the path in complex terrain by dynamically selecting the order of the motion primitive and introducing a mechanical penalty term. The path generation result can meet the vehicle dynamics constraints in complex unstructured terrain, and has high passability and control smoothness. The specific description is as follows:

[0139] According to the real-time calculation result of the terrain complexity index , select different orders of motion primitives for trajectory fitting:

[0140] ;

[0141] Different order polynomials are suitable for different levels of terrain structure, and the specific correspondence is shown in Table 1.

[0142] Table 1: Correspondence table of terrain structure and polynomial order

[0143]

[0144] When the terrain is evaluated as a flat area, a third-order polynomial trajectory is selected to meet the displacement and velocity continuity requirements, and to have high computational efficiency. When the path segment contains moderate obstacles or local undulations, a fifth-order polynomial trajectory is selected to introduce acceleration continuity constraints to ensure smooth dynamic response. When the path passes through complex terrain such as high-curvature fracture segments, slope transition depressions, and soft interface areas, a seventh-order polynomial trajectory is selected to add additional acceleration continuity constraints to ensure smooth controller execution and vehicle attitude stability. The variable order optimization unit mechanism not only improves the adaptability of the path generation structure, but also significantly improves the control discontinuity problem of traditional fixed-order paths in complex terrain, and improves the overall dynamic response performance of the system.

[0145] To make the solution more accurate, the present application designs a multi-objective optimization solver, which uses the limited memory BFGS algorithm (L-BFGS) to solve the trajectory parameters, and constructs the following hybrid loss function to realize multi-objective optimization:

[0146] ;

[0147] Variable definition: : Polynomial control point vector (decision variable), determines the spatial shape of the trajectory, : Initial reference path control point. The description of each item of the loss function is shown in Table 2.

[0148] Table 2 Description of each item of the loss function

[0149]

[0150] Further, in order to achieve a balance between safety and efficiency, the present application introduces two core technical features: first, when the mechanical cost function value exceeds the risk threshold (C_risk) > ), automatically increase the mechanical cost weight to 1.5, to preferentially avoid dynamic safety risks such as subsidence and overturning; second, the initial reference path control point is refreshed every 200 milliseconds according to the dynamic terrain environment, ensuring that the optimization target is consistent with the real-time scene and avoiding trajectory failure due to environmental mutations. Through dynamic adjustment of the weight and real-time update of the path reference, the safety is ensured while maintaining the efficiency of the trajectory optimization, solving the conflict between safety and efficiency in traditional methods. This part of the algorithm constitutes the fourth module of MMC-LPPA, the path optimization solver algorithm, and the specific algorithm steps are as follows:

[0151] S1: Construct the function AdaptiveOptimizer(T, constraints). This function realizes the adaptive optimization function, dynamically adjusts the decision order according to the terrain complexity, generates the motion primitive and performs path optimization through the multi-objective optimization algorithm. The parameter T is the terrain related information object, from which the terrain complexity C_terrain and other information can be obtained, and constraints is the constraint condition list.

[0152] S2: Variable order decision algorithm. Obtain the terrain complexity C_terrain = T.get_terrain_complexity() from the terrain information T. Dynamically determine the polynomial order n according to the terrain complexity C_terrain, if C_terrain is less than 0.3, n takes 3; if C_terrain is between 0.3 and 0.7, n takes 5; if C_terrain is greater than 0.7, n takes 7.

[0153] S3: Obtain the initial polynomial path p0. Use the motion primitive generation algorithm to perform polynomial fitting based on the global path and the determined polynomial order n, and obtain p0 = PolynomialFitting(global_path, n).

[0154] S4: Multi-objective solving algorithm. Define the loss function loss_function(p), which includes the mechanical cost term , is the weight coefficient, and Cost_mech(p) is the mechanical cost function; the jerk integral term is , is the weight coefficient, which measures the smoothness of the path; the deviation from the initial path term is , is the weight coefficient, which is used to measure the deviation degree of the current path p from the initial path p0.

[0155] S5: Use the L-BFGS optimizer LBFGS_Optimizer as the solver, set the loss function and the hard constraint condition constraints = [κ < 0.35, P_sink < 0.4], including the overturning index κ less than 0.35 and the sinking probability P_sink less than 0.4.

[0156] S6: Call the solve method of the solver to perform optimization solving and return the solving result.

[0157] The fifth module of the MMC-LPPA is to construct a hybrid loss function to train the path planning model, and use the trained model to plan the path.

[0158] Further, in order to solve the problems of frequent environmental changes in complex terrain environment, difficult experience migration, poor adaptability to new terrain, etc., the present application proposes a U-shaped segmentation learning-mechanical safety co-evolution mechanism, including a physical constraint enhanced U-shaped segmentation learning architecture and a mechanical experience driven double buffer training.

[0159] As Figure 5As shown, the physical constraint enhanced U-shaped segmentation learning architecture adopts a "physical constraint layer- learning layer" dual-channel design, realizing the deep integration of data-driven flexibility and physical safety certainty:

[0160] The architecture realizes the collaborative optimization of data-driven and physical safety through a dual-channel decision mechanism. In terms of physical safety barriers, the system verifies the overturning index and the slip rate at a high frequency of 200Hz in real time, thereby clearly defining the safety boundary ; when the verification passes, the local gradient can participate in U-shaped segmentation learning aggregation, and if the verification fails, it immediately switches to a rule-based backup strategy within 10ms, ensuring the instantaneous response capability in dangerous working conditions.

[0161] The U-shaped segmentation learning aggregation adopts a constraint weighted safety protocol to realize the collaborative evolution of the global model within the physical boundary, and its core formula is:

[0162] ;

[0163] wherein is the global U-shaped segmentation model parameter (including the encoder and the decoder) maintained by the central server at time step ; is the learning rate, which controls the global model update step size; is the number of clients participating in aggregation; is the local U-shaped segmentation model gradient update amount of the th client; is the safety indicator function, which returns 1 for clients passing the physical safety verification, and 0 otherwise; is the regularization coefficient (default value 0.05), which balances optimization efficiency and model stability; is the strategy consistency harmonics function, which suppresses the overfitting tendency of local models; is the feature extraction gradient of the th local model at time step . The formula realizes iterative update by adding a correction term to the current global parameter ; is the safety aggregation term, which represents the aggregation of only the gradients of clients passing the safety verification, avoiding malicious updates; is the regularization harmonics term, which adjusts the local gradient direction through the harmonics function to prevent local model overfitting.

[0164] As shown in Figure 6As shown, in the U-shaped evolutionary learning layer, the diversified samples generated by the perception-planning-control process are stored and resampled by the double-buffered experience pool, and the uploaded update is screened for safety by the physical barrier verification mechanism, thereby ensuring the stability and safety of the distributed-centralized hybrid framework in multi-vehicle cooperative path planning.

[0165] The architecture synchronously supports three types of dynamic update modes: periodic aggregation, event-driven response, and incremental update, and integrates Byzantine fault tolerance algorithm and mechanical feature watermark double protection: Krum algorithm: can accurately filter out malicious nodes, and the embedded resonance frequency and other mechanical fingerprints realize reliable tracking and verification of model integrity.

[0166] The mechanical experience-driven double-buffered training architecture collects terrain and vehicle state data in real time through the environment perception module. When the risk probability is detected to be greater than 0.27, the path execution result will be stored in the risk buffer pool first, and the rest of the data will be preprocessed through image sampling and laser sampling and then stored in the stable buffer pool. In the buffer pool management strategy, the stable buffer pool uses the FIFO principle plus the energy screening mechanism to retain only the 20% of the path samples with the lowest energy consumption; the risk buffer pool uses the LRU algorithm combined with risk level sorting to store the 30% of the event samples with the highest survival probability first.

[0167] The double-buffered pool is complementary in the label dimension. The stable buffer pool labels the terrain type and physical parameters, supporting basic strategy optimization; the risk buffer pool focuses on failure cause classification, driving robustness against training. The difference in the training objective function is designed as:

[0168] The stable buffer pool training minimizes action deviation and energy consumption ; the risk buffer pool training maximizes survival probability and backup strategy alignment . The two buffer pools realize atomic-level data migration through pointer exchange to ensure zero interruption during training. When the stable buffer pool is full, the system automatically triggers buffer exchange and global aggregation processes, and dynamically adjusts the loss weight according to the real-time risk event count:

[0169] ;

[0170] The data of the two buffer pools form a closed-loop mechanism, including forward and backward bidirectional enhancement flows. The forward flow is the terrain parameter tensor generated by the perception layer directly inputting into the coupling layer to calculate the passability cost ; the backward flow is the aggregation of multi-vehicle experience by the evolutionary layer through U-shaped split learning to dynamically update the perception layer weights. The global strategy update formula of the evolutionary layer is The verification result is fed back to the perception layer to adjust the feature extraction model.

[0171] The architecture realizes the coordinated evolution of safety and efficiency through differential sample management, a complementary label system and a dynamic weight mechanism, and provides a high-reliability training basis for U-shaped segmentation learning under complex terrain.

[0172] The application can comprehensively consider various features of complex terrain and the mechanical properties of the vehicle itself, so that the local path planning algorithm can better perceive different types of complex natural environment, make reasonable path planning decisions according to the actual terrain conditions and vehicle state, and thus enhance its adaptability in complex natural environment.

[0173] Embodiment 2

[0174] The embodiment provides a path planning system fusing multi-modal terrain perception and vehicle dynamics, comprising:

[0175] The first module is configured to: acquire multi-modal geographic data, fuse the multi-modal geographic data, generate terrain physical representation, and calculate terrain complexity based on the terrain physical representation;

[0176] The second module is configured to: input the terrain physical representation into a vehicle dynamics model, predict the overturning index and stability margin of the vehicle, and generate a passability cost function based on the overturning index and stability margin;

[0177] The third module is configured to: take the passability cost function as a constraint condition, select motion primitives of different orders according to the terrain complexity to perform trajectory fitting, and generate a vehicle path;

[0178] The fourth module is configured to: execute the vehicle path, obtain the vehicle state, correct the passability cost function using the vehicle state, and optimize the vehicle path;

[0179] The fifth module is configured to: construct a hybrid loss function, train the path planning model, and perform path planning using the trained model.

[0180] It should be noted that the above modules correspond to the steps in Embodiment 1, and the above modules and the examples and application scenarios realized by the steps they correspond to are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system.

[0181] In more embodiments, there is also provided:

[0182] An electronic device comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the method in Embodiment 1 is completed. For brevity, it will not be repeated here.

[0183] It should be understood that, in this embodiment, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or any conventional processor, or the like.

[0184] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method in embodiment 1.

[0185] The method in embodiment 1 can be directly executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software modules can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in a memory, and the processor reads information in the memory to complete the steps of the above method in combination with hardware. To avoid repetition, no further description is given here.

[0186] A computer program product including a computer program, which, when executed by a processor, implements the method in embodiment 1.

[0187] The present application also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, executed by devices at a destination real or virtual processor to perform processes / methods as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. In various embodiments, the functionality of program modules can be combined or split between program modules as desired. Machine-executable instructions for program modules can be executed within a local or distributed device. In a distributed device, program modules can be located in local and remote storage media.

[0188] Computer program code for carrying out operations of the present application can be written in one or more programming languages. These computer program code can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that the program code, when executed by the computer or other programmable data processing apparatus, causes the functions / operations specified in the flowcharts and / or block diagrams to be performed. The program code can be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0189] In the context of the present application, the computer program code or related data can be carried by any suitable carrier to enable the device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, and the like. Examples of signals can include electrical, optical, radio, sound or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0190] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the present embodiment can be realized in electronic hardware or in combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0191] The above describes the specific embodiments of the present application in conjunction with the drawings, but is not a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A path planning method that fuses multi-modal terrain perception with vehicle dynamics, characterized in that, The method comprises the following steps: acquiring multi-modal geographic data, fusing the multi-modal geographic data, generating a terrain physical representation, and calculating terrain complexity based on the terrain physical representation; inputting the terrain physical representation into a vehicle dynamics model, predicting a rollover index and a stability margin of the vehicle, and generating a passability cost function based on the rollover index and the stability margin; using the passability cost function as a constraint condition, selecting motion primitives of different orders according to the terrain complexity to perform trajectory fitting, and generating a vehicle path; executing the vehicle path, obtaining a vehicle state, correcting the passability cost function using the vehicle state, and optimizing the vehicle path; constructing a hybrid loss function, training a path planning model, and performing path planning using the trained model; the passability cost function is: wherein, is the ground stiffness, is a minimum safety stiffness threshold, , , is a weight parameter, is a rollover risk index, is a terrain gradient, is a Heaviside step function; the hybrid loss function is constructed as follows: wherein, , is a weight coefficient, p is a polynomial control point vector, is an initial reference path control point, is a mechanical cost function, is a jerk.

2. The method of claim 1, wherein the path planning is fused with multi-modal terrain perception and vehicle dynamics. The path planning method further comprises uploading the trained local model parameters to a central server, aggregating the model parameters by the central server, and forming global shared parameters.

3. The method of claim 1, wherein The multi-modal geographic data is acquired in the following manner: cameras provide texture entropy and infrared reflectivity, laser radars reconstruct terrain elevation curvature through three-dimensional point cloud coordinates, use point cloud density to deduce subsidence probability, millimeter wave radars detect soil dielectric constant through soil penetration, and combine a Bayesian subsidence probability model to dynamically correct ground stiffness coefficient and multi-spectral vegetation analysis to quantify adhesion coefficient of coverage and water distribution.

4. The method of claim 1, wherein The rollover risk index is calculated based on the terrain physical representation: ； wherein, is the terrain curvature, is the ground stiffness, is the vehicle speed.

5. The method of claim 1, wherein The stability margin is calculated based on the terrain physical representation: ； wherein, is the roll moment, is the pitch moment, is the roll moment critical threshold.

6. The path planning system fusing multi-modal terrain perception and vehicle dynamics, implements the path planning method fusing multi-modal terrain perception and vehicle dynamics as claimed in any one of claims 1-5, characterized in that, The method comprises the following steps: a first module configured to acquire multi-modal geographic data, fuse the multi-modal geographic data, generate a terrain physical representation, and calculate terrain complexity based on the terrain physical representation; a second module configured to input the terrain physical representation into a vehicle dynamics model, predict a rollover index and a stability margin of the vehicle, and generate a passability cost function based on the rollover index and the stability margin; a third module configured to use the passability cost function as a constraint condition, select motion primitives of different orders according to the terrain complexity to perform trajectory fitting, and generate a vehicle path; a fourth module configured to execute the vehicle path, obtain a vehicle state, correct the passability cost function using the vehicle state, and optimize the vehicle path; a fifth module configured to construct a hybrid loss function, train a path planning model, and perform path planning using the trained model.

7. An electronic device, comprising: The computer program is executed by the processor to implement the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-5.

9. A computer program product, characterised in that, The computer program is executed by the processor to implement the method of any one of claims 1-5.

Images