An intelligent decision-making system and method for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-22
- Publication Date
- 2026-08-14
AI Technical Summary
[0002]传统的无人系统路径规划如A、Dijkstra或基础人工势场法等方法在野外或工地等非结构化道路下面临诸多挑战：传统方法在处理“非几何障碍”（如浅坑或低矮突起）时，由于缺乏动力学约束，往往会将障碍物统一视为不可通过的几何禁区，导致路径搜索失败或路径极度冗余或是规划出的路径在几何上可行但物理动力学方法却不可行；其次现有的避障逻辑多基于固定的阈值（如离地间隙、接近角、离去角等），无法适应路面质量和路面特征复杂多变的非结构化道路
本发明针对传统越障路径规划方法在复杂非结构化路况下越障策略的局限性，路径规划中越障时对车辆动力学与物理通过性上的考虑不足，非结构化环境下车辆越障判据单一、动态响应差的问题，通过建立多体动力学模型，模拟车辆在凸起与凹坑路况下的载荷转移与姿态特征，引入深度强化学习（DRL）在仿真环境中进行海量训练，并依据包含通过性、平顺性及机械损耗等维度的复合奖励函数进行策略迭代，固化为一套多维越障判据矩阵，针对不同高度、跨度及坡度的障碍物自动输出最优方案。
Smart Images
Figure CN122569375A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent decision-making and path planning technology for autonomous driving, and more specifically, to an intelligent decision-making system and method for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning. Background Technology
[0002] Traditional unmanned system path planning, such as A Methods such as Dijkstra's algorithm or basic artificial potential field method face numerous challenges in unstructured roads, such as those in the field or construction sites. Traditional methods, when dealing with "non-geometric obstacles" (such as shallow potholes or low protrusions), often treat obstacles as insurmountable geometric no-go zones due to a lack of dynamic constraints. This leads to path search failures, extremely redundant paths, or geometrically feasible paths that are not feasible according to physical and dynamic methods. Secondly, existing obstacle avoidance logic is mostly based on fixed thresholds (such as ground clearance, approach angle, and departure angle), which cannot adapt to the complex and variable road surface quality and characteristics of unstructured roads. Furthermore, due to the complexity of the dynamic environment, generating obstacle-crossing decisions in real time that balance passability, stability, and efficiency is extremely difficult.
[0003] Improving the autonomous obstacle-crossing ability and intelligent decision-making of vehicles on unstructured roads is an urgent problem to be solved. Summary of the Invention
[0004] The purpose of this invention is to provide an intelligent decision-making system and method for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning, which can improve the autonomous obstacle crossing ability, decision-making intelligence, and survival ability of vehicles in the wild.
[0005] This invention provides an intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning, including an obstacle crossing behavior modeling module, an intelligent criterion generation module, an obstacle feature real-time extraction module, and an intelligent behavior decision-making module; The obstacle crossing behavior modeling module is used to simulate the mechanical response of a vehicle in unstructured terrain based on the original perception data of the unstructured road and the real-time vehicle status data. The intelligent criterion generation module is used to perform iterative training in a simulation environment based on the obstacle crossing behavior modeling module using deep reinforcement learning methods to obtain a multi-dimensional obstacle crossing criterion matrix. The real-time obstacle feature extraction module is used to extract the geometric features of obstacles in real time, and to evaluate the feasibility of passage based on the multi-dimensional obstacle crossing criterion matrix to obtain the probability of passage in the current state. The intelligent behavior decision-making module is used to select the best cost based on the probability of passage in the current state and multi-objective optimization decision logic to obtain a spatiotemporal trajectory point sequence.
[0006] Furthermore, the specific configuration of the obstacle crossing behavior modeling module mentioned above is as follows: Construct a vehicle model, which includes a parametrically configured suspension stiffness, damping, and tire mechanics model. An unstructured terrain database is constructed based on the raw sensing data of unstructured roads. Tire-ground interaction constraints were constructed based on Bekker's theory; Dynamic simulations are performed based on the vehicle model, unstructured terrain library, and tire-ground interaction constraints to simulate real-time vehicle status data. Based on the real-time vehicle status data, physical limit data for various road conditions is obtained, and obstacle-crossing behavior is marked based on the physical limit data.
[0007] Furthermore, the above physical limit data refers to dynamic rollover stability, and the calculation formula is as follows:
[0008] in, For dynamic rollover stability, and These represent the vertical loads on the left and right wheels, respectively.
[0009] Furthermore, the calculation formula for the above tire-ground interaction constraint is as follows:
[0010] in, Ground pressure; The depth of the subsidence; This refers to the tire width; and The cohesive modulus is related to soil cohesion and internal friction. It is a soil deformation index used to quantify the nonlinear settlement characteristics of different soil types (such as soft sand and compacted soil roads) under load.
[0011] Furthermore, the specific configuration of the aforementioned intelligent criterion generation module is as follows: The obstacle crossing behavior modeling module is used as a dynamic simulation environment to construct the state space and state vector of the intelligent agent; Based on the state space and state vector, a multi-objective reward function is constructed. The near-end policy optimization algorithm is used to explore massive scenes and automatically learn the optimal obstacle crossing strategy. The converged neural network decision boundary is extracted to obtain an obstacle crossing feasibility criterion table indexed by obstacle dimension, incoming angle, and incoming velocity, i.e., a multi-dimensional obstacle crossing criterion matrix.
[0012] Furthermore, the calculation formula for the above multi-objective reward function is as follows:
[0013] in, For multi-objective reward functions; The reward for reaching the goal, i.e., the passability score; The absolute value of the vertical acceleration representing stability, i.e., the attitude stability penalty; The rate of change of pitch angle; The weighting coefficient for the reward of achieving the goal. The weighting coefficient represents the absolute value of the vertical acceleration, indicating stability. This is the mechanical loss coefficient. This is the weighting coefficient for the rate of change of pitch angle.
[0014] Furthermore, the specific configuration of the aforementioned obstacle feature real-time extraction module is as follows: Ground filtering is performed on the local point cloud of the original road to obtain the filtered point cloud map. The remaining point cloud is clustered to identify obstacle instances and calculate the geometric bounding box of the obstacle instances. The height, span and slope of the geometric bounding box are extracted. A feature vector is constructed based on the height, span, current vehicle speed, and azimuth angle relative to the obstacle of the geometric bounding box. The feature vector is then mapped to the obstacle crossing feasibility criterion table to obtain the pass probability and expected risk value in the current state.
[0015] Furthermore, the aforementioned intelligent behavior decision-making module is specifically configured as follows: A decision objective function is constructed, and a cost decision is made based on the probability of passage and a preset safety threshold. The cost map is then adjusted to obtain an adjusted cost map. Based on the adjusted cost map, dynamic path planning is performed to search for the globally optimal path. The globally optimal path is then smoothed according to vehicle dynamics constraints to obtain the final spatiotemporal trajectory point sequence.
[0016] Furthermore, the formula for calculating the above decision objective function is as follows:
[0017] in, Let the objective function be the decision objective function; For efficiency index parameters, For safety risk indicator parameters, These are parameters for attitude stability. , and These are the weighting coefficients for efficiency index parameters, safety risk index parameters, and attitude stability index parameters, respectively.
[0018] The present invention also provides a method for applying the above-mentioned intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning, comprising: using the intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning to perform mechanical control on the target vehicle.
[0019] The intelligent decision-making system and method for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning provided by this invention has the following beneficial effects: This invention addresses the limitations of traditional obstacle-crossing path planning methods in complex, unstructured road conditions. These methods fail to adequately consider vehicle dynamics and physical passability during obstacle crossing, and suffer from single obstacle-crossing criteria and poor dynamic response in unstructured environments. By establishing a multibody dynamics model to simulate the load transfer and attitude characteristics of vehicles under bumpy and potholed road conditions, deep reinforcement learning (DRL) is introduced for massive training in a simulation environment. The strategy is iterated based on a composite reward function that includes dimensions such as passability, ride comfort, and mechanical loss, and solidified into a multidimensional obstacle-crossing criterion matrix. This matrix automatically outputs the optimal solution for obstacles of different heights, spans, and slopes.
[0020] Specifically, this invention addresses the shortcomings of traditional methods that lack dynamic constraints and cannot quantify vehicle-terrain mechanical interactions. It constructs a multibody dynamics simulation model, specifically establishing a simulation model that includes nonlinear suspension characteristics, a tire-ground Bekker mechanical model, and 14-DOF vehicle dynamics equations. This model simulates the transient mechanical response of a vehicle traversing bumps and potholes of varying heights and spans. A parallel simulation architecture is used to extract key dynamic indicators such as centroid acceleration, pitch angle change rate, and wheel load. Physical limit criteria for rollover critical angle and structural damage critical point are defined, quantifying the vehicle's dynamic passage limits. This overcomes the problems of traditional obstacle-crossing decision-making lacking dynamic constraints, unable to quantify the mechanical interaction between the vehicle and complex terrain, unable to assess the physical feasibility of the path, and prone to erroneous planning that is geometrically feasible but dynamically infeasible. It solves the problem of difficulty in accurately modeling obstacle-crossing behavior on unstructured roads, providing a high-precision physical benchmark for obstacle-crossing criterion generation, and avoiding the problem of traditional methods misjudging non-geometric obstacles as prohibited areas or planning dynamically infeasible paths.
[0021] This invention addresses the shortcomings of traditional obstacle crossing criteria, which rely on fixed manual thresholds and cannot adapt to dynamic road conditions. It intelligently generates dynamic obstacle crossing criteria based on Deep Reinforcement Learning (DRL). Specifically, it encapsulates a multibody dynamics simulation environment as a DRL training interface, designs a state space including real-time vehicle pose, velocity vector, and terrain elevation map, constructs a multi-dimensional reward function integrating passability rewards, attitude stability penalties, and mechanical loss penalties, and uses the PPO algorithm for iterative training on massive random terrain samples. The converged policy network is then inversely mapped to generate obstacle crossing criteria offline based on obstacle geometry features and vehicle approach direction. A multidimensional obstacle crossing criterion matrix indexed by state was developed, and its generalization ability under unknown noise road conditions was verified. This overcomes the problems of traditional obstacle crossing criteria relying on manually set fixed static thresholds, which cannot adapt to the dynamic changes of obstacle features, road conditions, and vehicle current state, resulting in poor criterion adaptability. It changes the limitation of traditional methods that rely on manual experience to set fixed thresholds, and realizes the adaptive generation of obstacle crossing criteria. It can automatically output the optimal passage scheme for obstacles of different heights, spans, and slopes, as well as different vehicle speeds, approach angles, and other vehicle states, greatly improving the adaptability and accuracy of the criteria.
[0022] This invention addresses the shortcomings of existing technologies in real-time decision-making, which lack multi-objective trade-offs and cannot dynamically balance obstacle crossing and avoidance. It is based on multi-objective optimization decision-making logic combined with obstacle crossing criteria to guide dynamic path planning. Specifically, it establishes a multi-objective decision function including path length (efficiency index), risk coefficient (safety index), and attitude stability index. A multi-dimensional obstacle crossing criterion matrix is invoked in real-time to evaluate the costs of "direct obstacle crossing" and "obstacle avoidance." The cost map weights are dynamically adjusted based on the comparison between the probability of passage and a preset safety threshold, guiding the path planner to generate the optimal trajectory in three-dimensional spacetime. Finally, dynamic constraints are used to smooth the path. This overcomes the limitations of traditional obstacle crossing decisions, which cannot dynamically balance obstacle crossing and avoidance behaviors, making it difficult to simultaneously consider multiple objectives such as passage efficiency, driving safety, and attitude stability. Furthermore, traditional methods suffer from insufficient real-time decision-making in dynamic environments and poor path quality. This invention achieves a dynamic balance between obstacle crossing and avoidance behaviors, generating paths that simultaneously consider passability, stability, and efficiency. This significantly improves the autonomous decision-making and survivability of special vehicles and unmanned vehicles in unstructured outdoor environments.
[0023] In summary, this invention receives environmental perception data and combines it with a real-time dynamic path planning algorithm. Based on obstacle-crossing criteria, it balances vehicle dynamics parameter requirements with path shortestification requirements to perform multi-objective optimization, selecting the optimal behavioral path that balances obstacle avoidance and obstacle crossing. This effectively improves the vehicle's path planning performance on unstructured roads and enhances its autonomous decision-making ability in complex dynamic environments. This system not only optimizes the decision criteria for vehicle obstacle crossing and introduces a dynamically-oriented agent that interacts with the terrain, but also achieves a dynamic balance between obstacle crossing and obstacle avoidance, improving the driving quality of the generated path and significantly enhancing the survivability of special vehicles and unmanned vehicles in wilderness environments. Attached Figure Description
[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a block diagram of the unstructured road obstacle crossing behavior intelligent decision-making system based on the fusion of dynamic simulation and deep reinforcement learning provided by the present invention. Figure 2 This is the overall system architecture logic diagram provided by the present invention; Figure 3 This is a flowchart of the dynamic modeling and simulation provided by the present invention; Figure 4 This is a flowchart of the deep reinforcement learning training process provided by the present invention; Figure 5 This is a flowchart of the real-time feature mapping and criterion matching provided by the present invention; Figure 6 This is a comparison image of the road point cloud data collected by vehicle-mounted LiDAR before and after filtering, provided by the present invention. Figure 7 This is the intelligent agent multi-objective optimization decision logic diagram provided by the present invention; Figure 8 This is a schematic diagram of path planning provided by the present invention. Detailed Implementation
[0025] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0026] Figure 1 This diagram illustrates an intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning, as shown in this embodiment. In this embodiment, the intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning includes an obstacle crossing behavior modeling module, an intelligent criterion generation module, a real-time obstacle feature extraction module, and an intelligent behavior decision-making module. The obstacle crossing behavior modeling module is used to simulate the mechanical response of a vehicle in unstructured terrain based on the original perception data of the unstructured road and the real-time vehicle status data. The intelligent criterion generation module is used to perform iterative training in a simulation environment based on the obstacle crossing behavior modeling module using deep reinforcement learning methods to obtain a multi-dimensional obstacle crossing criterion matrix. The real-time obstacle feature extraction module is used to extract the geometric features of obstacles in real time, and to evaluate the feasibility of passage based on the multi-dimensional obstacle crossing criterion matrix to obtain the probability of passage in the current state. The intelligent behavior decision-making module is used to select the best cost based on the probability of passage in the current state and multi-objective optimization decision logic to obtain a spatiotemporal trajectory point sequence.
[0027] In one exemplary embodiment, the obstacle-crossing behavior modeling module is specifically configured as follows: Construct a vehicle model, which includes a parametrically configured suspension stiffness, damping, and tire mechanics model. An unstructured terrain database is constructed based on the raw sensing data of unstructured roads. Tire-ground interaction constraints were constructed based on Bekker's theory; Dynamic simulations are performed based on the vehicle model, unstructured terrain library, and tire-ground interaction constraints to simulate real-time vehicle status data. Based on the real-time vehicle status data, physical limit data under various road conditions is obtained, and obstacle crossing behavior is marked based on the physical limit data. In one exemplary embodiment, the physical limit data is dynamic rollover stability, calculated using the following formula:
[0028] in, For dynamic rollover stability, and These are the vertical loads on the left and right wheels, respectively; In one exemplary embodiment, the method for marking obstacle-crossing behavior is: when dynamic rollover stability If the wheel is damaged or at the critical point of rollover, the obstacle-crossing behavior will be forcibly marked as impassable. In one exemplary embodiment, the vehicle model is a 14-DOF model; The unstructured terrain database includes raised models of different heights and pit models of different depths. In one exemplary embodiment, the protrusion model includes a triangular prism and a semi-cylinder; In one exemplary embodiment, the formula for calculating the tire-ground interaction constraint is:
[0029] in, Ground pressure; The depth of the subsidence; This refers to the tire width; and The cohesive modulus is related to soil cohesion and internal friction. It is a soil deformation index used to quantify the nonlinear settlement characteristics of different soil types (such as soft sand and compacted soil roads) under load; In one exemplary embodiment, the real-time vehicle status data includes the vehicle's center of gravity acceleration, pitch angle, and wheel load data when passing through an obstacle at different angles of approach and different speeds. In one exemplary embodiment, the intelligent criterion generation module is specifically configured as follows: The obstacle crossing behavior modeling module is used as a dynamic simulation environment to construct the state space and state vector of the intelligent agent; Based on the state space and state vector, a multi-objective reward function is constructed. The near-end policy optimization algorithm is used to explore massive scenes and automatically learn the best obstacle crossing strategy. The converged neural network decision boundary is extracted to obtain an obstacle crossing feasibility criterion table indexed by obstacle dimension, angle of approach, and velocity of approach, i.e., a multi-dimensional obstacle crossing criterion matrix. In one exemplary embodiment, the intelligent criterion generation module is further configured to: verify the generalization of the obstacle crossing feasibility criterion table under unknown noise road conditions, and ensure the robustness of the decision boundary; In one exemplary embodiment, the state space includes the vehicle's real-time pose, velocity vector, and local terrain elevation ahead; In one exemplary embodiment, the expression for the state vector is:
[0030] in, The state vector of the agent; and This represents the vehicle's real-time position and velocity vectors. Let be the vehicle's attitude angular velocity; This is a height map matrix of the local terrain ahead; In one exemplary embodiment, the multi-objective reward function is calculated as follows:
[0031] in, For multi-objective reward functions; The reward for reaching the goal, i.e., the passability score; The absolute value of the vertical acceleration representing stability, i.e., the attitude stability penalty; As a penalty for collision risk, it is used to determine whether the vehicle chassis has interfered with the obstacle (collision / bottoming out). If a collision occurs, it is marked as "1", otherwise it is marked as "0". The rate of change of pitch angle; The weighting coefficient for the reward of achieving the goal. The weighting coefficient represents the absolute value of the vertical acceleration, indicating stability. This is the penalty weighting coefficient for collision risk. This is the weighting coefficient for the rate of change of pitch angle; In one exemplary embodiment, the real-time obstacle feature extraction module is specifically configured as follows: Ground filtering is performed on the local point cloud of the original road to obtain the filtered point cloud map. The remaining point cloud is clustered to identify obstacle instances and calculate the geometric bounding box of the obstacle instances. The height, span and slope of the geometric bounding box are extracted. A feature vector is constructed based on the height, span, current vehicle speed, and azimuth angle relative to the obstacle of the geometric bounding box. The feature vector is then mapped to the obstacle crossing feasibility criterion table to obtain the pass probability and expected risk value in the current state. In one exemplary embodiment, the intelligent behavior decision-making module is specifically configured as follows: A decision objective function is constructed, and a cost decision is made based on the probability of passage and a preset safety threshold. The cost map is then adjusted to obtain an adjusted cost map. Based on the adjusted cost map, dynamic path planning is performed to search for the globally optimal path. The globally optimal path is then smoothed according to vehicle dynamics constraints to obtain the final spatiotemporal trajectory point sequence. In one exemplary embodiment, the formula for calculating the decision objective function is:
[0032] in, Let the objective function be the decision objective function; For efficiency index parameters, For safety risk indicator parameters, These are parameters for attitude stability. , and These are the weighting coefficients for efficiency index parameters, safety risk index parameters, and attitude stability index parameters, respectively. In one exemplary embodiment, the cost decision process is as follows: if the probability of passing is greater than a preset safety threshold, mark the obstacle crossing area as a passable low-cost area in the cost map and activate the obstacle crossing planning branch; if the probability of passing is not greater than the preset safety threshold, mark the current area as an impassable restricted area and trigger the obstacle avoidance and detour logic.
[0033] This embodiment provides a method for applying the above-mentioned intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning, including: using the intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning to perform mechanical control on the target vehicle.
[0034] In some embodiments, the above-described intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning can also be implemented in the following ways.
[0035] In this embodiment, the intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning is configured as follows: A fundamental model of obstacle-crossing behavior on unstructured roads is constructed using multibody dynamics simulation, enabling the quantification of mechanical characteristics during vehicle interaction with complex terrain. By establishing a model incorporating nonlinear suspension characteristics (such as vertical displacement and suspension travel limits), a tire-ground mechanics model, and multi-degree-of-freedom vehicle dynamic equations, the transient response of the vehicle when traversing bumps and potholes of varying geometric dimensions (height, depth, and slope) is simulated. A parallel simulation architecture is employed to extract key dynamic indicators such as vehicle center-of-gravity acceleration, pitch angle change rate, and load fluctuation, providing high-dimensional physical characteristics to support subsequent criterion generation. This method not only solves the problem of accurately modeling obstacle-crossing behavior on unstructured roads but also provides a physical benchmark for understanding vehicle obstacle-crossing limits.
[0036] This paper employs deep reinforcement learning algorithms to autonomously explore in a dynamic simulation environment, achieving intelligent generation of obstacle-crossing criteria for unstructured roads. Using vehicle passability, attitude stability, and structural safety as multi-dimensional reward functions, the agent learns the optimal passage strategy from massive random terrain samples. By inversely mapping the converged policy network, a nonlinear relationship mapping table between obstacle geometric features (such as height and span) and vehicle states (such as speed and angle of approach) is extracted to generate dynamic obstacle-crossing criteria. This method overcomes the limitations of traditional algorithms that rely on human experience to set fixed thresholds, achieving an intelligent breakthrough in the adaptive generation of criteria based on road conditions and vehicle performance.
[0037] This method leverages multi-objective optimization decision-making logic combined with generated obstacle-crossing criteria to guide dynamic path planning on unstructured roads. Specifically, the decision-making system acquires real-time terrain features ahead and uses a pre-generated criterion model to evaluate the costs of two behaviors: "direct obstacle crossing" and "obstacle avoidance." An objective function is established, incorporating path length, energy cost, and risk coefficients. Based on the criterion results, path weights are dynamically adjusted to guide the path planner in generating the optimal trajectory within three-dimensional spacetime. This approach significantly improves the real-time decision-making performance in roadless / wilderness conditions, ensuring that vehicles balance efficiency and driving safety in complex environments.
[0038] In some embodiments, the above-described intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamics simulation and deep reinforcement learning can also be implemented in the following ways.
[0039] Reference Figure 2 This is the overall system architecture logic diagram, showing the closed-loop process from environmental perception to dynamics modeling, then to reinforcement learning criterion generation and final decision execution, executing the following steps: Step 11: Obtain raw perception data of unstructured roads and real-time vehicle status data; Step 12: Construct a multibody dynamics model to simulate the mechanical response of the vehicle in unstructured terrain; Step 13: Iteratively train using deep reinforcement learning in a simulation environment to generate a multi-dimensional obstacle crossing criterion matrix; Step 14: Extract the geometric features of obstacles in real time and assess the feasibility of passage based on obstacle crossing criteria; Step 15: Based on multi-objective optimization decision-making logic, select the optimal cost between "obstacle crossing" and "detour" behaviors; Step 16: Output the dynamic path planning trajectory and actuator control instructions.
[0040] Reference Figure 3 This is a flowchart of dynamic modeling and simulation, describing the computational logic of vehicle parameter settings, terrain modeling, and mechanical interactions. The obstacle crossing behavior modeling module executes the following steps: Step 21: Establish a 14-DOF vehicle model and parameterize the suspension stiffness, damping, and tire mechanics model. Step 22: Construct an unstructured terrain library containing protrusions (triangular prisms, semi-cylinders) of different heights and pit models of different depths; Step 23: Based on Bekker theory, set the tire-ground interaction constraints and define the contact mechanical parameters between the tire and the ground, such as soil cohesion, internal friction angle, and friction coefficient. The formulas are as follows:
[0041] in, Ground pressure; The depth of the subsidence; This refers to the tire width; and The cohesive modulus, which is related to soil cohesion and internal friction; coefficient It is a soil deformation index used to quantify the nonlinear settlement characteristics of different soil types (such as soft sand and compacted soil roads) under load; Step 24: Run a dynamic simulation and record the vehicle's center of mass acceleration, pitch angle, and wheel load data when passing through obstacles at different angles of incidence and speeds; Step 25: Define the critical damage point and obtain physical limit data for various road conditions, such as the dynamic rollover stability (LTR) criterion formula as follows:
[0042] in, and The vertical loads on the left and right wheels are respectively set. A tilt angle exceeding 80% of the vehicle's rollover limit angle is considered a failure. Therefore, when... If the wheel is determined to be at the critical point of damage / rollover, the obstacle crossing behavior will be forcibly marked as "impassable".
[0043] Reference Figure 4 This is a flowchart of the deep reinforcement learning training process; a roadmap for criterion generation technology based on DRL, showing the reinforcement learning training loop, the construction of the reward function, and the derivation of the criterion matrix. The intelligent criterion generation module performs the following steps: Step 31: Encapsulate the dynamics simulation environment as a reinforcement learning training interface; Step 32: Design the agent's state space, including the vehicle's real-time pose, velocity vector, and local terrain elevation ahead, and define the agent's state space at time [time]. state vector for:
[0044] in, and This represents the vehicle's real-time position and velocity vectors. Let be the vehicle's attitude angular velocity; This is a height map matrix of the local terrain ahead; Step 33: Construct a multi-objective reward function and set the passability score, attitude stability penalty, and mechanical loss coefficient. The reward function formula is as follows:
[0045] in, A reward for reaching the goal; The absolute value of vertical acceleration, representing stability; The pitch angle change rate; and by adjusting the weighting coefficients This enables intelligent agents to find a balance between "passing through" and "safety"; Step 34: Optimize using a near-end strategy ( Proximal Policy Optimization Algorithms, The PPO algorithm explores massive scenarios and automatically learns the optimal obstacle avoidance strategy. Step 35: Extract the converged neural network decision boundary, process it offline, and generate an obstacle crossing feasibility criterion table indexed by "obstacle dimension, angle of approach, and velocity of approach"; Step 36: Verify the generalization of the criterion under unknown noise conditions to ensure the robustness of the decision boundary.
[0046] Reference Figure 5 This is a flowchart of real-time feature mapping and criterion matching: it illustrates how the perceived sensor data is transformed into input parameters for the criterion table, and the obstacle feature real-time extraction module performs the following steps: Step 41: Acquire local point clouds using vehicle-mounted LiDAR and perform ground filtering processing. The filtering effect is as follows: Figure 6 The image shows a comparison of the road point cloud data collected by the vehicle-mounted LiDAR before and after filtering. It shows the original road point cloud collected by the radar and the filtered road point cloud. Figure a is the original road point cloud image, and Figure b is the filtered point cloud image. Step 42: Cluster the remaining point cloud to identify obstacle instances and calculate their geometric bounding boxes, extracting their heights. ,span and slope ; Step 43: Obtain the current vehicle speed using the inertial navigation system. and the azimuth angle relative to the obstacle ; Step 44: Convert the feature vector Mapping to the criterion table generated in step 35, calculate the pass probability in the current state. Compared with expected risk value .
[0047] Reference Figure 7 This is a multi-objective optimization decision logic diagram: it shows the logical judgment process of cost function calculation, path weight adjustment, and behavior switching. The intelligent behavior decision module executes the following steps: Step 51: Define the objective function The formula is:
[0048] in For efficiency index parameters, For safety risk indicator parameters, These are parameters for attitude stability. Step 52: Determine the current probability of passing. Is it greater than the preset safety threshold? : Step 521: If Then the obstacle crossing area will be marked as a "passable low-cost area" on the cost map, activating the obstacle crossing planning branch; Step 522: If If so, the area will be marked as a "no-entry restricted area," triggering obstacle avoidance and detour logic; Step 53: Load the adjusted cost map into the dynamic path planner and search for the globally optimal path. Figure 8 The map shows the topographic map under the path planning, which is a schematic diagram of path planning. It shows a 2D map with the passable area delineated after considering the terrain features and a 2.5D map with the path planning decision route. Figure a is a 2D topographic map with the "passable area" and "inaccessible restricted area" classification markings, and Figure b is a 2.5D schematic diagram of the planned route after the cost decision. Step 54: Smooth the path by combining vehicle dynamics constraints and output the final spatiotemporal trajectory point sequence.
[0049] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. An intelligent decision-making system for unstructured road obstacle crossing behavior based on the fusion of dynamic simulation and deep reinforcement learning, characterized in that, It includes an obstacle crossing behavior modeling module, an intelligent criterion generation module, an obstacle feature real-time extraction module, and an intelligent behavior decision-making module; The obstacle crossing behavior modeling module is used to simulate the mechanical response of a vehicle in unstructured terrain based on the original perception data of the unstructured road and the real-time vehicle status data. The intelligent criterion generation module is used to perform iterative training in a simulation environment based on the obstacle crossing behavior modeling module using deep reinforcement learning methods to obtain a multi-dimensional obstacle crossing criterion matrix. The real-time obstacle feature extraction module is used to extract the geometric features of obstacles in real time, and to evaluate the feasibility of passage based on the multi-dimensional obstacle crossing criterion matrix to obtain the probability of passage in the current state. The intelligent behavior decision-making module is used to select the best cost based on the probability of passage in the current state and multi-objective optimization decision logic to obtain a spatiotemporal trajectory point sequence.
2. The system according to claim 1, characterized in that, The obstacle crossing behavior modeling module is specifically configured as follows: Construct a vehicle model, which includes parametrically configured suspension stiffness, damping, and tire mechanics models; An unstructured terrain database is constructed based on the raw sensing data of unstructured roads. Tire-ground interaction constraints were constructed based on Bekker's theory. Dynamic simulations are performed based on the vehicle model, unstructured terrain library, and tire-ground interaction constraints to simulate real-time vehicle status data. Based on the real-time vehicle status data, physical limit data for various road conditions is obtained, and obstacle-crossing behavior is marked based on the physical limit data.
3. The system according to claim 2, characterized in that, The physical limit data refers to dynamic rollover stability, and the calculation formula is as follows: in, For dynamic rollover stability, and These represent the vertical loads on the left and right wheels, respectively.
4. The system according to claim 2, characterized in that, The calculation formula for the tire-ground interaction constraint is as follows: in, Ground pressure; The depth of the subsidence; This refers to the tire width; and The cohesive modulus is related to soil cohesion and internal friction. This is the soil deformation index.
5. The system according to claim 1, characterized in that, The intelligent criterion generation module is specifically configured as follows: The obstacle crossing behavior modeling module is used as a dynamic simulation environment to construct the state space and state vector of the intelligent agent; Based on the state space and state vector, a multi-objective reward function is constructed. The near-end policy optimization algorithm is used to explore massive scenes and automatically learn the optimal obstacle crossing strategy. The converged neural network decision boundary is extracted to obtain an obstacle crossing feasibility criterion table indexed by obstacle dimension, incoming angle, and incoming velocity, i.e., a multi-dimensional obstacle crossing criterion matrix.
6. The system according to claim 5, characterized in that, The calculation formula for the multi-objective reward function is as follows: in, For multi-objective reward functions; The reward for reaching the goal, i.e., the passability score; The absolute value of the vertical acceleration representing stability, i.e., the attitude stability penalty; Penalty for collision risk; The rate of change of pitch angle; The weighting coefficient for the reward of achieving the goal. The weighting coefficient represents the absolute value of the vertical acceleration, indicating stability. This is the penalty weighting coefficient for collision risk. This is the weighting coefficient for the rate of change of pitch angle.
7. The system according to claim 1, characterized in that, The real-time obstacle feature extraction module is specifically configured as follows: Ground filtering is performed on the local point cloud of the original road to obtain the filtered point cloud map. The remaining point cloud is clustered to identify obstacle instances and calculate the geometric bounding box of the obstacle instances. The height, span and slope of the geometric bounding box are extracted. A feature vector is constructed based on the height, span, current vehicle speed, and azimuth angle relative to the obstacle of the geometric bounding box. The feature vector is then mapped to the obstacle crossing feasibility criterion table to obtain the pass probability and expected risk value in the current state.
8. The system according to claim 1, characterized in that, The intelligent behavior decision-making module is specifically configured as follows: A decision objective function is constructed, and a cost decision is made based on the probability of passage and a preset safety threshold. The cost map is then adjusted to obtain an adjusted cost map. Based on the adjusted cost map, dynamic path planning is performed to search for the globally optimal path. The globally optimal path is then smoothed according to vehicle dynamics constraints to obtain the final spatiotemporal trajectory point sequence.
9. The system according to claim 8, characterized in that, The formula for calculating the decision objective function is as follows: in, Let the objective function be the decision objective function; For efficiency index parameters, For safety risk indicator parameters, These are parameters for attitude stability. , and These are the weighting coefficients for efficiency index parameters, safety risk index parameters, and attitude stability index parameters, respectively.
10. A method applied to the unstructured road obstacle crossing behavior intelligent decision-making system based on the fusion of dynamic simulation and deep reinforcement learning as described in any one of claims 1-9, characterized in that, The aforementioned intelligent decision-making system for unstructured road obstacle crossing behavior, which integrates dynamic simulation and deep reinforcement learning, is used to perform mechanical control on the target vehicle.