A method and system for unmanned vehicle path planning and control based on speed obstacle avoidance and predictive interpolation

By using predictive interpolation action generation, risk assessment, and reward product mechanisms, the kinematic constraints and obstacle avoidance problems in the path planning and control of unmanned vehicles are solved, enabling unmanned vehicles to achieve efficient obstacle avoidance and target arrival in dynamic environments, and improving control executability and training efficiency.

CN123067477APending Publication Date: 2026-11-10SHANGHAI MARITIME UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611437570.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-16
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing path planning and control methods for autonomous vehicles based on reinforcement learning suffer from problems such as control variables exceeding kinematic constraints, redundant obstacle states, non-executable obstacle avoidance directions, and mutually canceling reward function designs. These issues make it difficult for autonomous vehicles to simultaneously meet kinematic constraints and dynamic obstacle avoidance requirements in dynamic environments.

Method used

By employing predictive interpolation action generation, risk assessment function screening, positive speed obstacle range in the sports school, and reward product mechanism, a dynamic control system model is constructed to limit the Actor network output within the feasible speed and steering angle range, screen key obstacles, and design a nonlinear reward mechanism to coordinate obstacle avoidance and target arrival.

Benefits of technology

It enables autonomous vehicles to efficiently avoid obstacles and reach targets in dynamic environments, improves control executability, training efficiency, feasibility of obstacle avoidance strategies, and coordination of reward guidance, and ensures the smoothness and executability of control commands within the physical range.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123067477A_ABST
    Figure CN123067477A_ABST
Patent Text Reader

Abstract

This invention provides a path planning and control method and system for unmanned vehicles based on speed obstacle avoidance and predictive interpolation, belonging to the field of intelligent path planning and control technology for unmanned vehicles. It includes: establishing kinematic, dynamic environment, and planning and control task models for the unmanned vehicle; calculating the feasible speed and steering angle intervals through interval prediction, and outputting actual control actions using linear interpolation; constructing a risk assessment function using a relative distance exponent, a relative speed line-of-sight projection truncation term, and an azimuth angle weight to screen key risk obstacles; introducing wheelbase constraints to calculate the oriented reachable interval, and intersecting it with the theoretical obstacle avoidance interval to obtain the kinematically corrected VO obstacle avoidance interval; inputting multi-source mixed observation states into the Actor network, and training the planning and control strategy using the product of directional interval rewards, orientation rewards, and proximity rewards. This invention ensures that control commands are always within the kinematically feasible range, reduces redundant state interference, ensures the actual reachability of the obstacle avoidance direction, and improves the strategy convergence speed and generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent path planning and control technology for unmanned vehicles, specifically to a path planning and control method and system for unmanned vehicles based on speed obstacle avoidance and predictive interpolation. Background Technology

[0002] When autonomous vehicles perform path planning and control tasks in dynamic environments, they need to simultaneously satisfy vehicle kinematic constraints, target arrival constraints, environmental boundary constraints, and dynamic obstacle collision avoidance requirements. Existing reinforcement learning-based path planning and control methods typically output control quantities such as linear velocity or steering angle directly. However, this approach has the following problems: First, the Actor network output is not subject to kinematic feasibility constraints, which can easily lead to control quantities exceeding the physical limitations of the actuator, resulting in a disconnect between simulation and actual control. Second, in scenarios with dense dynamic obstacles, inputting all obstacle states into the Actor network will cause state space redundancy, reduce training efficiency, and obstacles without collision tendency will interfere with policy learning. Third, traditional speed obstacle algorithms only provide collision-free speed ranges from a geometric perspective, without considering the nonholonomic kinematic constraints of the Ackerman steering chassis. Therefore, the obstacle avoidance direction output may not be executable on actual vehicles. Fourth, reward function designs are mostly linear superposition forms, and the reward for target approach and obstacle avoidance can easily cancel each other out, causing the policy to get stuck in local optima and making it difficult to simultaneously consider target arrival and dynamic obstacle avoidance.

[0003] Therefore, there is a need for an unmanned vehicle path planning and control method and system that can organically combine speed obstacle prior information, action feasible range, risk obstacle screening and obstacle avoidance guidance rewards, so that the Actor network can learn dynamic obstacle avoidance and target arrival control strategies within the action space that satisfies kinematic constraints. Summary of the Invention

[0004] To address the technical problems in existing technologies, such as control quantities exceeding kinematic constraints, obstacle state redundancy, non-executable obstacle avoidance directions, and mutual cancellation of reward guidance, this invention proposes an unmanned vehicle path planning and control method and system based on speed obstacle avoidance and predictive interpolation. By generating predictive interpolation actions, screening risk assessment functions, using the positive speed obstacle interval of the kinematic school, and employing a reward product mechanism, it achieves efficient obstacle avoidance and target arrival in dynamic environments. It features high control executability, high training efficiency, strong feasibility of obstacle avoidance strategies, and consistent reward guidance.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A path planning and control method for unmanned vehicles based on speed obstacle avoidance and prediction interpolation includes the following steps: A dynamic planning and control system model is established, which includes an unmanned vehicle kinematics model, a dynamic environment model, and a planning and control task model. A predictive interpolation action generation method is constructed, which calculates the feasible interval of an action in real time through interval prediction and outputs the control action through linear interpolation. A risk assessment function is constructed, which uses the relative distance exponent, the projection truncation term of relative velocity along the line of sight, and the azimuth angle weight to quantify the collision risk. Based on the risk assessment indicators, the obstacles with the highest risk level are selected as key risk obstacles. Construct a speed obstacle course algorithm based on motion correction, and introduce the wheelbase of the autonomous vehicle. The achievable orientation range for the next moment is calculated based on the current orientation, speed feasible range, and steering angle feasible range of the autonomous vehicle. And compare the achievable orientation range with the theoretical obstacle avoidance range. Find the intersection and obtain the VO obstacle avoidance range corrected by the sports school. ; The proximal policy optimization algorithm framework is used to train the rule and control policy. The multi-source mixed observation state is used as the observation data input to the Actor network. The action weights are output based on the Actor network, and the product of the direction interval reward, the orientation reward and the proximity reward is used as the obstacle avoidance interval reward to update the rule and control policy.

[0006] Accordingly, the present invention also provides an unmanned vehicle path planning and control system based on speed obstacle avoidance and predictive interpolation, comprising: The model building module is used to build the kinematic model, dynamic environment model and control task model of the autonomous vehicle, providing environmental state description and constraints for subsequent action generation, risk assessment and strategy training. The predictive interpolation action generation module is used to calculate the feasible speed range and the feasible steering angle range for the next moment based on the current state and kinematic constraints of the unmanned vehicle, and to map the interpolation weights output by the Actor network to the actual control actions that are always within the feasible speed range and the feasible steering angle range through linear interpolation. The risk assessment module is used to calculate the risk assessment index of each dynamic obstacle based on the relative distance, the projection of the relative speed along the line of sight, and the azimuth angle, and to screen out the obstacles with the highest risk level as key risk obstacles. The sports school positive speed obstacle module is used to introduce the wheelbase of the unmanned vehicle to calculate the orientation reachable interval at the next moment, and then intersect this orientation reachable interval with the theoretical speed obstacle interval to obtain the VO obstacle avoidance interval that satisfies the Ackermann steering kinematics constraint. The state construction module is used to assemble the target position state, the unmanned vehicle state, the action range state, the risk obstacle state, and the VO obstacle avoidance range state into a multi-source hybrid observation state. The training module is used to train the rule and control strategy using a near-end policy optimization algorithm framework. The obstacle avoidance interval reward is composed of the product of the direction interval reward, the orientation reward, and the proximity reward, so as to guide the rule and control strategy to simultaneously meet the requirements of safe obstacle avoidance and target arrival.

[0007] Compared with the prior art, the beneficial effects of the present invention are: First, by using a predictive interpolation motion generation method, the output of the Actor network is restricted to a feasible velocity and steering angle range determined by the current kinematic state and constraints. Then, a linear interpolation method is employed to map the normalized weights into actual control quantities. This approach ensures that control commands remain within the physical range executable by the autonomous vehicle, avoiding control failures caused by output exceeding limits or sudden changes, and improving the smoothness and actual executability of the actions.

[0008] Second, by truncating the projection of relative velocity along the line of sight in the risk assessment function, the risk contribution of obstacles with no positive collision tendency in the velocity dimension is set to zero. This allows the risk assessment index to effectively distinguish between obstacles with real collision threats and those that are far away. Based on this, only the most critical obstacles with the highest risk level are retained in the state space, significantly reducing the interference of redundant obstacle information on the Actor network and improving training efficiency and decision concentration.

[0009] Third, by introducing wheelbase constraints, the directional reachability range of the autonomous vehicle at the next moment is calculated, and its intersection with the theoretical obstacle avoidance range output by the speed obstacle algorithm is obtained to obtain the obstacle avoidance range corrected by motion correction. This process eliminates obstacle avoidance directions that are theoretically collision-free but cannot be executed in practice due to Ackerman steering constraints, ensuring that the obstacle avoidance decision is consistent with the actual reachability of the vehicle, thus enhancing the engineering feasibility and reliability of the dynamic obstacle avoidance strategy.

[0010] Fourth, a nonlinear safety gating effect is formed through the product structure of direction interval reward, orientation reward, and proximity reward. When the decision orientation is within the safe obstacle avoidance interval, the reward signal is effectively amplified; once it deviates from the safe interval, the overall reward drops rapidly. This mechanism avoids the offsetting effect of the target approach term on the safety penalty in traditional additive rewards, guides the control strategy to maintain coordination between dynamic obstacle avoidance and target arrival, reduces the probability of getting trapped in local optima, and improves the policy convergence speed and scenario generalization ability.

[0011] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the unmanned vehicle path planning and control method based on speed obstacle avoidance and prediction interpolation of the present invention; Figure 2This is a comparison chart of the reinforcement learning reward curves of VP-PPO and IP-PPO in an embodiment of the present invention; Figure 3 This is a comparison diagram of VP-PPO and IP-PPO strategy paths under different dynamic testing scenarios in embodiments of the present invention; Figure 4 This is a graph showing the change in the front wheel steering angle of the unmanned vehicle under the VP-PPO and IP-PPO strategies in this embodiment of the invention. Figure 5 This is a graph showing the speed change curves of the unmanned vehicle under the VP-PPO and IP-PPO strategies in this embodiment of the invention. Figure 6 This is a graph showing the change in reward for VP-PPO and IP-PPO policy transfer learning in an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings, so as to more clearly understand the purpose, features and advantages of this invention. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of this invention, but are only for illustrating the essential spirit of the technical solutions of this invention. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0014] It should be noted that, where there is no conflict, the technical features described in this invention can be combined in any way, and all such combinations are within the protection scope of this invention.

[0015] The unmanned vehicle path planning and control system based on speed obstacle avoidance and predictive interpolation provided by this invention includes a model building module, a predictive interpolation action generation module, a risk assessment module, a motion school positive speed obstacle module, a state construction module, and a training module. These modules operate in conjunction with each other according to their temporal relationships within the control cycle: the model building module provides environmental states and constraints; the predictive interpolation action generation module and the motion school positive speed obstacle module calculate the feasible action range and the VO obstacle avoidance range, respectively; the risk assessment module screens key obstacles; the state construction module assembles multi-source hybrid observation states; and the training module outputs action weights and updates the planning and control strategy based on the observed states. The specific implementation methods and functions of each module are described in detail below, along with the method steps.

[0016] like Figure 1 As shown, the unmanned vehicle path planning and control method based on speed obstacle avoidance and prediction interpolation provided by the present invention includes the following steps: Step S1: Establish a dynamic planning and control system model; The model building module is used to establish the autonomous vehicle's kinematics model, dynamic environment model, and control task model, providing a foundation for subsequent action generation, risk assessment, and strategy training. The core function of this module is to determine the autonomous vehicle's state evolution, environmental boundary conditions, dynamic obstacle distribution, target area definition, and task completion criteria. Specifically, the autonomous vehicle kinematics model is used to determine the autonomous vehicle's state evolution; the dynamic environment model is used to determine the environmental boundary conditions, dynamic obstacle distribution, and target area definition; and the control task model is used to determine the task completion criteria.

[0017] In some embodiments, the kinematic model of the autonomous vehicle is described by position, linear velocity, orientation angle, front wheel steering angle, linear acceleration, and front wheel steering angular velocity, and its differential equation is: The function of the aforementioned kinematic model is to describe the state evolution of the autonomous vehicle over continuous time. Among them, For a specific moment; and These are the horizontal and vertical coordinates of the autonomous vehicle in the global coordinate system; For the linear velocity scalar of the autonomous vehicle; The heading angle of the driverless car; This refers to the front wheel steering angle; Linear acceleration; This refers to the steering angular velocity of the front wheels; This refers to the wheelbase of the autonomous vehicle. , , , , These represent the state variables of the autonomous vehicle; their rates of change over time are given by the aforementioned kinematic differential equations. In this embodiment, the control interval... Total duration wheelbase .

[0018] To apply the continuous-time kinematic model described above to discrete-time control and subsequent policy training, it is necessary to define the control interval. and total duration of a single simulation Control interval Indicates the time step between two adjacent control decisions, and the total duration. Indicates the duration of a complete simulation round. Wheelbase The structural parameters of the autonomous vehicle have been introduced from the orientation angle rate of change equation in the aforementioned kinematic model. In this embodiment, the control interval... Total duration wheelbase .

[0019] The above state variables must satisfy the following kinematic constraints: These constraints define the physical execution boundaries of the autonomous vehicle and form the basis for subsequent feasible interval calculations and motion generation. The absolute value of the maximum linear acceleration is denoted as... .

[0020] The basis for the above parameter values ​​is as follows: Control interval A control frequency of 50Hz is chosen, which matches the response bandwidth of common automotive actuators, ensuring the real-time performance and smoothness of control commands. However, excessively small control intervals increase computational burden, while excessively large intervals reduce the timeliness of obstacle avoidance response. Therefore, a suitable frequency is selected. As a control interval.

[0021] Total duration It is based on the autonomous vehicle starting from its initial position Reach the target location The straight-line distance is approximately 21.2 meters, and the average speed is approximately... The simulation was estimated based on driving conditions. In situations where dynamic obstacles need to be navigated around, a 30-second time window can cover the typical task completion process while avoiding excessively long simulation times that could decrease training efficiency.

[0022] wheelbase The parameters are taken from the actual structural parameters of a small Ackerman steering autonomous vehicle. This wheelbase determines the kinematic relationship between the front wheel steering angle and the rate of change of the vehicle's orientation. Using actual parameters ensures that the speed obstacle range for kinematic correction is consistent with the behavior of real vehicles.

[0023] Autonomous vehicle speed range The selection of the speed limit takes into account the safe braking distance and sensor detection range in indoor or small-scale dynamic scenarios. At this speed limit, the autonomous vehicle decelerates at maximum braking speed. During braking, the braking distance is approximately Meters, enabling safe parking within the environmental boundaries and target area defined in this embodiment. Meanwhile, The speed meets the response time requirements of dynamic obstacle avoidance, avoiding the inability to adjust the direction in time due to excessive speed.

[0024] Absolute value of maximum linear acceleration The selection is based on the output characteristics of the drive motor of the small unmanned vehicle and the requirements for ride smoothness. Excessive acceleration will cause abrupt changes in control input, affecting trajectory smoothness; too little acceleration will result in slow speed adjustment, reducing the flexibility of dynamic obstacle avoidance.

[0025] Absolute value of maximum front wheel steering angular velocity The selection of this value is based on the actual response capability of the servo or steering actuator. This value ensures that the front wheel steering angle can be meaningfully adjusted within the control interval, while avoiding excessive steering angular velocity that could overload the actuator or cause vehicle sideslip.

[0026] Front wheel steering angle range This is a typical mechanical limit for Ackermann steering chassis. This angle range provides sufficient steering capability to avoid obstacles while preventing excessive front wheel steering angles that could lead to tire slippage and nonlinear effects, thus ensuring the effectiveness of the kinematic model.

[0027] In some embodiments, the dynamic environment model is defined as follows: the dynamic environment boundary is and Used to constrain the movement space of the autonomous vehicle; the target location is The target area is a circle with this location as the center and a radius of... A circular area; the environment contains 10 dynamic obstacles, each with a collision radius... linear velocity The direction of motion is randomly set and remains unchanged during the simulation. The initial state of the autonomous vehicle is... ,in ; ; ; ; ; Collision radius of autonomous vehicles If in Driverless cars and the first Euclidean distance of the obstacle satisfy If so, it is determined that the unmanned vehicle has collided with the obstacle.

[0028] The values ​​of the above dynamic environment model parameters are based on the following: Environmental boundary and The design aims to provide autonomous vehicles with sufficient maneuvering space while limiting their range of motion to areas covered by sensors and within manageable computational limits. Target location. Located inside the boundary and without static occlusion between it and the initial position, it is beneficial for verifying the effectiveness of the dynamic obstacle avoidance algorithm.

[0029] Target area radius The selection of this radius is based on a comprehensive consideration of the autonomous vehicle's size and stopping accuracy. This radius is larger than the autonomous vehicle's collision radius. This allows autonomous vehicles to safely stop at a lower speed within the target area; at the same time, the radius is not too large to ensure that speed control when reaching the boundary of the target area is practically meaningful.

[0030] The dynamic environment model in step S1 is used to determine the environment boundaries, target area, and obstacle distribution. In this embodiment, the number of dynamic obstacles is set to 10. This number is used to construct a dynamic scenario with sufficient collision risk and detour requirements within a limited simulation space: if the number of obstacles is too small, the autonomous vehicle's passable space is too large, making it difficult to fully verify the dynamic obstacle avoidance capability; if the number of obstacles is too large, the environment becomes too crowded, which may cause feasible paths to disappear, making training difficult to converge. The collision radius of each obstacle is specified. The collision radius is consistent with that of the autonomous vehicle, used to simulate moving objects of similar size. Obstacle linear velocity range. The speed is lower than the maximum speed of the driverless car, but the relative speed can exceed [a certain value] when moving towards each other. It can create challenging dynamic obstacle avoidance scenarios.

[0031] Initial state of driverless car In the middle, the initial position Located in the bottom left corner of the environment, initial orientation angle Initial linear velocity in the direction pointing to the target point At the usual driving speed, the initial front wheel steering angle To avoid oddly small non-zero values, the initial acceleration and steering angular velocity are both 0, indicating that the autonomous vehicle starts its mission from a stable cruise state.

[0032] In some embodiments, the regulatory task model is defined as follows: the autonomous vehicle starts from an initial state Starting from a point, the system outputs control actions through a regulatory strategy to achieve dynamic obstacle avoidance, and finally comes to a stable stop within the target area through braking. Definition Real-time location of driverless car relative to the target point location The Euclidean distance is And the absolute angle difference between the autonomous vehicle's orientation and the direction vector from the autonomous vehicle to the target position is .like If the vehicle successfully reaches the target area boundary, it is determined that the autonomous vehicle has successfully reached the boundary of the target area. Subsequently, the autonomous vehicle stops within the target area through braking. Assuming that during the braking process... This refers to the vehicle decelerating and braking along a straight line. The angle between the vehicle's orientation and the target area boundary when the autonomous vehicle reaches the target area boundary. and linear velocity To determine the task completion status. Define the expected target speed. for: This formula is used to calculate the maximum permissible speed at which an autonomous vehicle can safely brake to a stop at the boundary of a target area. Wherein, This is the maximum braking deceleration. If and Simultaneously satisfy and This means that the autonomous vehicle can decelerate and brake at maximum acceleration after reaching the boundary of the target area and eventually stop within the target area.

[0033] After completing the above model construction, we proceed to the action generation stage.

[0034] Step S2: Construct a method for generating prediction interpolation actions; The predictive interpolation action generation module performs interval prediction and linear interpolation, mapping the output of the Actor network to actual control variables that satisfy kinematic constraints. The function of this module is to ensure that the actions output by the Actor network do not exceed the physical execution capabilities of the autonomous vehicle.

[0035] Specifically, the steps for this interval prediction include: predicting the feasible movement interval for the next moment based on kinematic constraints and the current state of the autonomous vehicle. The kinematic constraints are... , , , The speed and steering angle boundary ranges of the autonomous vehicle are defined as follows: and Based on the speed at the previous moment and steering angle To obtain the control interval The achievable speed range and achievable steering angle range of the in-vehicle autonomous vehicle are as follows: ; ; By calculating the intersection of the reachable range and the boundary range, we obtain... Feasible speed range for driverless cars and feasible steering angle range Specifically: The function of the above formula is to obtain the upper and lower bounds of the actual executable speed and steering angle of the unmanned vehicle within the current control cycle, avoiding the strategy output from exceeding physical limitations. Among these, This is the lower bound of the speed for driverless cars. This represents the maximum linear velocity of the driverless car. and These are the lower and upper limits of the front wheel steering angle, respectively. This is the absolute value of the maximum linear acceleration. This is the absolute value of the maximum front wheel steering angular velocity. To control the interval. Define the action interval state. And input it into the Actor network.

[0036] In some embodiments, the specific steps of this linear interpolation are: to use the interpolation weights output by the Actor network... Actual control actions are generated through linear interpolation. .in, , The formula for calculating the actual control action is: The function of the above formula is to map the normalized network output weights to specific physical quantities, ensuring that the output always lies within the feasible range.

[0037] Once the action is generated, the risk and obstacle screening stage begins.

[0038] Step S3: Construct a risk assessment function and screen key risk barriers; The risk assessment module quantifies the collision risk of each dynamic obstacle to the autonomous vehicle and identifies the most threatening obstacles. This module reduces state space redundancy and prevents obstacles without collision potential from interfering with policy learning.

[0039] The expression for the risk assessment function is: This formula assesses the threat level of an obstacle by combining three factors: distance, relative speed, and azimuth. For the first Risk assessment indicators for dynamic obstacles; For driverless cars and the first The Euclidean distance between the obstacles; For the first The velocity vector of each obstacle; For the autonomous vehicle's velocity vector, ; To point from driverless cars to the first The unit vector of each obstacle; For the direction of the driverless car and The azimuth angle between them; This is the distance attenuation factor; and The weighting coefficients are satisfied. ; Let be the maximum linear velocity of the autonomous vehicle, and take . In this embodiment, , , .

[0040] The values ​​of the above parameters are based on the following: weighting coefficients. and This indicates that distance and speed factors are of equal importance in risk assessment. This balanced approach ensures a reasonable risk ranking when obstacles are far away but approaching at high speed, and when they are close but moving slowly. Distance attenuation factor. Make the distance term in There is a clear gradient change in scale, which matches the size of the obstacles and the autonomous vehicle in the example.

[0041] The risk assessment function mentioned above The truncation term has significant physical implications and technical benefits. It refers to the projection of the relative velocity along the line of sight. When the value is not greater than 0, it indicates that the obstacle is moving away from the driverless car or is relatively stationary with the driverless car. At this time, the obstacle does not have a forward collision tendency in the speed dimension. The truncation function forces the projection term to zero, ensuring that the risk index is determined solely by the distance exponent, without abnormally increasing or decreasing due to negative relative velocity. This design allows the risk assessment function to effectively distinguish between obstacles with a positive collision tendency and those that are far away, preventing the Actor network from wasting attention on obstacles without a collision threat, thus freeing up the neural network's exploration space to handle genuine risk sources.

[0042] In step S1, the dynamic environment is set to contain 10 obstacles. In this embodiment, the obstacles are sorted in descending order according to their risk assessment indicators, and the top 5 obstacles with the highest risk are selected as key risk obstacles. Selecting the top 5 obstacles is a trade-off between state space dimension and information sufficiency: in a scenario with 10 obstacles, the top 5 obstacles with the highest risk cover the vast majority of potential collision risks; retaining too many obstacles increases the input dimension of the Actor network, leading to training difficulties, while retaining too few may miss minor but not negligible threats.

[0043] After the screening is completed, the speed obstacle algorithm enters the sports school training stage.

[0044] Step S4: Construct a speed obstacle algorithm based on motion correction; The sports school's positive speed obstacle avoidance module combines the theoretical obstacle avoidance zone with the actual vehicle kinematic constraints to obtain an executable obstacle avoidance direction. The module's function is to ensure that the obstacle avoidance zone aligns with the reachability orientation of the Ackerman chassis, avoiding the output of unexecutable control directions.

[0045] Specifically, it is known The autonomous vehicle's facing angle at any given time is , The feasible speed range at any given time is The feasible range for the steering angle is And the theoretical obstacle avoidance range predicted by the speed obstacle algorithm. The theoretical obstacle avoidance range The obstacle avoidance range is calculated by the speed obstacle avoidance algorithm based on the relative position and relative velocity of the autonomous vehicle and each obstacle at time t. Specifically, the speed obstacle avoidance algorithm defines the set of velocities that would cause a collision between the autonomous vehicle and an obstacle at some future time as the speed obstacle cone, and the set of feasible velocities outside this speed obstacle cone constitutes the theoretical obstacle avoidance range. Introducing the wheelbase of autonomous vehicles The reachable range of the orientation at the next moment is calculated based on the Ackermann steering kinematic constraints. The calculation formula is as follows: This formula is used to calculate the actual range of the autonomous vehicle's heading angle that it can achieve in the next control cycle. Wherein, To control the interval, The lower bound of this interval is determined by the minimum speed and the minimum steering angle, while the upper bound is determined by the maximum speed and the maximum steering angle, thus ensuring that the orientation angles within the interval are all orientations that the unmanned vehicle can actually reach in the next control cycle.

[0046] The theoretical obstacle avoidance range predicted by the speed obstacle algorithm Accessible range by orientation Seeking to submit, and receiving approval from the sports school. Obstacle avoidance zone: The above intersection operation limits the theoretical collision-free direction to the range of orientations that the autonomous vehicle can actually reach in the next control cycle, eliminating control directions that, although meeting the speed obstacle conditions, cannot be executed due to wheelbase and front wheel steering angle constraints.

[0047] After the obstacle avoidance zone is determined, the multi-source hybrid observation state construction phase begins.

[0048] Step S5: Construct a multi-source hybrid observation state; The state construction module assembles the various state information into a unified observation vector for use by the Actor network. Its function is to provide the Actor network with a complete, concise input containing prior information.

[0049] Specifically, the multi-source hybrid observation state consists of the target position state, the unmanned vehicle state, the action range state, the risk obstacle state, and... The obstacle avoidance zone states together constitute the overall state, and are represented as follows: in, The target position status, In driverless mode, This is the state of the action interval. This is a risk obstacle state. for Obstacle avoidance zone status.

[0050] Target location status Represented as: This status provides the relative position information of the autonomous vehicle to the target point. Among other things, for Real-time location of driverless vehicles The target location.

[0051] The above-mentioned unmanned vehicle status Represented as: in, , and They are respectively The autonomous vehicle's orientation angle, front wheel steering angle, and linear velocity are monitored at all times.

[0052] The above-mentioned risk and obstacle status Represented as: This state provides the relative position and relative velocity information of the filtered key obstacles. Among them, and The first Individual risk barriers The position coordinate vector and velocity vector at any given moment. .

[0053] The above action range state Based on the current time of the driverless car Predicting the next moment based on state and kinematic constraints The actionable boundaries are represented as: The above Obstacle avoidance zone status Based on the current moment of the autonomous vehicle and obstacles Velocity vector prediction for the next moment The VO obstacle avoidance range is represented as: The aforementioned multi-source mixed observation state provides the Actor network with rich and concise observation information.

[0054] Once the state is established, the reward evaluation and strategy training phase begins.

[0055] Step S6: Construct an evaluation mechanism based on obstacle avoidance guidance and obstacle reward, and train the control strategy; The reward evaluation mechanism in the training module guides the update direction of the control strategy, ensuring that the strategy simultaneously meets the requirements of safe obstacle avoidance and goal achievement. This module's function is to prevent the control strategy from learning dangerous but short-sighted behaviors through a carefully designed reward structure.

[0056] The comprehensive reward function consists of obstacle avoidance zone reward, target area reward, obstacle reward, boundary penalty, and collision penalty, and is expressed as: in, As a reward for obstacle avoidance zones, Rewards for the target area. As an obstacle reward, For border punishment, As a collision penalty.

[0057] The above obstacle avoidance zone rewards Rewards from directional intervals Orientation Rewards and proximity reward Multiplication is represented as: Among them, the reward for this direction interval Defined as: the state when the autonomous vehicle's decision-making direction falls into the VO obstacle avoidance zone. hour, ;otherwise .in, This is the directional interval gain coefficient, and is a constant greater than 1, used to amplify the reward corresponding to the decision orientation falling within the VO obstacle avoidance interval; in this embodiment, .

[0058] Towards Rewards The expression is: This formula measures the degree of deviation between the autonomous vehicle's orientation and the target direction; the smaller the deviation, the higher the reward. This represents the angular deviation between the current orientation of the autonomous vehicle and the target orientation. It is a constant that takes a positive value. In this embodiment, .

[0059] Proximity Reward Represented as: This formula measures the change in distance between the autonomous vehicle and the target; the closer the distance, the greater the reward (the smaller the absolute value of the negative value). Among these... Let Euclidean distance be the distance between the autonomous vehicle and the target. It is a constant that takes a positive value. In this embodiment, .

[0060] The obstacle avoidance interval reward employs a multiplicative gating structure, exhibiting a non-linear "cliff effect" (i.e., when the decision orientation deviates from the safe zone, the absolute value of the obstacle avoidance interval reward drops abruptly). When the decision orientation is within the VO obstacle avoidance interval, the directional interval reward takes a gain greater than 1 and is multiplied by the orientation reward and proximity reward. In this case, the proximity reward is negative, the orientation reward is a positive value less than or equal to 1, and the product is negative. When the decision orientation leaves the VO obstacle avoidance interval, the directional interval reward degenerates to 1, and the absolute value of the overall obstacle avoidance interval reward decreases instantaneously. This multiplicative mechanism means that the superior performance of the control strategy on any single metric cannot compensate for the lack of a safe orientation: even if the autonomous vehicle is rapidly approaching the target, once the orientation deviates from the safe zone, the negative contribution of the proximity reward is weakened by the disappearance of the directional interval gain, causing the overall reward to drop rapidly. In contrast, in the traditional additive reward structure, a large positive reward for target approach may offset the obstacle avoidance penalty, causing the control strategy to learn dangerous but short-sighted behavior. The multiplicative design in this embodiment avoids this problem, enabling the control strategy to simultaneously satisfy the requirements of target approach and dynamic obstacle avoidance.

[0061] In some embodiments, the aforementioned target area reward Based on the terminal velocity of the autonomous vehicle when it reaches the boundary of the target area and expected speed Confirmed. When the autonomous vehicle reaches the boundary of the target area and meets the following conditions... and A positive reward is given when the condition is met; otherwise, a zero reward is given. The specific expression is: in, The target area reward coefficient is taken in this embodiment. .

[0062] In some embodiments, the aforementioned obstacle rewards The safe radius is determined based on the safe radius between the autonomous vehicle and each dynamic obstacle, the obstacle penalty distance, and the obstacle reward distance. Obstacle penalty distance Obstacle bonus distance For the first An obstacle, if If so, a positive reward will be given; if If a negative penalty is imposed, a zero reward is given in all other cases. In this embodiment, , The obstacle reward expression is: in, and These are the positive reward coefficient and the penalty coefficient for obstacles, respectively. In this embodiment, , .

[0063] In some embodiments, the aforementioned boundary penalty and collision penalty All are set to relatively large negative values; in this embodiment, we take... , .

[0064] The values ​​of the parameters in the above reward function are based on the following: Directional range gain coefficient The value of ...

[0065] Orientation reward coefficient and proximity reward coefficient The value of was determined through simulation and debugging: if If the size is too large, the control strategy may focus too much on orientation alignment and neglect lateral obstacle avoidance; if If the value is too high, the reward for getting closer to the target will outweigh the safety guidance, causing the strategy to tend to rush straight towards the target. The chosen value should maintain a balance between reaching the target and avoiding obstacles.

[0066] Target Area Reward Coefficient It is used to provide appropriate positive incentives when the task is completed, and its value is on the same order of magnitude as the obstacle avoidance interval reward, so as to avoid the end reward becoming dominant.

[0067] Obstacle Positive Reward Coefficient and obstacle penalty coefficient The ratio reflects a cautious approach to collision risk: a safe distance provides a positive reward, but a dangerous distance incurs a slightly lower penalty to prevent an overly conservative strategy from failing to reach the target.

[0068] Obstacle penalty distance and reward distance It is based on the sum of the collision radii of the autonomous vehicle and the obstacle ( This is set by [the system / mechanism]. When the distance between the autonomous vehicle and the obstacle is less than [a certain value]... Punishment should be given in time, greater than Rewards are given in certain areas, while no rewards are given in the middle areas, thus forming a reasonable buffer zone.

[0069] Boundary penalties and collision penalty The purpose of using negative values ​​that are much larger than the absolute value of the regular reward is to create a hard constraint effect during training. Once an out-of-bounds error or collision occurs, the current round is terminated to prevent the strategy from continuing to explore from a failed state.

[0070] After the reward mechanism is defined, the control policy is trained based on this mechanism using the Proximal Policy Optimization (PPO) framework. The training process includes the following steps: Step A1: Initialize the Actor network, Critic network, experience buffer, shearing coefficient, number of iterations, batch size, discount factor, and network learning rate; Step A2: Initialize the dynamic environment at the start of each training session by obtaining the initial observation state through the motion school positive velocity obstacle module, prediction interpolation method and risk assessment mechanism; Step A3: Within the current control cycle, the multi-source hybrid observation status will be... Input to the Actor network, output action interpolation weights from the Actor network. ; Step A4: Perform linear interpolation within the movable action range based on the action interpolation weights to generate the autonomous vehicle control actions. It interacts with the dynamic environment to obtain the state at the next moment. and rewards ; Step A5: Set the current state Action interpolation weights ,award and the state at the next moment Form experience tuples and store them in the experience buffer; Step A6: When the experience buffer capacity reaches the batch capacity, calculate the cumulative reward and advantage estimate, and update the Actor network and Critic network parameters in small batches; Step A7: Clear the experience buffer and enter the next control cycle until the maximum number of training steps per scene and the total number of reinforcement learning training steps are reached.

[0071] Within each control cycle of the above training process, the functional modules involved in steps A3 to A4 operate in the following sequence: Predictive interpolation, VO interval solution, and PPO network inference are performed within a single control cycle. Memory operates under strict timing constraints. Within the control cycle, the prediction interpolation action generation module first generates the action based on the state of the previous time step. Calculate the current actionable range Simultaneously, the speed obstacle avoidance module of the motion school calculates the theoretical obstacle avoidance range based on the current obstacle state and the autonomous vehicle state, and completes the motion school correction to obtain the VO obstacle avoidance range. ; then, , The data is assembled with the target location status, the unmanned vehicle status, and the risk obstacle status into a multi-source hybrid observation status. Actor network reception Post-output interpolation weights Finally, the linear interpolation module according to exist Generate actual control quantity within the range And it acts on the environment. In the above process, the output of each module strictly depends on the result of the previous module, and all calculations must be performed within the preceding module. Completed within the time window to ensure the real-time nature of control commands.

[0072] To verify the learning efficiency and control performance of the method and system of this invention in complex dynamic environments, the method of this invention (i.e., the PPO path control method based on speed obstacle and prediction interpolation, denoted as VP-PPO) is compared with the comparative method that uses the enhanced vector field histogram algorithm to output the safe interval (denoted as IP-PPO) under the same unmanned vehicle model, dynamic obstacle environment and reinforcement learning hyperparameter conditions.

[0073] like Figure 2 As shown, the reward values ​​of both methods increase with training, but the VP-PPO reward curve rises rapidly in the early stages of training and enters a stable oscillation range with higher reward values ​​after about 200 policy updates; the reward curve of the IP-PPO method rises relatively slowly. These results indicate that the motion-corrected VO obstacle avoidance range and risk assessment mechanism can reduce blind trial and error by the control strategy in dynamic environments.

[0074] like Figure 3 As shown, in different dynamic test scenarios, the paths planned by the IP-PPO method have response lag or redundant detours, while the VP-PPO method can adjust the driving direction in advance according to the movement trend of dynamic obstacles, forming a smoother detour path while maintaining target orientation.

[0075] like Figure 4 and Figure 5As shown, the front wheel steering angle under the IP-PPO strategy exhibits significant rapid changes and local oscillations, while the steering angle and speed control variables under the VP-PPO strategy show more continuous overall changes. This difference indicates that the combination of VO obstacle avoidance range and predictive interpolation action generation not only provides dynamic collision prediction but also reduces the adverse effects of abrupt control variables on the actuators.

[0076] like Figure 6 As shown, in the process of transfer learning in multiple dynamic scenarios, the smooth reward value of the VP-PPO method is generally higher than that of the IP-PPO method, and the fluctuation range is relatively smaller, indicating that the method of the present invention has better adaptability to different dynamic obstacle distributions.

[0077] In one example, across 100 random dynamic test scenarios, as shown in Table 1, the number of completions for VP-PPO(TL), IP-PPO(TL), VP-PPO(RL), and IP-PPO(RL) were 75, 48, 40, and 27, respectively. Here, TL represents the transfer learning strategy, and RL represents the strategy learned from scratch. The task completion rate for VP-PPO(TL) was 75%, a 27 percentage point improvement over IP-PPO(TL); the task completion rate for VP-PPO(RL) was 40%, a 13 percentage point improvement over IP-PPO(RL). These quantitative results demonstrate that in random dynamic scenarios containing multiple moving obstacles, the combined effect of the intersection of positive VOs in the motion school, risk obstacle screening, and multiplicative gating rewards can improve the task completion ability and environmental adaptability of the regulatory strategy.

[0078] Table 1. Number of times VP-PPO and IP-PPO tasks were completed in random dynamic test scenarios: To further summarize Figures 2 to 5 The comparison results shown, combined with the statistical results of 100 random dynamic test scenarios in Table 1, comprehensively compare the method of the present invention with the comparative methods. The above comparative experiments used the same unmanned vehicle kinematic model, dynamic obstacle environment, task completion judgment rules, and reinforcement learning training hyperparameters to ensure the consistency of the comparison results.

[0079] Table 2 summarizes the comparison results of the two methods in terms of reward convergence, path planning, control smoothness, and task completion rate. The results for reward convergence, path planning, and control smoothness are based on... Figures 2 to 5 For comparison, the task completion rate was determined based on the statistical results of 100 random dynamic test scenarios in Table 1.

[0080] Table 2 shows that IP-PPO exhibits a slower reward increase in dynamic environments, with path response lag or redundant detours, and significant fluctuations in steering angle and speed control. VP-PPO, on the other hand, achieves higher rewards early in training, can adjust its direction in advance based on the movement trends of dynamic obstacles, and outputs more continuous and smooth control values. In 100 random dynamic test scenarios, the task completion rates of VP-PPO(TL) and VP-PPO(RL) were 75% and 40%, respectively, both higher than IP-PPO(TL)'s 48% and IP-PPO(RL)'s 27%. These results indicate that the positive VO interval of the motion school, risk obstacle screening, and multiplicative gating rewards can synergistically improve training efficiency, control smoothness, and task completion ability in dynamic environments.

[0081] Table 2 compares the simulation results and phenomena: In summary, this invention, through the coordinated use of predictive interpolation action generation, risk assessment function screening, the positive velocity obstacle interval of the motion school, and the reward product mechanism, ensures that the control actions output by the regulatory strategy are always within the kinematically feasible range of the autonomous vehicle. Simultaneously, it reduces the interference of redundant obstacle states on strategy learning, ensures that the obstacle avoidance direction is consistent with the actual reachable orientation, and effectively coordinates target approach and dynamic obstacle avoidance. The results of the embodiments demonstrate that this invention outperforms the comparative methods in terms of task completion rate, reward convergence speed, control smoothness, and transfer learning performance in complex dynamic environments.

[0082] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention. Such modifications or substitutions should all fall within the scope of the invention, or any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. A path planning and control method for unmanned vehicles based on speed obstacle avoidance and predictive interpolation, characterized in that, Includes the following steps: A dynamic planning and control system model is established, which includes an unmanned vehicle kinematics model, a dynamic environment model, and a planning and control task model. A predictive interpolation action generation method is constructed, which calculates the feasible interval of an action in real time through interval prediction and outputs the control action through linear interpolation. A risk assessment function is constructed, which uses the relative distance exponent, the projection truncation term of relative velocity along the line of sight, and the azimuth angle weight to quantify the collision risk. Based on the risk assessment indicators, the obstacles with the highest risk level are selected as key risk obstacles. Construct a speed obstacle course algorithm based on motion correction, and introduce the wheelbase of the autonomous vehicle. The achievable orientation range for the next moment is calculated based on the current orientation, speed feasible range, and steering angle feasible range of the autonomous vehicle. And compare the achievable orientation range with the theoretical obstacle avoidance range. Find the intersection and obtain the VO obstacle avoidance range corrected by the sports school. ; The proximal policy optimization algorithm framework is used to train the rule and control policy. The multi-source mixed observation state is used as the observation data input to the Actor network. The action weights are output based on the Actor network, and the product of the direction interval reward, the orientation reward and the proximity reward is used as the obstacle avoidance interval reward to update the rule and control policy.

2. The method according to claim 1, characterized in that, The state variables of the autonomous vehicle kinematics model satisfy the following constraints: ; in, For driverless vehicle line speed, For linear acceleration, The front wheel steering angular velocity, This is the front wheel steering angle.

3. The method according to claim 1, characterized in that, In the aforementioned prediction interpolation action generation method, the interval prediction is based on the linear velocity of the previous moment. Front wheel steering angle Absolute value of maximum linear acceleration Absolute value of maximum front wheel steering angular velocity and control interval Calculate the feasible speed range within the current control cycle. and feasible steering angle range ,in: ; in, This is the lower bound of the speed for driverless cars. This represents the maximum linear velocity of the driverless car. and These are the lower and upper limits of the front wheel steering angle, respectively. The linear interpolation is based on the interpolation weights output by the Actor network. and Generate the linear velocity in the actual control action. and front wheel steering angle : ; 。 4. The method according to claim 1, characterized in that, The expression for the risk assessment function is: ; in, For the first Risk assessment indicators for dynamic obstacles For driverless cars and the first Euclidean distance between obstacles For the first The velocity vector of each obstacle For the autonomous vehicle's velocity vector, To point from driverless cars to the first The unit vector of each obstacle. For the direction of the driverless car and The azimuth angle between them For distance attenuation factor, and The weighting coefficients are and satisfy the following conditions: , This represents the maximum linear velocity of the autonomous vehicle.

5. The method according to claim 1, characterized in that, In the speed obstacle algorithm based on motion correction, the direction towards the reachable interval at the next moment is... The calculation formula is: ; in, for The driverless car is always facing the angle. and They are respectively The lower and upper bounds of the feasible velocity interval at any given moment. and They are respectively The lower and upper bounds of the feasible interval for the turning angle at any given time. To control the interval, This refers to the wheelbase of the autonomous vehicle. The VO obstacle avoidance range corrected by the sports school is: 。 6. The method according to claim 1, characterized in that, The multi-source hybrid observation state From the target position state Autonomous vehicle status Action range state Risk and obstacle status and VO obstacle avoidance zone status Composition, represented as: ; Wherein, the target location state , and The target location coordinates, and The current coordinates of the driverless car; The state of the unmanned vehicle , , and They are respectively The autonomous vehicle's orientation angle, front wheel steering angle, and linear velocity are monitored at all times. The action range state ; The state of the risk obstacle ,in For the first Individual risk barriers Position coordinates at that moment For the first Individual risk barriers The velocity vector at time t, For the autonomous vehicle's velocity vector, ; The VO obstacle avoidance zone state .

7. The method according to claim 1, characterized in that, The obstacle avoidance zone reward Rewards from directional intervals Orientation Rewards and proximity reward Multiplication is represented as: ; Specifically, when the autonomous vehicle's decision-making direction falls within the VO obstacle avoidance zone, ;otherwise ; Here is the gain coefficient for the directional interval, and Take a constant greater than 1; The orientation reward ,in This represents the angular deviation between the current orientation of the autonomous vehicle and the target orientation. It is a constant that takes a positive value; The proximity reward ,in Let Euclidean distance be the distance between the autonomous vehicle and the target. It is a constant that takes a positive value.

8. The method according to claim 7, characterized in that, The comprehensive reward function of the control strategy for: ; in, As a reward for obstacle avoidance zones, Rewards for the target area. As an obstacle reward, For border punishment, Penalty for collision; The target area reward For: when and hour, ;otherwise ,in The angle between the target's orientation when reaching the boundary of the target area. The linear velocity when reaching the boundary of the target area. , For maximum braking deceleration, The radius of the target area. The target area reward coefficient; The obstacle reward , among which when hour ,when hour Other cases ,and , for Driverless cars and the first The Euclidean distance between the obstacles Penalty distance for obstacles, Add distance as a bonus for obstacles. The positive reward coefficient for obstacles. The obstacle penalty coefficient, The collision radius of the autonomous vehicle. The radius of the obstacle collision; The boundary penalty and collision penalty All are negative constants.

9. The method according to claim 1, characterized in that, The training process for training the control strategy using the proximal policy optimization algorithm framework includes: Initialize the Actor network, Critic network, experience buffer, shearing coefficient, number of iterations, batch size, discount factor, and network learning rate; At the start of each training session, the dynamic environment is initialized by obtaining the initial observation state through the sports school positive speed obstacle module, prediction interpolation method and risk assessment mechanism; Within the current control period, the multi-source mixed observation state is input into the Actor network, and the Actor network outputs the action interpolation weights. Based on the action interpolation weights, linear interpolation is performed within the actionable range to generate autonomous vehicle control actions, and the vehicle interacts with the dynamic environment to obtain the state and reward for the next moment. The current state, action interpolation weights, reward, and next time step state are combined into an experience tuple and stored in the experience buffer. When the experience buffer capacity reaches the batch capacity, the cumulative reward and advantage estimate are calculated, and the parameters of the Actor network and Critic network are updated in small batches. Clear the experience buffer and enter the next control cycle until the maximum number of training steps per scene and the total number of reinforcement learning training steps are reached.

10. A path planning and control system for unmanned vehicles based on speed obstacle avoidance and predictive interpolation, characterized in that, include: The model building module is used to build the kinematic model, dynamic environment model and control task model of the autonomous vehicle, providing environmental state description and constraints for subsequent action generation, risk assessment and strategy training. The predictive interpolation action generation module is used to calculate the feasible speed range and the feasible steering angle range for the next moment based on the current state and kinematic constraints of the unmanned vehicle, and to map the interpolation weights output by the Actor network to the actual control actions that are always within the feasible speed range and the feasible steering angle range through linear interpolation. The risk assessment module is used to calculate the risk assessment index of each dynamic obstacle based on the relative distance, the projection of the relative speed along the line of sight, and the azimuth angle, and to screen out the obstacles with the highest risk level as key risk obstacles. The sports school positive speed obstacle module is used to introduce the wheelbase of the unmanned vehicle to calculate the orientation reachable interval at the next moment, and then intersect this orientation reachable interval with the theoretical speed obstacle interval to obtain the VO obstacle avoidance interval that satisfies the Ackermann steering kinematics constraint. The state construction module is used to assemble the target location state, the unmanned vehicle state, the action range state, the risk obstacle state, and the VO obstacle avoidance range state into a multi-source hybrid observation state. The training module is used to train the rule and control strategy using a near-end policy optimization algorithm framework. The obstacle avoidance interval reward is composed of the product of the direction interval reward, the orientation reward, and the proximity reward, so as to guide the rule and control strategy to simultaneously meet the requirements of safe obstacle avoidance and target arrival.