Path planning optimization methods, devices, electronic equipment, storage media and products

By simulating vehicle driving in a simulation environment and optimizing the path planning of unmanned articulated vehicles based on real-time status, the problems of path tracking deviation and safety accidents in existing technologies are solved, and precise path tracking control is achieved.

CN122083969BActive Publication Date: 2026-11-17NETEASE LINGDONG (HANGZHOU) TECHNOLOGY CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202610069686.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-01-19
Publication Date
2026-11-17
Estimated Expiration
2046-01-19

AI Technical Summary

Technical Problem

Existing path planning algorithms for unmanned articulated vehicles cannot accurately depict changes in vehicle shape and motion model, leading to path tracking errors and safety risks.

Method used

By simulating vehicle driving in a simulation environment, recommended driving actions are obtained based on real-time driving status, vehicle driving status and pose are updated, and the planned path is optimized.

Benefits of technology

It improves the accuracy of path planning, enables precise path tracking and control, and reduces the risk of collisions and safety accidents.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122083969B_ABST
    Figure CN122083969B_ABST
Patent Text Reader

Abstract

This application discloses a path planning optimization method, apparatus, electronic device, storage medium, and product, relating to the field of autonomous driving technology. The method includes: simulating a vehicle driving along a currently planned path in a simulation environment; predicting the vehicle's real-time recommended driving actions based on the vehicle's real-time driving state; updating the vehicle's driving state based on the real-time recommended driving actions and obtaining the vehicle's real-time pose until the vehicle reaches the end point of the currently planned path; and optimizing the currently planned path based on the vehicle's real-time pose. This application can optimize the planned path of the vehicle and improve the accuracy of path planning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to a path planning optimization method, device, electronic device, storage medium, and product. Background Technology

[0002] Currently, path planning for unmanned articulated vehicles generally reuses path planning algorithms used for passenger cars. This means that the path is only required to meet certain curvature constraints or curvature change constraints, or certain assumptions are made about the movement of the loader, and mathematical modeling is performed. Based on the assumed model, optimization methods are used to solve the planning problem.

[0003] However, for articulated vehicles such as loaders, their shape changes with movement, and this shape change determines the shape of the collision detection frame. Furthermore, their motion model also varies depending on whether they are under load. Therefore, path planning models or fixed mathematical models for passenger cars cannot accurately depict the motion of this type of vehicle, potentially leading to efficiency losses or tracking errors. This could result in safety accidents caused by planning paths that loaders cannot accurately track.

[0004] Optimizing path planning for unmanned articulated vehicles has become a pressing technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, this application provides a path planning optimization method, apparatus, electronic device, storage medium, and product to improve the path planning problem of unmanned articulated vehicles.

[0006] Firstly, this application provides a path planning optimization method, the method comprising: The simulated vehicle travels along the currently planned path in a simulation environment, and the real-time recommended driving action of the vehicle is predicted based on the real-time driving status of the vehicle. The vehicle's driving status is updated based on the vehicle's real-time recommended driving actions, and the vehicle's real-time pose is obtained until the vehicle reaches the end of the currently planned path. The current planned path is optimized based on the vehicle's real-time pose.

[0007] Secondly, this application provides a path planning optimization apparatus, the apparatus comprising: The action prediction module is used to simulate a vehicle driving along the currently planned path in a simulation environment and predict the vehicle's real-time recommended driving actions based on the vehicle's real-time driving status. The status update module is used to update the driving status of the vehicle based on the real-time recommended driving actions of the vehicle, and to obtain the real-time position and pose of the vehicle until the vehicle reaches the end of the current planned path. The path optimization module is used to optimize the currently planned path based on the real-time pose of the vehicle.

[0008] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the path planning optimization method of the first aspect or any corresponding embodiment described above.

[0009] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the path planning optimization method of the first aspect or any corresponding embodiment described above.

[0010] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the path planning optimization method described in the first aspect or any corresponding embodiment thereof.

[0011] The path planning optimization method provided in this application, after planning the vehicle's planned path, simulates the vehicle driving along the currently planned path in a simulation environment. During the simulated vehicle driving process, it obtains the vehicle's real-time recommended driving actions based on the vehicle's real-time driving status and updates the vehicle's driving status based on the real-time recommended driving actions, thereby optimizing the vehicle's real-time pose and achieving the purpose of optimizing the vehicle's planned path. This improves the accuracy of path planning and enables precise path tracking and control of the vehicle in application, which is beneficial to reducing the risk of safety accidents such as collisions. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 A schematic diagram illustrating an application scenario provided in an embodiment of this application; Figure 2 A flowchart illustrating a path planning optimization method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a method for obtaining recommended driving actions for a vehicle according to an embodiment of this application. Figure 4 This is a flowchart illustrating a method for updating the driving status of a vehicle according to an embodiment of this application. Figure 5 This is a flowchart illustrating a method for acquiring the real-time pose of a vehicle according to an embodiment of this application. Figure 6 This is a schematic diagram of the training architecture of a reinforcement learning model in an embodiment of this application; Figure 7 This is a schematic diagram illustrating an application scenario of path planning optimization in the embodiments of this application; Figure 8 A structural block diagram of a path planning optimization device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0016] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. As an optional application scenario for an embodiment of this application, such as... Figure 1 As shown, the system may include at least one terminal device and at least one server. Figure 1 The system, as exemplarily shown, includes a vehicle 101, a terminal device 102, and a server 103, which can be connected to the server 103 via a network 110.

[0017] Vehicle 101 can be an unmanned vehicle, which is a vehicle that can navigate, drive and work autonomously without human operation through sensors, artificial intelligence and control systems. It is usually equipped with various sensing modules, vehicle terminals, communication modules, autonomous driving systems, etc. Vehicle 101 can communicate and interact with terminal equipment 102 and server 103 through the vehicle communication module and upload the sensed vehicle data.

[0018] The terminal device 102 can be a smartphone, tablet, laptop, handheld computer, desktop computer, game console, smart TV, smart wearable device, vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. The terminal device 102 can communicate and interact with the vehicle 101 and the server 103 through the vehicle network to realize remote control of the vehicle 101.

[0019] Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server or edge server that provides cloud services. Server 103 can communicate and interact with vehicle 101 through network 110 to realize path planning, autonomous driving control, and other functions for vehicle 101.

[0020] Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranets, local area networks, wide area networks, mobile communication networks, and combinations thereof.

[0021] In some application scenarios, vehicle 101 is an articulated vehicle. An articulated vehicle (also known as an "articulated vehicle") is a special vehicle that connects two car bodies into a whole through an articulated structure. Its core features are "segmented load-bearing and flexible connection", which can expand the loading space while maintaining a certain degree of maneuverability. It is widely used in public transportation, special transportation, engineering operations and other fields.

[0022] Unmanned articulated vehicles (such as unmanned loaders) refer to unmanned articulated vehicles, which are articulated vehicles that can navigate, drive and work autonomously without human operation through sensors, artificial intelligence and control systems.

[0023] A loader is an articulated vehicle used for material handling and loading, typically found in construction sites, mines, ports, and railways. It features high mobility, high loading efficiency, and ease of operation, and is commonly used for loading heavy materials such as earth, rock, coal, and ore. Automated loaders can complete operations autonomously without human intervention, thereby improving loading and operational efficiency. In the operation of unmanned loaders, many steps require movement, and determining the appropriate movement trajectory is crucial.

[0024] Currently, path planning for unmanned loaders generally reuses path planning algorithms used for passenger cars. This means that the path is only required to meet certain curvature constraints or curvature change constraints, or certain assumptions are made about the loader's motion, and mathematical modeling is performed. Based on the assumed model, optimization methods are used to solve the planning problem.

[0025] However, for articulated vehicles such as loaders, their shape changes with movement, and this shape change determines the shape of the collision detection frame. Furthermore, their motion model also varies depending on whether they are under load. Therefore, path planning models or fixed mathematical models for passenger cars cannot accurately depict the motion of this type of vehicle, potentially leading to efficiency losses or tracking errors, and even causing safety accidents by planning paths that loaders cannot accurately track.

[0026] Optimizing path planning for unmanned articulated vehicles has become a pressing technical problem that needs to be solved.

[0027] Therefore, embodiments of this application provide a path planning optimization method, apparatus, electronic device, storage medium, and product. After planning the planned path for a vehicle, the method simulates the vehicle driving along the currently planned path in a simulation environment. During the simulated vehicle driving process, the method obtains the vehicle's real-time recommended driving actions based on the vehicle's real-time driving status and updates the vehicle's driving status based on the real-time recommended driving actions, thereby optimizing the vehicle's real-time pose and achieving the purpose of optimizing the vehicle's planned path. This improves the accuracy of path planning and enables precise path tracking and control of the vehicle in applications, which helps reduce the risk of safety accidents such as collisions.

[0028] In practical applications, the vehicle is an unmanned articulated vehicle. By optimizing the vehicle's pose through the path planning optimization method of this application embodiment, the path planning optimization of the unmanned articulated vehicle can be improved, thus reducing the deviation of the path tracking control of the unmanned articulated vehicle, achieving precise path tracking control, and reducing the risk of safety accidents such as collisions during path tracking control.

[0029] According to an embodiment of this application, a path planning optimization method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This application provides a path planning optimization method that can be used for the aforementioned server. Figure 2 A flowchart illustrating a path planning optimization method provided in this application embodiment is shown below. Figure 2 As shown, the path planning optimization method includes the following steps S201 to S204.

[0031] Step S201: Simulate the vehicle driving along the currently planned path in the simulation environment, and obtain the vehicle's real-time recommended driving actions based on the vehicle's real-time driving status.

[0032] The simulation environment is a simulation environment built based on the actual environment in which the vehicle is located. The current planned path is a driving path that is pre-planned and generated for the vehicle in the actual environment.

[0033] After planning the vehicle's current planned path in the actual environment, the vehicle is placed at the starting point of the current planned path in the simulation environment, and the vehicle is simulated to travel along the current planned path until the vehicle reaches the end point of the current planned path.

[0034] During the simulation of a vehicle traveling along the currently planned path, the recommended driving action of the vehicle is predicted in real time based on the real-time driving status of the vehicle in the simulation environment.

[0035] Real-time driving status refers to the vehicle's motion state at any given moment, including but not limited to the vehicle's speed and turning angle; real-time recommended driving actions are the optimal driving control commands predicted based on the vehicle's driving status at any given moment, including but not limited to the vehicle's speed adjustment and turning angle adjustment.

[0036] Step S202: Update the vehicle's driving status based on the vehicle's real-time recommended driving actions and obtain the vehicle's real-time pose until the vehicle reaches the end of the currently planned path.

[0037] After obtaining the recommended driving actions of the vehicle in real time, the control adjustment corresponding to the recommended driving actions is executed to update the vehicle's driving status. After the vehicle's driving status is updated, the recommended driving actions and the vehicle's driving status are iteratively updated as time progresses, thereby continuously simulating the vehicle's driving process along the current planned path until the vehicle reaches the end of the current planned path.

[0038] Real-time pose is the vehicle's pose at any given moment. Pose includes the vehicle's position and attitude information. The attitude information can be represented by the heading angle (also known as the yaw angle). As time progresses, the vehicle will exhibit different poses under different driving conditions.

[0039] Step S203: Optimize the current planned path based on the vehicle's real-time pose.

[0040] After the simulated vehicle reaches the end of the currently planned path, the current planned path is optimized based on the real-time pose of the vehicle during the simulated driving process.

[0041] During the simulated vehicle's journey along the currently planned path, as time progresses, the recommended driving actions of the vehicle are predicted and obtained in real time based on the vehicle's real-time driving status. The vehicle's driving status is then updated based on the real-time recommended driving actions, thereby optimizing the vehicle's real-time pose. The real-time pose represents the vehicle's position and attitude at each moment. By recording the vehicle's real-time pose during the simulated vehicle's journey, the simulated vehicle's driving path can be determined as the optimized driving path, thus optimizing the currently planned path.

[0042] According to the path planning optimization method provided in the embodiments of this application, after the planned path of the vehicle is planned, the vehicle is simulated to drive along the current planned path in a simulation environment. During the simulated driving process, the real-time recommended driving actions of the vehicle are obtained in real time based on the real-time driving status of the vehicle, and the driving status of the vehicle is updated according to the real-time recommended driving actions, thereby optimizing the real-time pose of the vehicle and achieving the purpose of optimizing the planned path of the vehicle. This improves the accuracy of path planning and enables precise path tracking and control of the vehicle in application, which is beneficial to reducing the risk of safety accidents such as collisions.

[0043] In some embodiments, a reinforcement learning model is used to predict the recommended driving action of the output vehicle in real time, and a kinematic model is used to update the driving state and pose of the vehicle in real time, thereby simulating the driving process of the vehicle along the current planned path in the simulation environment until the vehicle reaches the end of the current planned path.

[0044] Figure 3 This is a flowchart illustrating a method for obtaining recommended driving actions for a vehicle according to an embodiment of this application. In some embodiments, such as... Figure 3 As shown, in step S201 above, predicting the vehicle's real-time recommended driving action based on the vehicle's real-time driving status can further include: Step S301: During the simulated vehicle driving process, input the vehicle's current driving state into the pre-trained reinforcement learning model.

[0045] Initially, the vehicle's driving state is the state at the starting point of the current planned path. The driving state of the vehicle at each subsequent moment is the driving state updated by the kinematic model.

[0046] During the simulated vehicle driving process, the vehicle's current driving state (such as speed, turning angle, etc.) is acquired, and the current driving state of the vehicle is input into a pre-trained reinforcement learning model to predict recommended driving actions and output them in real time. The recommended driving actions of the vehicle at the current moment (such as speed adjustment amount and turning angle adjustment amount, etc.) are predicted.

[0047] Step S302: Using a reinforcement learning model, obtain the recommended driving action of the vehicle at the current moment based on the current driving state.

[0048] Reinforcement learning is a machine learning method that learns optimal strategies through the interaction between an agent and its environment to maximize cumulative rewards.

[0049] During the simulated vehicle driving process, the vehicle acts as a reinforcement learning agent. Based on the kinematic model and the simulation environment, an interactive environment for reinforcement learning is built. Through the interaction between the reinforcement learning environment and the agent (vehicle), the vehicle's driving status (such as speed, turning angle, etc.) is obtained in real time. The reinforcement learning model learns the control strategy for the vehicle to drive along the currently planned path in the simulation environment. Based on the vehicle's driving status, the recommended driving actions of the vehicle are predicted and output (such as speed adjustment and turning angle adjustment). The kinematic model updates the vehicle's driving status (such as speed, turning angle, etc.) in real time based on the recommended driving actions, thereby realizing real-time tracking and optimization of the vehicle's current planned path.

[0050] In some embodiments, the vehicle's driving state includes, but is not limited to, the vehicle's speed, turning angle, and load, and the vehicle's recommended driving actions include, but are not limited to, the vehicle's speed adjustment amount and turning angle adjustment amount.

[0051] Figure 4 This is a flowchart illustrating a vehicle driving status update method according to an embodiment of this application. In some embodiments, such as... Figure 4 As shown, in step S202 above, updating the vehicle's driving status based on the vehicle's real-time recommended driving actions can further include: Step S401: Input the vehicle's recommended driving action at the current moment into the preset kinematic model.

[0052] Step S402: Determine the vehicle's driving state at the next moment based on the recommended driving action at the current moment using a preset kinematic model.

[0053] A kinematic model is a mathematical model that describes the quantitative relationship between a vehicle's pose (position, heading angle) and control inputs (front wheel steering angle, vehicle speed) based on geometric relationships and kinematic principles. For example, a kinematic model can be implemented using a world model.

[0054] Specifically, during the simulated vehicle driving process, the recommended driving action of the vehicle at the current moment, output by the reinforcement learning model, is fed back to the preset kinematic model. The preset kinematic model determines the vehicle's driving state at the next moment based on the recommended driving action of the vehicle at the current moment, thereby realizing the vehicle driving state update.

[0055] For example, the recommended driving action of the vehicle at the current moment includes the vehicle's speed adjustment and steering angle adjustment. These adjustments are input into a kinematic model. The kinematic model determines the vehicle's speed at the next moment based on the current speed and its speed adjustment, and determines the vehicle's steering angle at the next moment based on the current steering angle and its steering angle adjustment. The model then simulates and controls the vehicle to drive at the next moment based on this speed and steering angle. Here, the speed at the next moment is the sum of the current speed and its speed adjustment, and the steering angle at the next moment is the sum of the current steering angle and its steering angle adjustment.

[0056] Figure 5 This is a flowchart illustrating a method for acquiring the real-time pose of a vehicle according to an embodiment of this application. In some embodiments, such as... Figure 5 As shown, in step S202 above, obtaining the real-time pose of the vehicle may further include: Step S501: Using a kinematic model, obtain the change in the vehicle's pose at the current moment based on the vehicle's current driving state.

[0057] Step S502: Determine the vehicle's pose at the next moment based on the vehicle's pose at the current moment and the change in pose at the current moment.

[0058] The pose change at the current moment refers to the change in the vehicle's pose from the start of the current moment to the end of the current moment. The vehicle's pose includes, but is not limited to, the vehicle's position and heading angle. The pose change includes position change and heading angle change. The position change can include lateral position change and longitudinal position change. The vehicle's pose at the initial moment is the initialized pose. The pose at each subsequent moment is predicted based on the driving state of the previous moment.

[0059] A kinematic model is a mathematical model that describes the quantitative relationship between a vehicle's pose (position, heading angle) and control inputs (front wheel steering angle, vehicle speed). The current driving state can include the vehicle's speed and steering angle at the current moment. By inputting the vehicle's current driving state into the kinematic model, the change in the vehicle's pose at the current moment can be predicted, and the vehicle's pose at the next moment can be determined based on the current pose and the change in pose at the current moment.

[0060] For example, the vehicle's driving state includes speed and turning angle, where speed is represented by v and turning angle by δ. The vehicle's pose includes position and orientation angle, where position is represented by x and y, and orientation angle by θ. x is the vehicle's lateral position coordinate, and y is the vehicle's longitudinal position coordinate. Therefore, the vehicle's driving state at the current moment can be expressed as (speed v...). k δ kThe vehicle's pose at the current moment can be represented as (lateral position coordinate x). k Vertical position coordinate y k Direction angle θ k ), k represents the current time, and the change in vehicle pose at the current time can be expressed as (lateral position change). x k Change in longitudinal position y k Change in angle θ k ).

[0061] In the kinematic model, the change in the vehicle's lateral position at the current moment. x k With the vehicle's current speed v k and rotation angle δ k There exists a certain geometric relationship, the change in vertical position at the current moment. y k With the vehicle's current speed v k and rotation angle δ k There exists a certain geometric relationship between the change in angle at the current moment. θ k With the vehicle's current speed v k and rotation angle δ k There are certain geometric relationships. Based on the geometric relationships in the kinematic model, and according to the vehicle's current driving state (speed v)... k δ k This can determine the vehicle's pose change (lateral position change) at the current moment. x k Change in longitudinal position y k Change in angle θ k Based on the vehicle's pose at the current moment (lateral position coordinate x) k Vertical position coordinate y k Direction angle θ k ) and the change in pose at the current moment (change in lateral position) x k Change in longitudinal position y k Change in angle θ k This allows us to determine the vehicle's pose (lateral position coordinate x) at the next time step k+1. k+1 Vertical position coordinate y k+1 Direction angle θ k+1 ), where x k+1 =x k+ x k y k+1 =y k + y k θ k+1 =θ k + θ k .

[0062] In some embodiments, in step S203 above, optimizing the current planned path based on the vehicle's real-time pose may further include: generating a target planned path based on the vehicle's real-time pose; and adjusting the current planned path to the target planned path.

[0063] The vehicle's real-time pose includes the vehicle's pose at every moment from the start to the end of the current planned path. Based on the vehicle's pose at every moment during the simulated vehicle driving process, a target planned path is generated, which is the actual simulated vehicle driving trajectory. This target planned path replaces the current planned path and is used as the optimized planned path for the vehicle.

[0064] Specifically, the vehicle's pose at each moment during the simulated driving process is connected in chronological order to obtain the target planned path.

[0065] During the simulated vehicle driving process, the vehicle's pose at each moment is determined through real-time optimization and updating by combining reinforcement learning models and kinematic models. Therefore, based on the vehicle's pose at each moment determined during the simulated vehicle driving process, the vehicle's current planned path can be optimized.

[0066] In some embodiments, when an abnormal situation occurs during the simulated vehicle driving process, the path planning optimization is terminated, and the currently planned path is output. Abnormal situations include, for example, the reinforcement learning model failing to execute, a timeout, or a collision being detected between the vehicle and an obstacle in the simulation environment at the current driving state. For collision detection, shape detection can be used to determine whether a collision has occurred between the vehicle and an obstacle in the simulation environment; alternatively, the distance between the vehicle and the obstacle can be determined by comparing the vehicle's pose at the current driving state with the pose of the obstacle in the simulation environment, and the collision status can be determined based on the distance.

[0067] In some embodiments, after adjusting the current planned path to the target planned path, the path planning optimization method further includes: controlling the vehicle to travel along the target planned path in the actual environment.

[0068] Specifically, the target planned path is sent to the vehicle's underlying controller, which then tracks and controls the vehicle to travel along the target planned path in the actual environment.

[0069] In some embodiments, before the simulated vehicle travels along the currently planned path in the simulation environment, the path planning optimization method further includes: constructing a simulation environment based on map information of the actual environment in which the vehicle is currently located.

[0070] In some embodiments, the path planning optimization method further includes: identifying obstacle information in the actual environment through an environmental sensing device; and updating the simulation environment based on the obstacle information.

[0071] For example, the vehicle is an unmanned loader, and the actual operating environment of the unmanned loader is a mixing plant scenario. A simulation environment is constructed using a mixing plant scenario map.

[0072] Environmental sensing devices can be cameras, radar devices, or other equipment capable of sensing and identifying obstacles in the real environment. They can be used to identify obstacle information in the real environment and add corresponding obstacles at the corresponding locations in the simulation environment.

[0073] In some embodiments, before the simulated vehicle travels along the currently planned path in the simulation environment, the path planning optimization method further includes: performing path planning on the vehicle using a preset path planning algorithm to generate the currently planned path.

[0074] This application does not impose any special restrictions on the path planning algorithm; any suitable path planning algorithm can be used. For example, the preset path planning algorithm can be a graph search-based path planning algorithm, a sampling-based path planning algorithm, or an intelligent optimization algorithm. Among them, graph search-based path planning algorithms include Dijkstra's algorithm, A* algorithm, D*Lite algorithm, LPA* algorithm, etc.; sampling-based path planning algorithms include RRT (Rapidly-exploring RandomTree) algorithm, PRM (Probabilistic Roadmap Method) algorithm, etc.; and intelligent optimization algorithms include genetic algorithms, ant colony algorithms, particle swarm optimization algorithms, etc.

[0075] In some embodiments, the vehicle's current planned path may also be a path planned manually.

[0076] In some embodiments, before optimizing the path planning of the vehicle, a kinematic model of the vehicle is trained. The training method of the kinematic model includes, but is not limited to, the following steps: collecting historical vehicle driving data of the training vehicle, which includes the driving state of the training vehicle at each historical moment and the corresponding vehicle pose; training the kinematic model based on the historical vehicle driving data, which is used to predict the pose change of the training vehicle at the current moment based on the driving state of the training vehicle at the historical moment. For example, a data-driven neural network model is used to train the kinematic model based on historical vehicle driving data to predict the pose change.

[0077] Specifically, by installing data acquisition devices (such as sensors, cameras, radar, etc.) on training vehicles, the driving state (such as speed, turning angle, load, etc.) and corresponding vehicle pose (such as position, direction angle, etc.) of the training vehicles at each historical moment are collected by the data acquisition devices when the training vehicles perform the operation tasks. The kinematic model is trained by the neural network algorithm based on the historical vehicle driving data collected by the training vehicles. The model takes the driving state of the training vehicles at historical moments, such as turning angle, speed, load, etc., as input and predicts the change in pose of the training vehicles at the current moment until the model converges. This kinematic model can fit the actual operation of the training vehicles in the environment as closely as possible.

[0078] In some embodiments, before performing path planning optimization on the vehicle, a reinforcement learning model is trained to train a reinforcement learning model for path planning optimization. The training method of the reinforcement learning model includes, but is not limited to, the following steps: Step 1: Construct a training environment based on the map information of the actual environment and the preset obstacle information.

[0079] For example, the vehicle is an unmanned loader, and the actual environment in which the loader is located is usually a mixing plant scenario. The mixing plant scenario map is used as the training environment, and preset obstacle information is randomly added to the training environment.

[0080] After the training environment is built, a set of training paths is obtained based on the training environment. This set includes multiple training paths, which may include manually deployed training paths and training paths generated using path planning algorithms (such as the RRT algorithm).

[0081] Step 2: In the current training batch, for each preset training path, perform the following training steps: Steps 3 to 5.

[0082] Step 3: Simulate the training vehicle driving along the current training path in the training environment. Using a reinforcement learning model, obtain the real-time recommended driving actions of the training vehicle based on its real-time driving status.

[0083] For example, a reinforcement learning model is used to obtain the real-time recommended driving actions of the training vehicle based on the real-time driving status of the training vehicle using the proximal policy optimization (PPO) algorithm.

[0084] In the training environment, the training vehicle is placed at the starting point of the current training path, and the training vehicle is simulated to travel along the current training path until the training vehicle reaches the end point of the current training path.

[0085] During the simulation of the training vehicle traveling along the current training path, the real-time driving status of the training vehicle is input into the reinforcement learning model. The reinforcement learning model outputs the probability of each driving action according to the policy and selects the recommended driving action from it.

[0086] The driving status can include the speed and turning angle of the training vehicle, and the recommended driving actions can include the speed adjustment and turning angle adjustment of the vehicle.

[0087] Step 4: Evaluate the immediate state reward of the training vehicle according to the preset reward function.

[0088] The reward functions include, but are not limited to, the following functions: a reward function for whether the training vehicle has reached the end of the current training path; a reward function for whether the training vehicle collides with an obstacle in the training environment at the current moment; a reward function for the shortest distance between the training vehicle and an obstacle at the current moment; a reward function for the speed of the training vehicle at the current moment; a reward function for the shortest distance between the pose of the training vehicle and the training path at the current moment; and a fixed reward function for each moment of the training vehicle's movement.

[0089] By designing the above reward function, the training vehicle can be encouraged to move along the training path toward the end of the training path, while unreasonable behaviors such as driving close to obstacles or driving too fast when turning are punished. The specific reward function can be designed according to actual needs, and this application embodiment does not impose any special restrictions on it.

[0090] Step 5: Update the vehicle's driving status and corresponding pose based on the real-time recommended driving actions of the training vehicle until the vehicle reaches the end of the current training path.

[0091] In some embodiments, the vehicle's driving state and pose are updated based on the real-time recommended driving actions of the training vehicle using a preset kinematic model, and the process returns to step 3 to continue learning the recommended driving actions of the vehicle in the updated driving state until the vehicle reaches the end of the current training path.

[0092] Step 6: After the current training batch is completed, update the model parameters of the reinforcement learning model according to the accumulated instantaneous state rewards corresponding to each training path.

[0093] In some embodiments, the PPO (Proximal Policy Optimization) algorithm is used to train the reinforcement learning model for updating model parameters. PPO is a policy gradient algorithm in reinforcement learning that improves training stability by limiting the policy update magnitude.

[0094] Step 7: Train the next training batch until the model converges and a reinforcement learning model is obtained.

[0095] Figure 6 This is a schematic diagram of the training architecture of a reinforcement learning model in an embodiment of this application, as shown below. Figure 6 As shown, in some embodiments, the training architecture of the reinforcement learning model includes an agent, an environment, and a reinforcement learning algorithm. The agent loads the reinforcement learning model, and the environment loads the kinematic model. The agent uses the reinforcement learning model to predict and output recommended driving actions for the training vehicle in real time. The environment uses the kinematic model to update the driving state and pose of the vehicle in real time and provides immediate state rewards for the training vehicle. The reinforcement learning algorithm iteratively updates the model parameters of the reinforcement learning model. When the model convergence condition is met, the reinforcement learning model is trained and obtained.

[0096] In some embodiments, the kinematic model and reinforcement learning model can be trained offline. After training, the converged kinematic model and reinforcement learning model are deployed to the edge machine (server) of the vehicle for online path planning optimization.

[0097] Figure 7 This is a schematic diagram illustrating an application scenario of path planning optimization in an embodiment of this application, such as... Figure 7 As shown, in practical applications, the vehicle is an unmanned loader, and the actual environment is a mixing plant scenario. In the vehicle's driving route in the figure, the solid line is the current planned path before path planning optimization, and the dashed line is the target planned path generated by the path planning optimization method according to the embodiment of this application. According to actual tests, the overall length of the optimized target planned path is basically unchanged compared with the current planned path before optimization, the average length is slightly reduced, and the shortest distance between the vehicle and obstacles in the environment is improved after path optimization.

[0098] This application also provides a path planning optimization device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0099] Figure 8 This is a structural block diagram of a path planning optimization device provided in an embodiment of this application. The embodiment of this application provides a path planning optimization device, such as... Figure 8 As shown, the path planning optimization device includes, but is not limited to, the following modules: action prediction module 801, state update module 802, and path optimization module 803.

[0100] The motion prediction module 801 is used to simulate a vehicle driving along the currently planned path in a simulation environment and predict the vehicle's real-time recommended driving actions based on the vehicle's real-time driving status.

[0101] The status update module 802 is used to update the vehicle's driving status based on the vehicle's real-time recommended driving actions and to obtain the vehicle's real-time pose until the vehicle reaches the end of the currently planned path.

[0102] The path optimization module 803 is used to optimize the currently planned path based on the real-time pose of the vehicle.

[0103] The path planning optimization apparatus provided in this application can execute the path planning optimization method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0104] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0105] The following is a detailed reference. Figure 9 This diagram illustrates a suitable structural schematic for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 901, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 902 or a program loaded from memory 908 into random access memory (RAM) 903. RAM 903 also stores various programs and data required for the operation of the electronic device. The processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0106] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 9 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0107] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a memory 908, or installed from a ROM 902. When the computer program is executed by the processor 901, it performs the functions defined in the path planning optimization method of embodiments of this application.

[0108] Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0109] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the path planning optimization method shown in the above embodiments is implemented.

[0110] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0111] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A path planning optimization method, characterized in that, The method includes: The simulated vehicle travels along the currently planned path in a simulation environment, and the real-time recommended driving action of the vehicle is predicted based on the real-time driving status of the vehicle. The vehicle's driving status is updated based on the vehicle's real-time recommended driving actions, and the vehicle's real-time pose is optimized based on the updated driving status until the vehicle reaches the end of the currently planned path. Optimize the current planned path based on the vehicle's real-time pose; the optimization of the current planned path based on the vehicle's real-time pose includes: generating a target planned path based on the vehicle's real-time pose; and adjusting the current planned path to the target planned path.

2. The method of claim 1, wherein, The step of predicting the real-time recommended driving action of the vehicle based on its real-time driving status includes: During the simulated vehicle driving process, the vehicle's current driving state is input into a pre-trained reinforcement learning model; The reinforcement learning model is used to obtain the recommended driving action of the vehicle at the current moment based on the current driving state.

3. The method of claim 1, wherein, The step of updating the vehicle's driving status based on the vehicle's real-time recommended driving actions includes: Input the vehicle's recommended driving action at the current moment into a preset kinematic model; The vehicle's driving state at the next moment is determined based on the recommended driving action at the current moment using a preset kinematic model.

4. The method of claim 1, wherein, The step of optimizing the real-time pose of the vehicle based on the updated driving status includes: The vehicle's pose change at the current moment is obtained based on the vehicle's driving state at the current moment using a preset kinematic model. The vehicle's pose at the next moment is determined based on the vehicle's pose at the current moment and the change in pose at the current moment.

5. The method according to claim 1, characterized in that, Before the simulated vehicle travels along the currently planned path in the simulation environment, the method further includes: The simulation environment is constructed based on map information of the actual environment in which the vehicle is currently located.

6. The method according to claim 5, characterized in that, The method further includes: The obstacle information in the actual environment is identified through an environmental sensing device; The simulation environment is updated based on the obstacle information.

7. The method according to claim 1, characterized in that, Before the simulated vehicle travels along the currently planned path in the simulation environment, the method further includes: The vehicle is route-planned using a preset route planning algorithm to generate the current planned route.

8. The method according to claim 1, characterized in that, The vehicle in question is an articulated vehicle.

9. The method according to claim 4, characterized in that, The method further includes: Collect historical vehicle driving data of the training vehicle, which includes the driving status and corresponding pose of the training vehicle at each historical moment. The kinematic model is trained based on the historical vehicle driving data, and the kinematic model is used to predict the change in pose of the training vehicle at the current moment based on the driving state of the training vehicle at historical moments.

10. The method according to claim 2, characterized in that, The method further includes: The training environment is constructed based on map information of the actual environment and pre-set obstacle information; In the current training batch, for each preset training path, perform the following training steps: The simulated training vehicle travels along the current training path in the training environment. Through a reinforcement learning model, the real-time recommended driving action of the training vehicle is obtained based on the real-time driving status of the training vehicle. The training vehicle's immediate state reward is evaluated based on a preset reward function. The vehicle's driving status is updated based on the real-time recommended driving actions of the training vehicle until the vehicle reaches the end of the current training path. After the current training batch is completed, the model parameters of the reinforcement learning model are updated according to the accumulated instantaneous state rewards corresponding to each training path. The next training batch is then trained until the model converges, thus obtaining the reinforcement learning model.

11. The method according to claim 10, characterized in that, The reward function includes: The reward function for determining whether the training vehicle has reached the end of the current training path; The reward function for determining whether the training vehicle collides with obstacles in the training environment at the current moment; The reward function corresponding to the shortest distance between the training vehicle and the obstacle in the current driving state; The reward function corresponding to the speed of the training vehicle in its current driving state; The reward function corresponding to the pose of the training vehicle in its current driving state and the shortest distance of the training path; The fixed reward function corresponding to each moment of the training vehicle's operation.

12. A path planning optimization device, characterized in that, The device includes: The action prediction module is used to simulate a vehicle driving along the currently planned path in a simulation environment and predict the vehicle's real-time recommended driving actions based on the vehicle's real-time driving status. The status update module is used to update the driving status of the vehicle based on the real-time recommended driving actions of the vehicle, and optimize the real-time pose of the vehicle based on the updated driving status until the vehicle reaches the end of the current planned path. The path optimization module is used to optimize the currently planned path based on the real-time pose of the vehicle. The optimization of the currently planned path based on the real-time pose of the vehicle includes: generating a target planned path based on the real-time pose of the vehicle; and adjusting the currently planned path to the target planned path.

13. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the path planning optimization method according to any one of claims 1 to 11 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the path planning optimization method according to any one of claims 1 to 12.

15. A computer program product, characterized in that, It includes computer instructions for causing a computer to perform the path planning optimization method according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • Unmanned vehicle path planning method based on deep reinforcement learning and A star algorithm

    CN115933629A