Unmanned intelligent vehicle path planning method and unmanned intelligent vehicle

Abstract

The invention relates to an unmanned intelligent vehicle path planning method and an unmanned intelligent vehicle, and belongs to the technical field of path planning, and the method comprises the steps: obtaining the state information of the unmanned intelligent vehicle at each time step; the state information comprises position information and traffic congestion information of an area where the unmanned intelligent vehicle is located; determining a moving action according to the state information, and determining an instant reward after the unmanned intelligent vehicle executes the moving action; the instant reward comprises a traffic congestion negative reward; calculating a Q value according to the instant reward through a Q-Learning algorithm; learning a path planning strategy according to the Q value; and performing path planning on the target unmanned intelligent vehicle to be subjected to path planning according to the learned target path planning strategy. When path planning is carried out through the target path planning strategy, path planning can be carried out in combination with real-time traffic congestion information, traffic congestion hot spot areas are avoided, balanced distribution of road network loads is achieved, and the overall utilization efficiency of a road network is improved.

Description

Technical Field

[0001] This invention relates to the field of path planning technology, and in particular to a path planning method for unmanned intelligent vehicles and an unmanned intelligent vehicle. Background Technology

[0002] With the rapid development of intelligent manufacturing and automated logistics, the deployment scale of unmanned intelligent vehicle systems in industrial workshops, warehousing and logistics parks, and unmanned production lines continues to expand. In complex and dynamic environments, unmanned intelligent vehicles need to collaboratively complete tasks such as material transfer, equipment inspection, and cargo sorting, and the rationality of their path planning directly affects the system's efficiency and safety.

[0003] Existing autonomous vehicle path planning methods, such as the A-Star Algorithm, Dijkstra's algorithm, and Rapidly-exploring Random Tree (RRT) algorithm, have solved the autonomous vehicle path planning problem to some extent. However, in autonomous vehicle collaborative scenarios, they still have problems such as high computational complexity and the tendency for planned paths to lead to local congestion while other areas remain relatively idle. These issues make it difficult to meet the needs of efficient collaboration of autonomous vehicles in complex dynamic environments. Summary of the Invention

[0004] In view of this, it is necessary to provide a path planning method and an autonomous vehicle to solve the problems of high computational complexity and the inability of the planned path to effectively utilize the overall road network.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a path planning method for unmanned intelligent vehicles, comprising: The state information of the unmanned intelligent vehicle at each time step is obtained; the state information includes location information and traffic congestion information of the area where the unmanned intelligent vehicle is located; The movement action is determined based on the state information, and the immediate reward is determined after the unmanned intelligent vehicle performs the movement action. The Q-value is calculated based on the instantaneous reward using the Q-Learning algorithm; Based on the Q-value, a learning path planning strategy is developed. Based on the learned target path planning strategy, perform path planning for the target autonomous vehicle to be planned; The instant reward is calculated using the following formula:

[0006]

[0007] In the formula, t For time steps, For autonomous intelligent vehicles in status Perform movement actions at time The immediate reward afterwards For autonomous intelligent vehicles in status Perform movement actions at time The subsequent positive reward, Negative incentives for traffic congestion. Let the traffic congestion cost function be... The traffic congestion penalty coefficient is a non-negative hyperparameter. and As a weighting factor, For autonomous intelligent vehicles in status Perform movement actions at time The local traffic density of the area where the autonomous vehicle is located. For autonomous intelligent vehicles in status Perform movement actions at time The path overlap rate after that.

[0008] In one possible implementation, calculating the Q-value based on the reward using the Q-Learning algorithm includes: The Q value is calculated using the following formula:

[0009] In the formula, In the state Perform movement actions at time Q-value estimation at time, To The Q-value estimate obtained by updating For learning rate, As a discount factor, From state The maximum Q value among all moving actions initiated.

[0010] In one possible implementation, the immediate reward also includes at least one of the following: a negative reward for encountering an obstacle, a conflict negative reward, and a deadlock negative reward.

[0011] In one possible implementation, the conflict negative reward includes negative rewards for oncoming conflicts and intersection conflicts.

[0012] In one possible implementation, obtaining the state information of the unmanned intelligent vehicle at each time step includes: A grid environment map is constructed based on the mobile environment of the unmanned intelligent vehicle; The location information of the unmanned intelligent vehicle in the grid environment map is determined, and the traffic congestion information of the area where the unmanned intelligent vehicle is located is determined based on the location information.

[0013] In one possible implementation, determining the movement action based on the state information includes: The movement action is determined based on the state information using a greedy strategy.

[0014] In one possible implementation, determining the movement action based on the state information includes: The movement action is determined based on the state information using the DQN model; The calculation of the Q-value based on the immediate reward using the Q-Learning algorithm includes: The Q value is calculated based on the immediate reward using the DQN model. The learning path planning strategy based on the Q-value includes: The loss value is determined based on the Q value, and the parameters of the DQN model are adjusted in reverse based on the loss value until the DQN model is trained; wherein the trained DQN model includes the target path planning strategy.

[0015] In one possible implementation, the method further includes: Samples are drawn from the experience replay pool to train the DQN model.

[0016] Secondly, the present invention also provides an unmanned intelligent vehicle, including a memory and a processor, wherein the memory is used to store a program; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the unmanned intelligent vehicle path planning method described in any of the above claims.

[0017] The beneficial effects of this invention are: This invention employs a reinforcement learning method based on the Q-Learning algorithm to learn a target path planning strategy. During the learning process, traffic congestion information is incorporated into the autonomous vehicle's state information. After selecting a movement action based on the traffic congestion information, the immediate reward after the autonomous vehicle executes that action is calculated. This immediate reward includes a traffic congestion negative reward. A Q-value is then calculated based on this immediate reward, and the path planning strategy is learned based on the Q-value. This allows the learned target path planning strategy to incorporate real-time traffic congestion information, avoiding traffic hotspots, achieving a balanced distribution of road network load, and improving the overall utilization efficiency of the road network. Furthermore, this invention uses the Q-Learning algorithm for path planning strategy learning, which, while ensuring computational efficiency, allows for faster learning and finding of the optimal path planning strategy, i.e., the target path planning strategy. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an embodiment of the unmanned intelligent vehicle path planning method provided by the present invention; Figure 2 This is a schematic diagram of an embodiment of the unmanned intelligent vehicle provided by the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] In the description of the embodiments of this invention, unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," etc., used in the embodiments of this invention are used to distinguish similar objects, and are not used to describe a specific order or sequence, nor to indicate or imply their relative importance or implicitly specify the number of indicated technical features. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, and the number of objects is not limited; for example, a first object can be one or more.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] Reference Figure 1 The diagram illustrates a flowchart of an embodiment of the unmanned intelligent vehicle path planning method provided by the present invention. The method includes: S101, acquire the status information of the unmanned intelligent vehicle at each time step; the status information includes location information and traffic congestion information of the area where the unmanned intelligent vehicle is located.

[0024] Unmanned intelligent vehicles are intelligent mobile vehicles based on artificial intelligence (AI), autonomous driving, sensor fusion, and decision-making and control technologies. They are capable of autonomously perceiving the environment, planning routes, and executing tasks with little or no intervention. Unmanned intelligent vehicles can include: unmanned logistics delivery vehicles, unmanned industrial operation vehicles, unmanned special operation vehicles, and so on.

[0025] Location information can include the location information of the autonomous vehicle corresponding to the time step, and traffic congestion information can include the number of vehicles in the area where the autonomous vehicle is located and the degree of traffic congestion in the area where the autonomous vehicle is located.

[0026] S102 determines the movement action based on the status information, and determines the immediate reward after the unmanned intelligent vehicle performs the movement action.

[0027] A greedy strategy can be used to select a movement action based on state information, and then the immediate reward after the autonomous vehicle performs the movement action can be determined according to a preset reward function. The immediate reward includes the traffic congestion negative reward obtained by comparing and evaluating the traffic congestion information before and after the movement action.

[0028] Specifically, the instant reward is calculated using the following formula:

[0029]

[0030] In the formula, t For time steps, For autonomous intelligent vehicles in status Perform movement actions at time The immediate reward afterwards For autonomous intelligent vehicles in status Perform movement actions at time The subsequent positive reward, Negative incentives for traffic congestion. Let the traffic congestion cost function be... The traffic congestion penalty coefficient is a non-negative hyperparameter. and As a weighting factor, For autonomous intelligent vehicles in status Perform movement actions at time The local traffic density of the area where the autonomous vehicle is located. For autonomous intelligent vehicles in status Perform movement actions at time The path overlap rate after that.

[0031] In this embodiment, positive rewards can reflect the movement actions performed by the autonomous vehicle. The positive benefits afterward include approaching the target location and successfully avoiding obstacles, where the target location can be the endpoint of the path planning. Local traffic density can be calculated by the number of vehicles within a preset area centered on the autonomous vehicle. Path overlap rate can be calculated based on the number of vehicles within the autonomous vehicle's movement actions. Then, the path overlap rate between the path already traveled by the autonomous vehicle and the paths already traveled by other autonomous vehicles is calculated. When there are multiple other autonomous vehicles, the path overlap rate between the autonomous vehicle and each of the other autonomous vehicles can be calculated separately, and then the path overlap rates can be added together and divided by the number of other autonomous vehicles.

[0032] S103 calculates the Q value based on immediate rewards using the Q-Learning algorithm.

[0033] Q-Learning is a classic model-free reinforcement learning algorithm, belonging to the temporal difference learning method. It learns an action-value function (Q-function), enabling the agent to find the optimal policy through trial and error in an unknown environment.

[0034] The calculated Q value can be used to update the Q value in the Q table (path planning strategy).

[0035] S104, learn path planning strategy based on Q value.

[0036] When the algorithm converges, the optimal path planning strategy, also known as the target path planning strategy, can be obtained.

[0037] S105 performs path planning for the target unmanned intelligent vehicle to be planned based on the learned target path planning strategy.

[0038] The unmanned intelligent vehicle path planning method provided in this embodiment can be applied to an unmanned intelligent vehicle path planning system, which can be a software system running on a terminal device. The terminal device can be a tablet computer, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), mobile phone, etc. This embodiment does not impose any restrictions on the specific type of terminal device.

[0039] In summary, this embodiment employs a reinforcement learning method based on the Q-Learning algorithm to learn the target path planning strategy. During the learning process, traffic congestion information is incorporated into the autonomous vehicle's state information. After selecting a movement action based on the traffic congestion information, the reward value of the autonomous vehicle after executing that action is calculated. This reward value includes negative rewards for traffic congestion. The Q-value is then calculated based on this reward value, and the path planning strategy is learned based on the Q-value. This allows the learned target path planning strategy to incorporate real-time traffic congestion information, avoiding traffic hotspots, achieving a balanced distribution of road network load, and improving the overall utilization efficiency of the road network. Furthermore, this embodiment uses the Q-Learning algorithm for path planning strategy learning, which, while ensuring computational efficiency, allows for faster learning and finding of the optimal path planning strategy, i.e., the target path planning strategy.

[0040] The existing Q-value update strategy is shown in the following formula:

[0041] In the formula, t For time steps, In the state Perform movement actions at time Q-value estimation at time, To The Q-value estimate obtained by updating For learning rate, As a discount factor, From state The maximum Q value among all starting moves. For instant rewards.

[0042] In some embodiments of the present invention, an improved reward mechanism is introduced into the above-mentioned Q-value update strategy, incorporating local congestion information into the reward and penalty values ​​of the Q-Learning algorithm, resulting in the following Q-value update strategy:

[0043] In the formula, This is the immediate reward calculated based on the negative reward for traffic congestion.

[0044] In some embodiments of the present invention, the immediate reward may also include at least one of the following: a negative reward for encountering an obstacle, a conflict negative reward, and a deadlock negative reward.

[0045] Conflicts can include oncoming conflicts and intersection conflicts. Oncoming conflicts refer to two or more autonomous vehicles traveling towards each other in the same lane or narrow space. Intersection conflicts refer to conflicts caused by multiple autonomous vehicles crossing paths at intersections without traffic lights or complex intersections. Therefore, the negative rewards for conflicts can include the negative rewards for oncoming conflicts and intersection conflicts.

[0046] A deadlock occurs when multiple autonomous vehicles become stuck in an indefinite state of congestion because they are waiting for each other to release resources (such as lanes or intersections), making it impossible for them to continue moving. For example, four vehicles are blocking one side of an intersection, each waiting for other vehicles to go first (similar to a "four-way parking deadlock"). Another example is resource contention deadlock, where multiple vehicles simultaneously request the same narrow lane and refuse to give way to each other.

[0047] In one example of this embodiment, the instant reward can be calculated using the following formula:

[0048] In the formula, This represents the positive reward for performing a movement action to reach the target location. This represents the negative reward when encountering an obstacle. This indicates a negative reward for traffic congestion.

[0049] This embodiment introduces negative rewards for encountering obstacles, conflicts, and deadlocks, enabling the learned target path planning strategy to adjust the path in real time to cope with dynamic obstacles, conflicts, and deadlocks while ensuring global optimality.

[0050] In some embodiments of the present invention, S101 includes: Construct a grid-based environment map based on the mobile environment of the autonomous intelligent vehicle; Determine the location information of the autonomous vehicle in the grid environment map, and determine the traffic congestion information of the area where the autonomous vehicle is located based on the location information.

[0051] In some embodiments of the present invention, the mobile environment of the unmanned intelligent vehicle includes: a workshop, a logistics park, or a production line.

[0052] When using the Q-Learning algorithm for path planning in gridded environments, its core principle is to leverage the powerful search capabilities of computers to find the optimal solution through continuous trial and error between the agent and the environment. However, when the number of autonomous vehicles is large, the algorithm's search speed decreases, the processing time increases, and it becomes difficult to achieve efficient path planning in complex scenarios. Therefore, a path planning algorithm based on Deep Q-Network (DQN) in deep reinforcement learning is considered to realize path planning for autonomous vehicles. The DQN model is an extended version of Q-Learning. While retaining the core ideas of Q-Learning, this algorithm introduces a deep neural network to approximate the Q function, allowing reinforcement learning to be applied to high-dimensional state spaces. At the same time, the experience replay mechanism improves data utilization, reduces training time, and improves sample processing efficiency and training stability.

[0053] Therefore, in some embodiments of the present invention, the step of determining a movement action based on state information may include: determining the movement action based on the state information using a DQN model. S103 may include: calculating a Q-value based on an immediate reward using a DQN model. S104 may include: determining a loss value based on the Q-value, and adjusting the parameters of the DQN model in reverse based on the loss value until the DQN model is trained; wherein the trained DQN model includes a target path planning strategy.

[0054] Once trained, the DQN model learns the optimal mapping relationship (target path planning strategy) between state information and movement actions. This mapping relationship can guide the autonomous vehicle to make corresponding movement actions based on the state information at each time step.

[0055] In this embodiment, the loss value is calculated using the following formula:

[0056] In the formula, It is the action network of the DQN model that pairs states. and actions Q-value estimation, Indicates the target network in state Next, calculate all possible actions. The corresponding Q value is then selected, and the largest Q value is chosen.

[0057] In some embodiments of the present invention, firstly, a regular grid environment map is constructed as the operating scenario for the autonomous vehicle. The state of the autonomous vehicle is defined as its position within the grid, and its action space includes being stationary and moving in four directions. A reward function is designed to guide the autonomous vehicle's learning; for example, a positive reward is given for reaching the target position, and a negative reward is given for encountering obstacles, conflicts, or congestion.

[0058] Then, the Q-table is initialized. At each time step, the autonomous vehicle perceives the current state and, based on... The greedy strategy selects a movement action, receives an immediate reward after execution, and transitions to a new state. Then, it adjusts the Q-values ​​in the Q-table according to the Q-Learning update rule. After multiple training iterations and parameter optimizations, the algorithm converges, forming an optimal path planning strategy that considers load balancing (the final determined Q-table). This optimal path planning strategy enables autonomous vehicles to avoid load hotspots, achieving a balanced distribution of road network load and improving the overall system operating efficiency.

[0059] Furthermore, a DQN model can be used instead of a Q-table. The process of learning the optimal path planning strategy using a DQN model is as follows: Initialize the DQN model; the input of the DQN model is state information, and the output is the Q-value of the corresponding movement action; at each time step, the DQN model obtains the current state and, based on... - A greedy strategy is used to select actions. After the autonomous vehicle performs a movement action, it receives an immediate reward and transitions to a new state. The state transition data is placed in an experience replay pool. Samples are randomly drawn from the experience replay pool to train the DQN model and update the Q-value. The parameters of the target network of the DQN model are periodically frozen to stabilize the learning process. After multiple training and parameter optimizations, the algorithm converges, forming the optimal path planning strategy for the autonomous vehicle, ensuring that the autonomous vehicle system can operate efficiently and stably.

[0060] This embodiment constructs a rasterized model of the environment map and combines it with Q-Learning reinforcement learning to achieve accurate path initialization planning, ensuring the global feasibility of the path planning. Compared with traditional path planning methods, this embodiment uses Q-Learning for path search and optimization. By continuously updating the state-action value function (Q-value), it improves the accuracy of path planning while ensuring computational efficiency, enabling the autonomous vehicle to learn and find feasible paths more quickly.

[0061] In terms of global path optimization, this invention, through the design of a reasonable state space, action space, and reward function, enables Q-Learning to gradually learn the optimal path strategy and minimize path conflicts in autonomous vehicle collaborative scenarios. Compared to traditional path planning methods, Q-Learning can continuously optimize path planning strategies through autonomous learning in dynamic environments, reducing computational complexity, improving the adaptability of path planning, and enabling autonomous vehicle systems to achieve better task execution efficiency in complex environments. This solution has moderate computational load, wider applicability, and stronger robustness, making it particularly suitable for applications with high requirements for real-time and collaborative path planning, such as intelligent manufacturing, warehousing and logistics, and disaster relief.

[0062] Reference Figure 2The present invention illustrates an unmanned intelligent vehicle 300. The unmanned intelligent vehicle 300 includes a processor 301, a memory 302, and a display 303. Figure 2 Only some components of the autonomous intelligent vehicle 300 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0063] In some embodiments, processor 301 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 302 or process data, such as the unmanned intelligent vehicle path planning method of the present invention.

[0064] In some embodiments, processor 301 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 301 may be local or remote. In some embodiments, processor 301 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, etc., or any combination thereof.

[0065] In some embodiments, memory 302 may be an internal storage unit of the autonomous vehicle 300, such as a hard drive or memory of the autonomous vehicle 300. In other embodiments, memory 302 may also be an external storage device of the autonomous vehicle 300, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the autonomous vehicle 300.

[0066] Furthermore, the memory 302 may include both internal storage units of the autonomous vehicle 300 and external storage devices. The memory 302 is used to store application software and various types of data installed on the autonomous vehicle 300.

[0067] In some embodiments, display 303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 303 is used to display information from the autonomous vehicle 300 and to display a visual user interface. Components 301-303 of the autonomous vehicle 300 communicate with each other via a system bus.

[0068] In one embodiment, when the processor 301 executes the autonomous vehicle path planning program in the memory 302, the following steps can be implemented: Acquire the status information of the autonomous vehicle at each time step; the status information includes location information and traffic congestion information of the area where the autonomous vehicle is located. The movement action is determined based on the status information, as well as the immediate reward after the autonomous vehicle performs the movement action. The Q-value is calculated based on immediate rewards using the Q-Learning algorithm; Learning path planning strategies based on Q-value; Based on the learned target path planning strategy, perform path planning for the target autonomous vehicle to be planned; The instant reward is calculated using the following formula:

[0069]

[0070] In the formula, t For time steps, For autonomous intelligent vehicles in status Perform movement actions at time The immediate reward afterwards For autonomous intelligent vehicles in status Perform movement actions at time The subsequent positive reward, Negative incentives for traffic congestion. Let the traffic congestion cost function be... The traffic congestion penalty coefficient is a non-negative hyperparameter. and As a weighting factor, For autonomous intelligent vehicles in status Perform movement actions at time The local traffic density of the area where the autonomous vehicle is located. For autonomous intelligent vehicles in status Perform movement actions at time The path overlap rate after that.

[0071] It should be understood that when the processor 301 executes the unmanned intelligent vehicle path planning program in the memory 302, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0072] This embodiment employs a reinforcement learning method based on the Q-Learning algorithm to learn the target path planning strategy. During the learning process, traffic congestion information is incorporated into the autonomous vehicle's state information. After selecting a movement action based on the traffic congestion information, the reward value of the autonomous vehicle after executing the movement action is calculated. This reward value includes a negative reward for traffic congestion. The Q-value is then calculated based on this reward value, and the path planning strategy is learned based on the Q-value. This allows the learned target path planning strategy to incorporate real-time traffic congestion information, avoiding traffic hotspots, achieving a balanced distribution of road network load, and improving the overall utilization efficiency of the road network. Furthermore, this invention uses the Q-Learning algorithm for path planning strategy learning, which, while ensuring computational efficiency, can more quickly learn and find the optimal path planning strategy, i.e., the target path planning strategy.

[0073] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0074] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A path planning method for an unmanned intelligent vehicle, characterized in that, include: Obtain the status information of the unmanned intelligent vehicle at each time step; The status information includes location information and traffic congestion information in the area where the unmanned intelligent vehicle is located; The movement action is determined based on the state information, and the immediate reward is determined after the unmanned intelligent vehicle performs the movement action. The Q-value is calculated based on the instantaneous reward using the Q-Learning algorithm; Based on the Q-value, a learning path planning strategy is developed. Based on the learned target path planning strategy, perform path planning for the target autonomous vehicle to be planned; The instant reward is calculated using the following formula: In the formula, t For time steps, For autonomous intelligent vehicles in status Perform movement actions at time The immediate reward afterwards For autonomous intelligent vehicles in status Perform movement actions at time The subsequent positive reward, Negative incentives for traffic congestion. Let the traffic congestion cost function be... The traffic congestion penalty coefficient is a non-negative hyperparameter. and As a weighting factor, For autonomous intelligent vehicles in status Perform movement actions at time The local traffic density of the area where the autonomous vehicle is located. For autonomous intelligent vehicles in status Perform movement actions at time The path overlap rate after that.

2. The unmanned intelligent vehicle path planning method according to claim 1, characterized in that, The calculation of the Q-value based on the immediate reward using the Q-Learning algorithm includes: The Q value is calculated using the following formula: In the formula, In the state Perform movement actions at time Q-value estimation at time, To The Q-value estimate obtained by updating For learning rate, As a discount factor, From state The maximum Q value among all moving actions initiated.

3. The unmanned intelligent vehicle path planning method according to claim 1, characterized in that, The immediate reward also includes at least one of the following: a negative reward for encountering an obstacle, a conflict negative reward, and a deadlock negative reward.

4. The unmanned intelligent vehicle path planning method according to claim 3, characterized in that, The negative rewards for conflict include negative rewards for oncoming conflicts and intersection conflicts.

5. The unmanned intelligent vehicle path planning method according to claim 1, characterized in that, The acquisition of the state information of the unmanned intelligent vehicle at each time step includes: A grid environment map is constructed based on the mobile environment of the unmanned intelligent vehicle; The location information of the unmanned intelligent vehicle in the grid environment map is determined, and the traffic congestion information of the area where the unmanned intelligent vehicle is located is determined based on the location information.

6. The unmanned intelligent vehicle path planning method according to claim 1, characterized in that, Determining the movement action based on the status information includes: The movement action is determined based on the state information using a greedy strategy.

7. The unmanned intelligent vehicle path planning method according to claim 1, characterized in that, Determining the movement action based on the status information includes: The movement action is determined based on the state information using the DQN model; The calculation of the Q-value based on the immediate reward using the Q-Learning algorithm includes: The Q value is calculated based on the immediate reward using the DQN model. The learning path planning strategy based on the Q-value includes: The loss value is determined based on the Q value, and the parameters of the DQN model are adjusted in reverse based on the loss value until the DQN model is trained; wherein the trained DQN model includes the target path planning strategy.

8. The unmanned intelligent vehicle path planning method according to claim 7, characterized in that, The method further includes: Samples are drawn from the experience replay pool to train the DQN model.

9. An unmanned intelligent vehicle, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the unmanned intelligent vehicle path planning method according to any one of claims 1 to 8.

Images