Robot path planning method and planning device

Abstract

The present invention provides a robot path planning method, device and storage medium. The robot path planning method includes the following steps: obtaining the first initial information of the robot at the current moment and the second initial information of the pedestrian at the current moment; predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment based on the first initial information and the second initial information; determining the action values ​​of multiple actions of the robot at the current moment based on the first initial information, the second initial information, the first state and the second state; and performing real-time local path planning based on the action value of each action. By executing these steps, the path planning method enables the robot to move in a more efficient and human social norm-compliant manner, has high environmental adaptability and obstacle avoidance success rate, and can achieve local obstacle avoidance planning in complex pedestrian environments.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation, and in particular to a robot path planning method, a robot path planning device and a corresponding computer-readable storage medium. Background Art

[0002] In recent years, with the development of mobile robots and artificial intelligence technology, mobile robots have begun to move from laboratory environments to the public domain to provide services for humans. However, the scenes in the public service field are more complex, especially the pedestrian environment, which poses new challenges to the motion planning algorithms of mobile robots.

[0003] Traditional local path planning methods for mobile robots use mathematical models or physical models to construct the interaction state between the robot and pedestrians, and then combine traditional search algorithms such as genetic algorithms to complete the path planning task. This type of method requires setting different parameters according to different experimental scenarios, has limited generalization capabilities for unfamiliar scenarios, and has poor results. Although it can ensure the stable operation of mobile robots in structured environments such as factories, it still faces many theoretical and engineering difficulties in complex pedestrian environments.

[0004] With the development of machine learning, data-driven methods have become a popular research direction for robot path planning in pedestrian environments. This method enables mobile robots to have "learning ability" and greatly improves scene adaptability, but it also faces problems such as low learning efficiency and difficulty in convergence.

[0005] In order to overcome the above-mentioned defects of the prior art, there is an urgent need in the art for a robot path planning technology that enables the robot to move in a more efficient and socially compliant manner, has high environmental adaptability and obstacle avoidance success rate, and can achieve local obstacle avoidance planning in complex pedestrian environments. Summary of the invention

[0006] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or decisive elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0007] In order to overcome the above-mentioned defects of the prior art, the present invention provides a robot path planning method, a robot path planning device and a corresponding computer-readable storage medium, which can enable the robot to move in a more efficient and socially compliant manner, have high environmental adaptability and obstacle avoidance success rate, and can realize local obstacle avoidance planning in complex pedestrian environments.

[0008] Specifically, the above-mentioned vehicle communication method provided according to the first aspect of the present invention includes the following steps: obtaining the first initial information of the robot at the current moment and the second initial information of the pedestrian at the current moment; predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment based on the first initial information and the second initial information; determining the action values ​​of multiple actions of the robot at the current moment based on the first initial information, the second initial information, the first state and the second state; and performing real-time local path planning based on the action value of each of the actions.

[0009] Furthermore, in some embodiments of the present invention, the step of obtaining the first initial information of the robot at the current moment and the second initial information of the pedestrian at the current moment includes: setting the environment state space, the robot action space and the environment reward function; obtaining the pedestrian position and pedestrian speed at the current moment; determining the posture information of the robot based on the odometer data of the robot; and determining the first initial information and the second initial information based on the environment state space, the robot action space, the environment reward function, the pedestrian position, the pedestrian speed, the posture information, fixed target point information, and maximum linear speed information.

[0010] Further, in some embodiments of the present invention, the step of setting the environmental state space includes: t represents the state of the robot at time t, and represents the observable state of the i-th pedestrian at time t; and in two-dimensional space, the robot and each pedestrian are assumed to be a circle with a radius of r, and the state information of the robot is represented as The current posture of the robot speed Own radius r, target position [g x , g y ], maximum linear velocity v pref The state information of each pedestrian is represented as The coordinate position of the pedestrian speed and radius r, the state information of n pedestrians at time t is expressed as And the environment joint state is expressed as

[0011] Furthermore, in some embodiments of the present invention, the step of setting the robot motion space includes: in the motion space A, setting the linear velocity to v = {0.2, 0.4, 0.6, 0.8, 1.0}, setting the angular velocity to ω = {-π / 4, -π / 6, -π / 12, 0, π / 12, π / 6, π / 4}, and expressing the motion space A as A = [{0, 0}, {v, ω}], wherein the motion space A includes multiple discrete actions.

[0012] Furthermore, in some embodiments of the present invention, the step of setting the environmental reward function includes: setting the reward function R t Defined as in, To get closer to the target reward, Penalty for hitting pedestrians, Punishment for violating social norms, Used to guide the robot to reach the target position quickly and eventually: where r g = 0.25 is the reward for reaching the target position, P t is the position of the robot at time t, g is the target position, To ensure the safety of sports: where r c =-0.25 is the penalty for the robot when it encounters a pedestrian, r robot is the radius of the robot, r i is the radius of the ith pedestrian, P t is the position of the robot at time t, is the position of the i-th pedestrian at time t, To ensure that the robot's motion meets social attribute requirements and prevent the robot from getting too close to pedestrians during motion, causing discomfort:

[0013]

[0014]

[0015] in is the distance between the robot and the i-th pedestrian.

[0016] Furthermore, in some embodiments of the present invention, the step of obtaining the pedestrian position and pedestrian speed at the current moment includes: detecting the legs of the pedestrian through multiple frames of lidar data; and matching the corresponding pedestrian according to the detected leg information, and tracking the pedestrian.

[0017] Further, in some embodiments of the present invention, the step of predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment according to the first initial information and the second initial information comprises: constructing a pedestrian state prediction model; using the robot state information s t and pedestrian status information W t As input, through two multi-layer perceptron models f r 、f h The robot status information s t and the pedestrian status information W t The dimension becomes consistent s′ t =f r (s t ; W r )W′ t =f h (W t ; W h )W r , W h is a trainable weight matrix; A graph attention network is constructed as a feature matrix to predict the first state of the robot at the next moment and the second state of the pedestrian at the next moment.

[0018] Furthermore, in some embodiments of the present invention, the step of constructing a graph attention neural network includes: calculating the attention matrix under each attention head in the first layer of the graph attention network

[0019]

[0020] in is a trainable weight matrix, k1 is the number of attention heads in the first-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, so as to calculate the output result of the first-layer graph attention network:

[0021]

[0022]

[0023]

[0024] Among them, || represents feature concatenation, σ represents function, set to ELU, W1 is a trainable weight matrix; and calculate the attention matrix of each attention head in the second layer of the graph attention network

[0025]

[0026] in is a trainable weight matrix, k2 is the number of attention heads in the second-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between the mobile robot and pedestrians, and between pedestrians, to calculate the output result of the second-layer graph attention network:

[0027]

[0028]

[0029]

[0030] Among them, || represents feature concatenation, σ represents a function, which is set to ELU, and W2 is a trainable weight matrix.

[0031] Further, in some embodiments of the present invention, the step of predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment comprises: Through a multi-layer perceptron model f predict Predict the next state of pedestrians Where W is a trainable weight matrix; and based on the mobile robot state information and action space A, calculate each action strategy a in the action space t =[v t ,ω t ] The corresponding next moment state information s t+1 :

[0032] θ t+1 =θ t+1 +ω t

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] Further, in some embodiments of the present invention, Figure 3As shown, the step of determining the action values ​​of the multiple actions of the robot at the current moment according to the first initial information, the second initial information 31, the first state and the second state includes: based on the D3QN reinforcement learning method, the first initial information, the second initial information, the first state and the second state are respectively input into the value network model to calculate the action values ​​Q(J) of the multiple actions of the robot in its action space at the current moment t , a t ;ω).

[0039] Furthermore, in some embodiments of the present invention, the calculation of the action value Q(J) of the robot in its action space at the current moment t , a t ; ω) The steps include: using the D3QN reinforcement learning framework to build a value network model; using the robot state information s t and pedestrian status information W t 31 as input, through two multi-layer perceptron models f′ r , f′ h 32 The robot status information s t and the pedestrian status information W t The dimensions become consistent:

[0040] s′ t =f′ r (s t ; W′ r )

[0041] W′ t =f′ h (W t ; W′ h )

[0042] Among them, W′ r , W′ h is a trainable weight matrix; is the feature matrix, and calculates the attention matrix of each attention head in the first layer of the graph attention network

[0043] in is a trainable weight matrix, k1 is the number of attention heads in the first-layer graph attention network 33, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, so as to calculate the output result of the first-layer graph attention network 33:

[0044]

[0045]

[0046]

[0047] Where || represents feature concatenation, σ represents a function, set to ELU, and W′1 is a trainable weight matrix; calculate the attention matrix of each attention head in the second-layer graph attention network 34

[0048]

[0049] in is a trainable weight matrix, k2 is the number of attention heads in the second-layer graph attention network 34, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, and calculate the output result of the second-layer graph attention network 34:

[0050]

[0051]

[0052]

[0053] Among them, || represents feature concatenation, σ represents a function, which is set to ELU, and W′2 is a trainable weight matrix; let Based on the principle of D3QN reinforcement learning algorithm, H is input into the subsequent value network model to calculate each action a t The action value Q(J t , a t ;ω):

[0054]

[0055] Where V and A represent two multi-layer perceptron models 35, and Noisy Net is introduced to add Gaussian noise to the fully connected layer. The two multi-layer perceptron models take H as input and output state value and advantage function respectively; the robot state information s at the next moment is t+1 and pedestrian status information W t+1 As input to the value network model, to calculate each action a in the next moment t+1 The action value Q(J t+1 , a t+1 ;ω):

[0056] as well as

[0057] Based on Q(J t , a t;ω) and Q(J t+1 , a t+1 ; ω), recalculate the current environment state J t Next, each action a in the action space t The value of Q(J t , a t ;ω):

[0058]

[0059] Where γ is the discount factor and Δt is the time interval between two decisions made by the robot.

[0060] Further, in some embodiments of the present invention, the step of performing real-time local path planning according to the action value of each action includes: according to the calculated value Q(J t , a t ; ω), select the current state J t The optimal strategy output is formulated by taking the action with the largest action value to realize the real-time local path planning of the robot.

[0061] Furthermore, in some embodiments of the present invention, the step of performing real-time local path planning according to the action value of each action further includes: obtaining the coordinate information of the robot at the current moment To calculate the distance d between the coordinate information and the target point g ; Determine the distance d g Is it less than a preset threshold; in response to the distance d g The judgment result is greater than or equal to the preset threshold, and the next state J is further determined. t+1 The action with the largest action value is used to formulate the optimal strategy output; and in response to the distance d g If the result is less than the preset threshold, planning is stopped.

[0062] In addition, the robot path planning device provided according to the second aspect of the present invention includes a memory and a processor. The processor is connected to the memory and is configured to implement the robot path planning method provided by the first aspect of the present invention.

[0063] In addition, the computer-readable storage medium provided according to the second aspect of the present invention stores computer instructions, which, when executed by a processor, implement the robot path planning method provided by the first aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The above features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or features may have the same or similar reference numerals.

[0065] Figure 1 A flow chart of a robot path planning method provided according to some embodiments of the present invention is shown.

[0066] Figure 2 A diagram of a pedestrian state prediction model in a robot path planning method provided according to some embodiments of the present invention is shown.

[0067] Figure 3 A value network model diagram in a robot path planning method provided according to some embodiments of the present invention is shown.

[0068] Figure 4 A schematic diagram of experimental results in a robot path planning method provided according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0069] The following specific embodiments illustrate the implementation of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will include many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description.

[0070] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0071] In addition, the terms "upper", "lower", "left", "right", "top", "bottom", "horizontal" and "vertical" used in the following description should be understood as the directions shown in the paragraph and the related drawings. Such relative terms are only used for the convenience of description and do not mean that the device described therein must be manufactured or operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0072] It is understood that although the terms "first", "second", "third", etc. may be used herein to describe various components, regions, layers and / or parts, these components, regions, layers and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers and / or parts. Therefore, the first component, region, layer and / or part discussed below may be referred to as a second component, region, layer and / or part without departing from some embodiments of the present invention.

[0073] As mentioned above, in recent years, with the development of robots and artificial intelligence technology, robots have begun to move from laboratory environments to the public domain to provide services for humans. However, the scenes in the public service field are more complex, especially the pedestrian environment, which poses new challenges to the motion planning algorithm of mobile robots. Traditional mobile robot local path planning methods use mathematical models or physical models to construct the interaction state between robots and pedestrians, and then combine traditional search algorithms such as genetic algorithms to complete the path planning task. This type of method requires setting different parameters according to different experimental scenarios, and has limited generalization ability for unfamiliar scenes and poor results. Although it can ensure the stable operation of mobile robots in structured environments such as factories, it still faces many theoretical and engineering difficulties in complex pedestrian environments. With the development of machine learning, data-driven methods have become a popular research direction for robot path planning in pedestrian environments. This method enables mobile robots to have "learning ability" and greatly improves scene adaptability, but it also faces problems such as low learning efficiency and convergence difficulties.

[0074] In order to overcome the above-mentioned defects of the prior art, the present invention provides a robot path planning method, a robot path planning device and a corresponding computer-readable storage medium, so that the robot can move in a more efficient and socially compliant manner, has high environmental adaptability and obstacle avoidance success rate, and can realize local obstacle avoidance planning in complex pedestrian environments.

[0075] In some non-limiting embodiments, the robot path planning method provided in the first aspect of the present invention can be implemented via the robot path planning device provided in the second aspect of the present invention. Specifically, the planning device is configured with a memory and a processor. The memory includes but is not limited to the computer-readable storage medium provided in the third aspect of the present invention, on which computer instructions are stored. The processor is connected to the memory and is configured to execute the computer instructions stored on the memory to implement the robot path planning method provided in the first aspect of the present invention.

[0076] The working principle of the above-mentioned path planning device will be described below in conjunction with some examples of path planning methods. Those skilled in the art will understand that these examples of communication methods are only some non-limiting implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit all functions or all working modes of the communication system. Similarly, the path planning device is also only a non-limiting implementation method provided by the present invention, and does not constitute a limitation on the execution subject of each step in these path planning methods.

[0077] Please refer to Figure 1 , Figure 1 A schematic flowchart of a robot path planning method provided according to some embodiments of the present invention is shown.

[0078] like Figure 1 As shown in step S1, in the process of planning the robot path, the planning device can first set the environment state space, the action space of the mobile robot, and the environment reward function.

[0079] Specifically, in some embodiments of the present invention, the environment state space can be defined as follows: t represents the state of the mobile robot at time t, represents the observable state of the i-th pedestrian at time t. In the two-dimensional space (XY plane), the robot and each pedestrian are assumed to be a circle with a radius of r. The state information of the mobile robot is expressed as The current position of the mobile robot speed Own radius r, target position [g x , g y ], maximum linear velocity v pref Each pedestrian status information can be expressed as is the coordinate position, speed and radius of the pedestrian. The state information of n pedestrians at time t is expressed as The joint state of the environment is expressed as

[0080] Optionally, in some other embodiments, the environment state space may also be defined as follows: t represents the state of the mobile robot at time t, represents the observable state of the i-th pedestrian at time t. In the two-dimensional space (XY plane), the robot and each pedestrian are assumed to be a rectangle with a length of a and a width of b. The state information of the mobile robot is expressed as The current position of the mobile robot speed Its own length a, its own width b, target position [g x , g y ], maximum linear velocity v pref The state information of each pedestrian can be expressed as is the coordinate position, speed and radius of the pedestrian. The state information of n pedestrians at time t is expressed as The joint state of the environment is expressed as

[0081] In addition, in some embodiments of the present invention, the action space of the robot can be defined as follows: Set the action space A of the mobile robot. The action space A is composed of the linear velocity v and angular velocity ω of the robot. Considering the dynamic limitations of the robot and the safety requirements in the human-machine coexistence environment, the linear velocity v is set to {0.2, 0.4, 0.6, 0.8, 1.0}, and the angular velocity ω is set to {-π / 4, -π / 6, -π / 12, 0, π / 12, π / 6, π / 4}. The action space A is expressed as A = [{0, 0}, {v, ω}], which consists of 36 discrete actions.

[0082] Optionally, in some other embodiments, the action space of the robot can also be defined as follows: the action space B of the mobile robot is set. The action space B is composed of the linear velocity v and angular velocity ω of the robot. Considering the dynamic limitations of the robot and the safety requirements in the human-machine coexistence environment, the linear velocity v is set to be {0.3, 0.6, 0.9, 1.2, 1.5}, and the angular velocity ω is set to be {-π / 6, -π / 12, 0, π / 12, π / 6}. The action space B is expressed as B = [{0, 0}, {v, ω}], which consists of 26 discrete actions.

[0083] In addition, in some embodiments of the present invention, the robot's environmental reward function can be defined as follows: t It consists of three parts: Penalty for collision with pedestrians and penalties for violating social norms

[0084]

[0085] Used to guide the robot to reach the target position quickly and eventually:

[0086]

[0087] where r g The reward for reaching the target position is that the present invention can automatically adjust r according to the different target environments and the different convergence speeds of the algorithm. g The size range can be set to achieve rapid convergence or accurate target value. For example, the present invention can set r g = 0.4 to achieve rapid convergence so that the robot can quickly reach the final target position. For example, the present invention can set r g = 0.1 to achieve the effect of accurate obstacle avoidance of the robot, P t is the position of the robot at time t, g is the target position, and for another example, the present invention can set r g =0.25 to enable the robot to avoid obstacles more accurately and reach the target area faster. t is the position of the robot at time t, and g is the target position;

[0088] To ensure the safety of movement, the robot will receive a certain degree of penalty when it collides with pedestrians:

[0089]

[0090] where r c is the penalty for the robot when it encounters a pedestrian. Similarly, the present invention can adjust r c The size range can achieve fast convergence or accurately reach the target value.

[0091] Optionally, the present invention may set r g =0.4 to achieve the effect of accurate obstacle avoidance of the robot.

[0092] Optionally, the present invention may set r g = 0.1 to achieve fast convergence so that the robot can reach the target position quickly.

[0093] Preferably, the present invention can set r c = 0.25 to enable the robot to avoid obstacles more accurately and reach the target area faster, where r robot is the radius of the robot, r i is the radius of the ith pedestrian, P t is the position of the robot at time t, is the position of the i-th pedestrian at time t.

[0094] Finally, in order to ensure the social attribute requirements of the robot's movement, the present invention can add a social distance penalty based on the social dilemma to prevent the robot from getting too close to pedestrians during movement and causing discomfort. For example, the present invention can set the penalty function to the following form:

[0095]

[0096]

[0097] in is the distance between the mobile robot and the ith pedestrian. The present invention can be set that when the mobile robot is within 0.5m of the pedestrian, it will cause the pedestrian's movement mode to change in the crowd, resulting in an impact on other pedestrians; when the mobile robot is within 0.2m of the pedestrian, it will directly cause discomfort to the pedestrian.

[0098] Optionally, the present invention may set the penalty function to be a quadratic exterior point penalty function or an interior point penalty function of an equality constraint.

[0099] Further, in some embodiments of the present invention, Figure 1 As shown in step S2, after setting the spatial environment state, the robot's action space, and the environment reward function, the present invention can obtain the current pedestrian state information and robot state information through a laser radar and an odometer.

[0100] Specifically, the present invention can detect the legs of pedestrians through multi-frame laser radar data through the leg detection algorithm, match the corresponding pedestrians according to the detected leg information, and then track the pedestrians using the pedestrian tracking algorithm to obtain the speed of each pedestrian. and coordinates Afterwards, the present invention can set the pedestrian radius r to 0.3m and obtain the status information of each pedestrian at time t Finally, the state information of n pedestrians at time t is expressed as

[0101] Furthermore, in some embodiments of the present invention, the present invention can obtain the position information of the mobile robot at the current moment based on the adaptive Monte Carlo algorithm. And get the current speed of the mobile robot Here, the present invention can set the radius r of the mobile robot itself to 0.3m and the maximum linear speed v pref Set to 1m / s, target position [g x , g y] Initially given, get the state information of the mobile robot at time t

[0102] Please refer to further Figure 1 and Figure 2 , Figure 2 FIG. 2 shows a pedestrian state prediction model diagram of the present invention. Figure 1 Step S3 is the same as Figure 2 As shown, the present invention can predict the first state of the robot at the next moment and the second state of the pedestrian at the next moment based on the first initial information and the second initial information.

[0103] Specifically, in the process of predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment, the present invention can first construct a pedestrian state prediction model. Figure 2 As shown in module 21, the present invention can be based on robot status information s t and pedestrian status information W t As input, the robot state information s is transformed into t and pedestrian status information W t The dimensions become consistent.

[0104] Preferably, the present invention can be implemented by two multi-layer perceptron models 22f r 、f h Send robot status information t and pedestrian status information W t The dimensions become consistent:

[0105] s′ t =f r (s t ; W r ) W′ t =f h (W t ; W h )

[0106] Among them, W r , W h is a trainable weight matrix;

[0107] Optionally, the present invention can transform the robot state information s through a customized multi-layer neural network t and pedestrian status information W t The dimensions become consistent.

[0108] Optionally, the present invention can transform the robot state information s via a support vector machine model. t and pedestrian status information W t The dimensions become consistent.

[0109] Preferably, the present invention can also be As the feature matrix, a graph attention network is constructed to predict the first state of the robot at the next moment and the second state of the pedestrian at the next moment.

[0110] Optionally, the present invention can also regularize the feature matrix as a new feature matrix and construct a graph attention network to predict the first state of the robot at the next moment and the second state of the pedestrian at the next moment.

[0111] Optionally, the present invention can also normalize the feature matrix as a new feature matrix and construct a graph attention network to predict the first state of the robot at the next moment and the second state of the pedestrian at the next moment.

[0112] Furthermore, in some embodiments of the present invention, the step of constructing a graph attention neural network includes: calculating the attention matrix under each attention head in the first layer of the graph attention network 23

[0113]

[0114] Optionally, the present invention can replace the activation function LeakyReLu with the activation function sigmoid, or replace the activation function LeakyReLu with the activation function tanh, where is a trainable weight matrix, k1 is the number of attention heads in the first-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between robots and pedestrians, and between pedestrians, to calculate the output result of the first-layer graph attention network 23:

[0115]

[0116]

[0117]

[0118] Among them, || represents feature concatenation, σ represents function, which is set to ELU, and W1 is a trainable weight matrix.

[0119] In addition, the present invention can calculate the attention matrix under each attention head in the second layer graph attention network 24

[0120]

[0121] Optionally, the present invention can replace the activation function LeakyReLu with the activation function sigmoid, or replace the activation function LeakyReLu with the activation function tanh, where is a trainable weight matrix, k2 is the number of attention heads in the second-layer graph attention network 24, and the multi-head attention mechanism is combined to extract the interaction feature information between the mobile robot and the pedestrian, and between pedestrians, so as to calculate the output result of the second-layer graph attention network:

[0122]

[0123]

[0124]

[0125] Among them, || represents feature concatenation, σ represents the function, which is set to ELU, and W2 is the trainable weight matrix.

[0126] Further, in some embodiments of the present invention, the step of predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment includes:

[0127] make

[0128] Preferably, the present invention can be implemented by a multi-layer perceptron model f predict 25 Predict the next state of pedestrians Where W is a trainable weight matrix;

[0129] Optionally, the present invention can be implemented by a custom multi-layer neural network model f predict1 Predict the next state of pedestrians Where W is a trainable weight matrix.

[0130] In addition, the present invention can also be based on the mobile robot state information and action space A, calculate each action strategy a in the action space t =[v t ,ω t ] The corresponding next moment state information s t+1 :

[0131] θ t+1 =θ t+1 +ω t

[0132]

[0133]

[0134]

[0135]

[0136]

[0137] Further, in some embodiments of the present invention, Figure 1 As shown in step S4, after obtaining the next moment state information, i.e., the first state information and the second state information, the action value function of each action in the action space is calculated based on D3QN to calculate the action value Q(J) of multiple actions of the robot in its action space at the current moment. t , a t ω) includes: using the D3QN reinforcement learning framework to build a value network model. Afterwards, the present invention can use the robot state information s t and pedestrian status information W t As input, through two multi-layer perceptron models f′ r , f′ h Send robot status information t and pedestrian status information W t The dimensions become consistent:

[0138] s′ t =f′ r (s t ; W′ r )

[0139] W′ t =f′ h (W t ; W′ h )

[0140] Among them, W′ r , W′ h is a trainable weight matrix;

[0141] Afterwards, the present invention can be is the feature matrix, and the attention matrix under each attention head in the first layer of the graph attention network 33 is calculated

[0142]

[0143] in is a trainable weight matrix, k1 is the number of attention heads in the first-layer graph attention network 33, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, so as to calculate the output result of the first-layer graph attention network 33:

[0144]

[0145]

[0146]

[0147] Among them, || represents feature concatenation, σ represents a function, which is set to ELU, and W′1 is a trainable weight matrix.

[0148] After that, the present invention can calculate the attention matrix of each attention head in the second layer graph attention network

[0149]

[0150] in is a trainable weight matrix, k2 is the number of attention heads in the second-layer graph attention network 34, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, and calculate the output result of the second-layer graph attention network 34:

[0151]

[0152]

[0153]

[0154] Among them, || represents feature concatenation, σ represents a function, which is set to ELU, and W′2 is a trainable weight matrix.

[0155] After that, the present invention can make Based on the principle of D3QN reinforcement learning algorithm, H is input into the subsequent value network model to calculate each action a t The action value Q(J t , a t ;ω):

[0156]

[0157] Wherein V and A represent two multi-layer perceptron models 35, and Noisy Net is introduced to add Gaussian noise to the fully connected layer. The two multi-layer perceptron models 35 take H as input and output state value and advantage function respectively.

[0158] Afterwards, the present invention can obtain the robot state information s at the next moment. t+1 and pedestrian status information W t+1 As input to the value network model, to calculate each action a in the next moment t+1 The action value Q(J t+1 , a t+1 ;ω):

[0159]

[0160] After that, the present invention can be based on Q(Jt , a t ;ω) and Q(J t+1 , a t+1 ; ω), recalculate the current environment state J t Next, each action a in the action space t The value of Q(J t , a t ;ω):

[0161]

[0162] Where γ is the discount factor and Δt is the time interval between two decisions made by the robot.

[0163] Optionally, the present invention can achieve a better obstacle avoidance effect by adjusting the value of the free adjustment discount factor according to the obstacle avoidance effect of the actual robot.

[0164] Further, in some embodiments of the present invention, Figure 1 As shown in step S5, the present invention can select the optimal action strategy according to the action value function and publish it to the robot for execution.

[0165] Specifically, the present invention can first obtain the coordinate information of the robot at the current moment To calculate the distance d between the coordinate information and the target point g , and then determine the distance d g Is it less than a preset threshold? Then, in response to the distance d g If the judgment result is greater than or equal to the preset threshold, the present invention can further determine the next state J t+1 The action with the largest action value is used to formulate the optimal strategy output. g If the judgment result is less than the preset threshold, the present invention can determine that the robot has reached the target point and stop planning.

[0166] Alternatively, if Figure 4 As shown, the present invention can also compare the robot's obstacle avoidance effect and adjust the preset threshold according to the schematic diagram of the experimental results in the robot path planning method until the best obstacle avoidance effect is achieved.

[0167] In summary, compared with the current robot navigation technology in this field, the present invention can realize the movement of the robot more effectively and in compliance with human social norms based on the robot path planning method, so it has high environmental adaptability and obstacle avoidance success rate, and can realize local obstacle avoidance planning in complex pedestrian environments.

[0168] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0169] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0170] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0171] Although the path planning device described in the above-mentioned embodiment can be implemented by a combination of software and hardware. However, it is understood that the path planning device can also be implemented in software or hardware alone. For hardware implementation, the path planning device can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for performing the above functions, or a selection combination of the above devices. For software implementation, the path planning device can be implemented by independent software modules such as procedures and function modules running on a general-purpose chip, wherein each module performs one or more functions and operations described herein.

[0172] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0173] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robot path planning method, characterized in that: The following steps are involved: Get the coordinate position and speed of the pedestrian at the current moment; Determine the current position of the robot according to the odometer data of the robot; Set the environment reward function; Setting an environment state space and a robot motion space, and determining the state information of the robot according to the current posture, first speed, radius, target position and maximum linear speed of the robot in a two-dimensional space, and determining the state information of the pedestrian according to the coordinate position, second speed and radius of the pedestrian; Determine, according to the state information of the robot, the state information of the pedestrian, and the environmental reward function, first initial information of the robot at a current moment and second initial information of the pedestrian at a current moment; Build a pedestrian status prediction model; Taking the first initial information of the robot and the second initial information of the pedestrian as input, the dimensions of the first initial information and the second initial information are made consistent through two multi-layer perceptron models; Constructing a graph attention network with a feature matrix consisting of first initial information and second initial information of consistent dimension to predict a first state of the robot at a next moment and a second state of the pedestrian at a next moment; Determining action values ​​of multiple actions of the robot at a current moment according to the first initial information, the second initial information, the first state, and the second state; as well as According to the action value of each of the actions, real-time local path planning is performed.

2. The robot path planning method according to claim 1, characterized in that: The current pose of the robot is represented as The first speed is expressed as The radius is denoted as r, and the target position is denoted as [g x ,g y ], the maximum linear velocity is denoted as v pref , the robot's state information is represented as The coordinate position of the pedestrian is expressed as The second speed is expressed as The state information of pedestrian i is expressed as The state information of n pedestrians at time t is expressed as 3. The robot path planning method according to claim 1, characterized in that: The steps of setting the robot motion space include: In the action space A, the linear velocity is set to v = {0.2, 0.4, 0.6, 0.8, 1.0}, the angular velocity is set to ω = {-π / 4, -π / 6, -π / 12, 0, π / 12, π / 6, π / 4}, and the action space A is expressed as A = [{0,0}, {v,ω}], wherein the action space A includes multiple discrete actions.

4. The robot path planning method according to claim 1, characterized in that: The steps of setting the environment reward function include: The reward function R t Defined as in, To get closer to the target reward, Penalty for hitting pedestrians, Punishment for violating social norms, Said Used to guide the robot to reach the target position quickly and eventually: where r g = 0.25 is the reward for reaching the target position, P t is the position of the robot at time t, g is the target position, Said To ensure the safety of sports: where r c =-0.25 is the penalty for the robot when it encounters a pedestrian, r robot is the radius of the robot, r i is the radius of the ith pedestrian, P t is the position of the robot at time t, is the position of the i-th pedestrian at time t, Said To ensure that the robot's motion meets social attribute requirements and prevent the robot from getting too close to pedestrians during motion, causing discomfort: in is the distance between the robot and the i-th pedestrian.

5. The robot path planning method according to claim 1, characterized in that: The step of obtaining the pedestrian position and pedestrian speed at the current moment includes: Detecting the legs of pedestrians through multiple frames of LiDAR data; and The corresponding pedestrian is matched according to the detected leg information and the pedestrian is tracked.

6. The robot path planning method according to claim 2, characterized in that: The first initial information and the second initial information with consistent dimension are expressed as: s′ t =f r (s t ;W r )W′ t =f h (W t ;W h ) Among them, W r , W h is the trainable weight matrix, The feature matrix is ​​represented as 7. The robot path planning method according to claim 6, characterized in that: The steps of constructing the graph attention neural network include: Calculate the attention matrix for each attention head in the first layer of the graph attention network in, is a trainable weight matrix, k1 is the number of attention heads in the first-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, so as to calculate the output result of the first-layer graph attention network: Among them, || represents feature concatenation, σ represents a function, set to ELU, and W1 is a trainable weight matrix; and Calculate the attention matrix for each attention head in the second layer graph attention network 1,2,3): in, is a trainable weight matrix, k2 is the number of attention heads in the second-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between the mobile robot and pedestrians, and between pedestrians, to calculate the output result of the second-layer graph attention network: Among them, || represents feature concatenation, σ represents the function, which is set to ELU, and W2 is the trainable weight matrix.

8. The robot path planning method according to claim 7, characterized in that: The step of predicting the first state of the robot at the next moment and the second state of the pedestrian at the next moment comprises: make Through a multi-layer perceptron model f predict Predict the next state of pedestrians Where W is a trainable weight matrix; and Based on the mobile robot status information and action space A, calculate each action strategy a in the action space t =[v t ,ω t ] The corresponding next moment state information s t+1 : i t+1 =θ t+1 +oh t 9. The robot path planning method according to claim 1, characterized in that: The step of determining the action values ​​of the multiple actions of the robot at the current moment according to the first initial information, the second initial information, the first state and the second state comprises: Based on the D3QN reinforcement learning method, the first initial information, the second initial information, the first state and the second state are respectively input into the value network model to calculate the action value Q(J) of multiple actions of the robot in its action space at the current moment. t ,a t ;ω).

10. The robot path planning method according to claim 9, characterized in that: The calculation of the action value Q(J) of the robot in its action space at the current moment t ,a t ;ω) the steps include: Adopt D3QN reinforcement learning framework to build value network model; With robot status information t and pedestrian status information W t As input, through two multi-layer perceptron models f′ r , f′ h The robot status information s t and the pedestrian status information W t The dimensions become consistent: s′ t =f′ r (s t ;W′ r ) W′ t =f′ h (W t ;W′ h ) Among them, W′ r , W′ h is a trainable weight matrix; by is the feature matrix, and calculates the attention matrix of each attention head in the first layer of the graph attention network in is a trainable weight matrix, k1 is the number of attention heads in the first-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between the robot and the pedestrian, and between pedestrians, so as to calculate the output result of the first-layer graph attention network: Among them, || represents feature concatenation, σ represents function, set to ELU, and W′1 is a trainable weight matrix; calculate the attention matrix of each attention head in the second-layer graph attention network in, is a trainable weight matrix, k2 is the number of attention heads in the second-layer graph attention network, and the multi-head attention mechanism is combined to extract the interaction feature information between robots and pedestrians, and between pedestrians, and calculate the output result of the second-layer graph attention network: Among them, || represents feature concatenation, σ represents a function, which is set to ELU, and W′2 is a trainable weight matrix; make Based on the principle of D3QN reinforcement learning algorithm, H is input into the subsequent value network model to calculate each action a t The action value Q(J t ,a t ;ω): Where V and A represent two multi-layer perceptron models, and Noisy Net is introduced to add Gaussian noise to the fully connected layer. The two multi-layer perceptron models take H as input and output state value and advantage function respectively; The robot state information s at the next moment t+1 and pedestrian status information W t+1 As input to the value network model, to calculate each action a in the next moment t+1 The action value Q(J t+1 ,a t+1 ;ω): as well as Based on Q(J t ,a t ;ω) and Q(J t+1 ,a t+1 ; ω), recalculate the current environment state J t Next, each action a in the action space t The value of Q(J t ,a t ;ω): Where γ is the discount factor and Δt is the time interval between two decisions made by the robot.

11. The robot path planning method according to claim 1, characterized in that: The step of performing real-time local path planning according to the action value of each action comprises: According to the calculated value of each action Q(J t ,a t ; ω), select the current state J t The optimal strategy output is formulated by taking the action with the largest action value to realize the real-time local path planning of the robot.

12. The robot path planning method according to claim 11, characterized in that: The step of performing real-time local path planning according to the action value of each action further includes: Get the coordinate information of the robot at the current moment To calculate the distance d between the coordinate information and the target point g ; Determine the distance d g Is it less than the preset threshold? In response to the distance d g The judgment result is greater than or equal to the preset threshold, and the next state J is further determined. t+1 The action with the highest action value is used to formulate the optimal strategy output; and In response to the distance d g If the result is less than the preset threshold, planning is stopped.

13. A robot path planning device, characterized in that: include: Memory; as well as A processor is connected to the memory and is configured to implement the robot path planning method according to any one of claims 1 to 12.

14. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the robot path planning method according to any one of claims 1 to 12 is implemented.

Images