Robot navigation method and device, storage medium and computer program product

Abstract

The invention provides a robot navigation method and device, a storage medium and a computer program product. The method comprises the following steps: acquiring a first motion state space parameter of a robot; inputting the first motion state space parameter into a pre-trained local path planning model to obtain first space feature information, and determining first time sequence feature information based on the first space feature information; based on the first time sequence feature information, behavior action information of the robot is determined, and the behavior action information is used for planning a navigation path of the robot.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a robot navigation method, device, storage medium and computer program product. Background Art

[0002] Robot navigation refers to the process of enabling a robot to autonomously plan and execute a safe, collision-free moving path based on a given target location. The following problems still exist in the robot navigation methods in related technologies: the robot consumes a lot of computing resources and storage space, and the environmental feature information extracted by the model in a dynamically changing environment is insufficient, which will cause the robot to be unable to identify new obstacles or route changes, and the planned robot driving path is less safe. Summary of the invention

[0003] In view of this, the present application hopes to provide a robot navigation method, device, storage medium and computer program product, which can improve the safety of the planned robot driving path while reducing computing resources and storage space.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, the present application provides a robot navigation method, which is applied to an edge device, and the method includes:

[0006] Obtaining the first motion state space parameters of the robot;

[0007] Inputting the first motion state space parameter into a pre-trained local path planning model to obtain first spatial feature information, and determining first temporal feature information based on the first spatial feature information;

[0008] Based on the first time series feature information, the behavior action information of the robot is determined, and the behavior action information is used to plan a navigation path of the robot.

[0009] In a second aspect, the present application provides an edge device, the edge device comprising:

[0010] An acquisition unit, used for acquiring a first motion state space parameter of the robot;

[0011] A determination unit, configured to input the first motion state space parameter into a pre-trained local path planning model to obtain first spatial feature information, and determine first temporal feature information based on the first spatial feature information;

[0012] The determination unit is further used to determine the behavior action information of the robot based on the first time series feature information, and the behavior action information is used to plan the navigation path of the robot.

[0013] In a third aspect, the present application provides an edge device, comprising: a processor and a memory; the processor implements the above-mentioned robot navigation method when executing a running program stored in the memory.

[0014] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, which implements the above-mentioned robot navigation method when executed by a processor.

[0015] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the above-mentioned robot navigation method when executed by a processor.

[0016] The present application provides a robot navigation method, device, storage medium and computer program product, the method comprising: obtaining a first motion state spatial parameter of the robot; inputting the first motion state spatial parameter into a pre-trained local path planning model to obtain first spatial feature information, and determining first temporal feature information based on the first spatial feature information; determining the robot's behavior action information based on the first temporal feature information, and the behavior action information is used to plan the robot's navigation path. By adopting the above implementation scheme, by executing the confirmation process of the robot's behavior action information on the edge device side, the consumption of computing and storage resources on the robot side can be reduced, and the pre-trained local path planning model is adopted on the edge device side, and the spatial feature information and temporal feature information of the robot's motion process are simultaneously obtained through the robot's first motion state parameter, so that the extracted scene information is richer, so that the robot's behavior action information output by the local path planning model is more in line with the actual scene, so the navigation path planned by the robot through the pre-trained local path planning model is more secure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a robot navigation method flow chart provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the structure of a strategy network provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a spatial feature extraction network structure provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of a time series feature extraction network structure provided in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of the overall process of a robot navigation method provided in an embodiment of the present application;

[0022] Figure 6A schematic diagram of a robot navigation system architecture provided in an embodiment of the present application;

[0023] Figure 7 A schematic diagram of the structure of an edge device provided in an embodiment of the present application Figure 1 ;

[0024] Figure 8 A schematic diagram of the structure of an edge device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0025] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the technical solution of the present application is further elaborated in detail below in combination with the drawings and specific embodiments of the specification. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0026] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0027] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first / second / third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0028] Robot navigation refers to the process of enabling a robot to autonomously plan and execute a safe, collision-free movement path based on a given target location. Robot navigation has a wide range of applications in many fields, such as industrial manufacturing, warehousing and logistics, rescue detection, etc.

[0029] Robot navigation can be divided into two basic research directions:

[0030] 1. Global path planning method: It is usually necessary to build an environmental map in advance, and then use a search algorithm to find the optimal or approximately optimal collision-free path. However, it requires a large amount of calculation and storage space, and has high requirements on the accuracy and completeness of environmental information. It is not adaptable in dynamically changing or unknown environments.

[0031] 2. Local path planning method: The local path planning method in related technologies usually plans the robot's movement based on sensor information. This method mostly only considers the current state and ignores the interaction between objects, which may cause oscillation or unnatural phenomena in the movement trajectory. The local path planning method based on deep reinforcement learning allows the robot to continuously interact with the environment and evaluates its behavior based on the reward function. Although it can learn the optimal navigation strategy and adapt to complex and dynamic environmental changes, it is difficult to design a reasonable representation network to model the surrounding environmental information.

[0032] Based on this, in the related technologies, the global path planning method and local path planning method used by most robot navigation still have the following problems:

[0033] 1. The computing resources and storage space consumption are large. The robot navigation process involves multiple computing modules, such as mapping module, planning module, obstacle avoidance module, control module, etc. These modules require a lot of computing resources and data processing capabilities. Calculation on the robot side is limited by the performance and capacity of the local hardware, and complex and advanced computing tasks cannot be achieved. For example, local hardware cannot support the large amount of computing requirements of deep reinforcement learning, cannot store large-scale map data, or cannot replace the correct global map in time. If the cloud computing model is adopted, the process of data transmission between the robot side and the cloud will reduce the real-time performance of the robot navigation.

[0034] 2. Actual scenarios are complex and changeable. The global path planning method in related technologies needs to rely on fixed map information. In a dynamically changing environment, it may cause the robot to be unable to identify new obstacles or route changes, reduce the accuracy and robustness of navigation, and be difficult to adapt to complex and changing scenarios.

[0035] 3. The extraction of environmental information is not sufficient. Local path planning algorithms represented by Reciprocal Velocity Obstacle (RVO) are short-term in space and time, which will cause oscillation or unnatural phenomena in the motion trajectory. Some local path planning methods based on deep reinforcement learning use a combination of multiple sensors to extract environmental information of the current state, and use a representation network for modeling to generate a natural and smooth motion trajectory. Reinforcement learning methods based on Markov decision-making also make decisions based only on the current state. None of the above schemes consider the long-term dependencies between states in decision-making. Therefore, some methods use recurrent neural networks (RNN) or long short-term memory networks (LSTM) to capture the long-term dependencies between states, but these methods often ignore the interactions between moving objects.

[0036] In summary, the local path planning methods in related technologies do not comprehensively and fully consider the interaction relationship between moving objects and the long-term dependency between states, resulting in insufficient extracted scene features, unreasonable environment modeling or unrealistic planned motion trajectories, making it difficult to train the optimal strategy.

[0037] Based on this, in the embodiments of the present application, a method for robot navigation through deep reinforcement learning in an edge computing scenario is proposed to address the above-mentioned technical problems. Because edge computing usually has higher computing power and storage space, robot navigation tasks in edge computing scenarios have the advantages of low latency, reliability, and flexibility. In the embodiments of the present application, by utilizing the advantages of low latency, high efficiency, and high flexibility of edge computing on the edge device side, combining the global path planning method and the local path planning method, the global map is stored on the edge side and updated regularly, and path planning is performed on the edge device side, which can reduce the storage and resource consumption of the robot and improve the reliability and adaptability of navigation. And by extracting richer scene features through a representation network that can simultaneously capture spatiotemporal relationships, the robot can not only dynamically learn the interactive relationship with the surrounding environment, but also have human-like memory capabilities, which improves the robot's adaptability to dynamic and complex environments, and ultimately enables the robot to reach the target navigation point safely and efficiently.

[0038] The specific implementation scheme of the embodiment of this application is as follows:

[0039] The present application embodiment provides a robot navigation method, such as Figure 1 As shown, applied to an edge device, the method may include:

[0040] S101. Obtain the first motion state space parameters of the robot.

[0041] In the embodiment of the present application, obtaining the first motion state space parameters of the robot can be achieved in the following manner:

[0042] The first motion state information of the robot, the local target point information in the driving path and the second motion state information of the obstacle are obtained; and the first motion state space parameter is determined based on the first motion state information, the local target point information and the second motion state information.

[0043] In an embodiment of the present application, the first motion state information may be environmental state information collected by the robot end, and the environmental state information collected by the robot end may include: position information, speed information, direction information, and the like.

[0044] In an embodiment of the present application, obtaining the first motion state information of the robot can be that the edge device interacts with the robot end, the robot end collects the first motion state information, and transmits the collected first motion state information to the edge device (also referred to as the edge end), that is, the edge device obtains the first motion state information from the robot end.

[0045] In the embodiment of the present application, the local target point information in the driving path can be obtained by the following methods:

[0046] Obtain target navigation position information of the robot; based on the first motion state information, obtain global map information corresponding to the first motion state information from the cloud device; input the global map information and the target navigation position information into a pre-trained global path planning model to obtain the global path information of the robot; based on the global path information, determine the local target point information.

[0047] In the embodiment of the present application, the target navigation position information is the global target navigation position of the robot, which may specifically include the starting position information and the end position information of the robot's travel.

[0048] In an embodiment of the present application, the pre-trained global path planning model may be a pre-trained Hybrid A* algorithm model.

[0049] In an embodiment of the present application, the target navigation position information can be obtained through manual input.

[0050] In an embodiment of the present application, the cloud device stores global map information corresponding to different environmental information, and the edge device selects the global map information in the current environment corresponding to the first motion state information from the multiple global map information stored in the cloud device based on the first motion state information collected by the robot.

[0051] In an embodiment of the present application, an upper-level global path planning model is used to plan a global path for the robot, that is, a pre-trained Hybrid A* algorithm model is used to input the acquired global map information and the acquired starting position information and the end position information into the pre-trained Hybrid A* algorithm model, and a global path information is planned. It can be understood that the global path information output by the pre-trained Hybrid A* algorithm model can be a collision-free safe path for the robot from the starting position information to the end position information on the global map information.

[0052] It should be noted that the Hybrid A* algorithm is used as the upper-level global path planning model in the embodiment of the present application. Compared with the commonly used A* algorithm, the Hybrid A* algorithm can consider the kinematic constraints of the robot in a continuous space and generate smoother and more feasible global path information.

[0053] In the embodiment of the present application, after obtaining the global path information, the local target point information is determined based on the global path information, which can be implemented in the following manner:

[0054] The target number of target point information is determined from the global path information at intervals of a preset distance value; and the target number of target point information is determined as local target point information.

[0055] In the embodiment of the present application, the preset distance value can be selected according to actual conditions, such as every 20 meters.

[0056] In an embodiment of the present application, the number of targets can be obtained based on the global path information and the preset distance value. For example, if the global path length is 100 meters and the interval is 20 meters, then the number of target point information can be determined to be 5 (including the starting point location information and the end point location information).

[0057] In the embodiment of the present application, coordinate points (ie, target point information) are uniformly collected on the global path information, and the collected multiple coordinate points are used as local target point information.

[0058] For example, if the global path information is a path from point A to point B, coordinate points can be selected at intervals of preset distance values ​​between points A and B. For example, point C, point D, point E, etc. can be selected as coordinate points between points A and B to obtain local target point information.

[0059] In the embodiment of the present application, the obtained local target point information is sequentially assigned to the lower-level local path planning model (i.e., the pre-trained local path planning model) for further processing. Among them, each local target point information can be used as temporary terminal position information during the robot's driving process.

[0060] In the embodiment of the present application, the first motion state space parameter is determined based on the first motion state information, the local target point information and the second motion state information, that is, the first motion state space parameter of the robot defined includes the robot's position information, speed information, direction information, local target point information and surrounding environment information. Specifically, the first motion state space parameter can be expressed in the form of the following formula (1):

[0061]

[0062] Wherein, S represents the state space parameter of the robot (ie, the first motion state space parameter).

[0063] h0 is the motion information of the robot, where v x and v y Indicates the robot's movement speed, v pref represents the preferred speed of the robot, and represents the relative position of the robot and the assigned local target point information, θ represents the angle between the robot's motion speed and the positive x-axis, and d g Represents the distance between the robot and the assigned local target point information, and r represents the radius of the robot.

[0064] H is a set of motion information (i.e., second motion state information) of all dynamic obstacles within a certain range around the robot obtained by the multi-sensor combination system.

[0065] h i is the motion information of a single dynamic obstacle. x and v y Indicates the movement speed of the dynamic obstacle. and It represents the relative position of the dynamic obstacle and the local target point information assigned by the robot, and r represents the radius of the dynamic obstacle.

[0066] d is the robot's m laser ranging results, indicating the distance between the robot and the obstacle in m directions, and is used to help the robot avoid static and dynamic obstacles.

[0067] S102: Input the first motion state space parameter into a pre-trained local path planning model to obtain first spatial feature information, and determine first temporal feature information based on the first spatial feature information.

[0068] In the embodiment of the present application, the structure of the pre-trained local path planning model is as follows: Figure 2 As shown, it includes a state input layer, a spatial feature extraction network, a temporal feature extraction network and an action output layer.

[0069] In the embodiment of the present application, the pre-trained local path planning model can use the proximal policy optimization (PPO) algorithm to train the local path planning model of the robot until the model converges. The PPO algorithm usually includes a policy network and a value network, wherein the policy network is responsible for generating the robot's action instructions according to the current state, and its structure can be referred to Figure 2 The final result obtained by the policy network training is the pre-trained local path planning model used in the embodiment of the present application. Except for the output layer, the other modules of the value network are the same as those of the policy network, and its output layer is used to estimate the value function V(s) of the state.

[0070] In an embodiment of the present application, the first motion state space parameter includes a first parameter, a second parameter and a third parameter, the pre-trained local path planning model includes a spatial feature extraction network, and the first motion state space parameter is input into the pre-trained local path planning model to obtain the first spatial feature information, which can be specifically implemented in the following manner:

[0071] The first parameter and the second parameter are processed through a spatial feature extraction network to obtain first motion feature information of the robot and second motion feature information of obstacles in the robot's driving path; third motion feature information is determined based on the first motion feature information and the second motion feature information; the third parameter is processed through the spatial feature extraction network to obtain fourth motion feature information between the robot and the obstacle; the third motion feature information and the fourth motion feature information are determined as the first spatial feature information.

[0072] In the embodiment of the present application, the first parameter may be h0 in the first motion state space parameter, the second parameter may be H in the first motion state space parameter, and the third parameter may be d in the first motion state space parameter.

[0073] In the embodiment of the present application, the specific structure of the spatial feature extraction network is as follows: Figure 3 As shown in the figure, it mainly includes a multi-layer perceptron, a multi-layer attention network and an attention layer. The spatial feature extraction network mainly uses a multi-layer graph attention network (GAT) and an additional attention network (i.e., attention layer) to dynamically learn the interaction between the robot and dynamic obstacles and between dynamic obstacles. Compared with the graph convolution operation, the multi-layer graph attention network can adaptively assign different attention weights according to the relative position and motion state between different moving objects to extract the high-level spatial state expression of dynamic obstacles in the environment.

[0074] In an embodiment of the present application, the input first parameter and second parameter are processed by a multi-layer perceptron and a multi-layer graph attention network in a spatial feature extraction network to obtain first motion feature information of the robot and second motion feature information of obstacles in the robot's driving path.

[0075] In the present application embodiment, using Figure 3 The process of extracting the first motion feature information and the second motion feature information by the spatial feature extraction network shown is as follows:

[0076] First, the robot's motion information h0 and the motion information of all dynamic obstacles H = [h1, h2, ..., h n ] are sequentially input into the multi-layer perceptron to extract the fixed-length latent features X = [x0, x1, ..., x i ,…,x n ], where x0 is the potential feature of the robot motion information, [x1,…,x i ,…,x n ] is the potential feature of the dynamic obstacle motion information. Then, the extracted potential feature X=[x0,x1,…,x i ,…,x n ] is input into the multi-layer graph attention network to obtain the corresponding hidden features W = [w0,w1,…,w i ,…,w n ].

[0077] In the embodiment of the present application, the first motion feature information is w0 in W, and the second motion feature information is the remaining features w1, ..., w i ,…,w n .

[0078] In an embodiment of the present application, a residual structure can also be added to the multi-layer graph attention network, so as to prevent network degradation and accelerate the training process.

[0079] In the embodiment of the present application, when determining the third motion feature information based on the first motion feature information and the second motion feature information, the hidden feature w0 of the robot may be selected, and the hidden feature w1, ..., w2 may be selected from the second motion feature information w1, ..., w3. i ,…,w n Select the hidden features of the k dynamic obstacles closest to the robot [w t ,w t+1 ,…], the selected w0 and [w t ,w t+1 ,…] is input into the attention layer module for weighted summation to obtain the new hidden state f1, that is, the third motion feature information.

[0080] It should be noted that if the number of dynamic obstacles around the robot is less than k, the robot's hidden feature w0 is used to replace the missing hidden features of the dynamic obstacles. For example, if the number of k is 5, but the hidden features of the dynamic obstacles are 3, two w0 are used to replace the missing hidden features of the dynamic obstacles.

[0081] In an embodiment of the present application, while executing the above-mentioned generation of the third motion feature information, the robot's laser ranging result d (i.e., the third parameter) is input into the multi-layer perceptron of the spatial feature extraction network to obtain the hidden feature f2 (i.e., the fourth motion feature information) between the robot and the obstacle.

[0082] In the embodiment of the present application, f1 and f2 are determined as the first spatial feature information extracted by the spatial feature extraction network, and the final output result of the spatial feature extraction network is {f1, f2}.

[0083] In an embodiment of the present application, after {f1, f2} is obtained through the first spatial feature extraction network, {f1, f2} is further processed through the temporal feature extraction network to obtain the first temporal feature information.

[0084] In the embodiment of the present application, the first spatial feature information includes the third motion feature information (i.e., f1) and the fourth motion feature information (i.e., f2), the pre-trained local path planning model also includes a temporal feature extraction network, and the first temporal feature information is determined based on the first spatial feature information, which can be implemented in the following manner:

[0085] The third motion feature information is processed through a temporal feature extraction network to obtain second temporal feature information; the fourth motion feature information is processed through a temporal feature extraction network to obtain third temporal feature information; and the first temporal feature information is determined based on the second temporal feature information and the third temporal feature information.

[0086] In the embodiment of the present application, the structure of the temporal feature extraction network is shown in Figure 4, which mainly includes two parallel multi-layer gated recurrent unit (GRU) networks. The temporal feature extraction network inputs the first spatial feature information into the multi-layer parallel gated recurrent unit (GRU) network. The GRU network can selectively forget the state information of the past moments according to the input and hidden state at the current moment (such as the feature information output by the residual structure), and achieve more reasonable local path planning through this human-like memory ability.

[0087] It should be noted that a residual structure can also be added to each GRU network to prevent network degradation and speed up the training process.

[0088] In the present application example, Figure 4 The specific process of determining the first temporal feature information by the temporal feature extraction network shown is to take f1 and f2 as input, respectively input them into the parallel GRU network in the temporal feature extraction network, obtain the second temporal feature information g1 and the third temporal feature information g2, associate g1 and g2 (which can be expressed as concat in English), and obtain the final first temporal feature information g.

[0089] S103: Determine the behavior information of the robot based on the first time series feature information, where the behavior information is used to plan a navigation path of the robot.

[0090] In the embodiment of the present application, the first time series characteristic information g obtained above is input into Figure 2 In the action output layer shown, the robot's behavioral action information (which can be represented by a) is output through the action output layer.

[0091] It should be noted that the behavior action information can be the robot's behavior actions such as turning left, turning right or moving forward, or other actions. The specific ones can be selected according to the actual situation, and the embodiments of the present application do not make specific limitations.

[0092] In an embodiment of the present application, after determining the behavior action information of the robot, the edge device sends the behavior action information to the robot, so that the robot plans a navigation path based on the received behavior action information.

[0093] In an embodiment of the present application, the edge device determines the action behavior information, and ultimately the robot side determines how to travel the path based on the action behavior information.

[0094] It can be understood that a robot navigation method provided in an embodiment of the present application can reduce the consumption of computing and storage resources on the robot side by executing the confirmation process of the robot's behavior and action information on the edge device side, and adopts a pre-trained local path planning model on the edge device side. Through the robot's first motion state parameters, the spatial feature information and the timing feature information of the robot's motion process are simultaneously obtained, so that the extracted scene information is richer, so that the robot's behavior and action information output by the local path planning model is more in line with the actual scene. Therefore, the navigation path planned by the robot through the pre-trained local path planning model is safer.

[0095] In the embodiment of the present application, before inputting the first motion state space parameter into the pre-trained local path planning model, the pre-trained local path planning model needs to be trained in advance, and the pre-trained local path planning model is deployed on the edge device, so as to enable the edge device to determine the robot's behavior action information. The specific steps of building and training the pre-trained local path planning model can be shown as follows:

[0096] In the embodiment of the present application, the PPO algorithm is used to train the local path planning model of the robot, specifically including:

[0097] (1) First, define the motion state space parameters of the robot: The motion state space parameters of the robot are defined as shown in the above formula (1). The process of defining the first motion state space parameters can be referred to above and will not be repeated here.

[0098] (2) Design the spatial feature extraction network module: The structure of the spatial feature extraction network module can be referenced Figure 3 First, the robot’s motion information h0 and the motion information H of all dynamic obstacles are sequentially input into the multi-layer perceptron to extract the fixed-length latent features X = [x0, x1, …, x i ,…,x n ], where x0 is the potential feature of the robot motion information, [x1,…,x i ,…,x n ] is the potential feature of the dynamic obstacle motion information. Then, the potential feature is input into the multi-layer graph attention network to obtain the hidden feature W = [w0, w1, ..., w i ,…,w n ]. The embodiment of the present application adds a residual structure to the multi-layer graph attention network, which can prevent network degradation and accelerate the training process. Finally, the hidden feature w0 and the hidden features of the k dynamic obstacles closest to the robot [w t ,w t+1 ,…], and input it into the attention layer module for weighted summation to obtain the new hidden state f1. If the number of surrounding dynamic obstacles is less than k, the robot's hidden feature w0 is used to replace the hidden features of the missing dynamic obstacles. At the same time, the laser ranging result d is input into the multi-layer perceptron to obtain the hidden feature f2.

[0099] (3) Design the timing feature extraction network module: refer to Figure 4The structure of the temporal feature extraction network module proposed in the embodiment of the present application is composed of two parallel multi-layer GRU neural networks, each of which is added with a residual structure to prevent network degradation and accelerate the training process. f1 and f2 are input as inputs to the parallel GRU neural networks to obtain temporal features g1 and g2, and then g1 and g2 are concat-operated to obtain the final temporal feature g.

[0100] (4) Design the reward function, which can be expressed as r t The specific expression of the reward function is shown in the following formula (2):

[0101]

[0102] When the distance between the robot and the assigned target point is less than a certain threshold, the robot is considered to have reached the target point and a positive reward r is given to the robot. goal .

[0103] When the robot collides with a static obstacle (i.e. if robot collides with a staticobstacle in the above formula), the robot is given a negative reward r s_collision .

[0104] When the robot collides with a dynamic obstacle (i.e. if robot collides with a dynamic obstacle in the above formula), the robot is given a negative reward r d_collision .

[0105] During the training process (i.e. in the above formula), the robot will continue to receive rewards r approach , as shown in the following formula (3):

[0106]

[0107] When the robot is at the first moment in the training round, a in formula (3) is a positive hyperparameter. It represents the difference between the robot's current and previous distances to the target point. When the robot approaches the target point, it will receive a positive reward.

[0108] (5) Build and train the strategy network and value network in the PPO algorithm model.

[0109] In an embodiment of the present application, the above-mentioned reward function is used to evaluate the value of the behavior information output by the local path planning model. If the robot encounters a static obstacle during driving, a corresponding negative reward will be given. In this way, the value evaluation result, such as a negative reward, is obtained by evaluating the execution result of the behavior according to the reward function. When it is a negative reward, the parameters of the local path planning model are adjusted. After continuous training, the robot can eventually achieve the goal of not colliding and safely reaching the terminal position during driving, that is, the trained local path planning model converges, and the pre-trained local path planning model finally deployed on the edge device is obtained.

[0110] In the embodiment of the present application, the quality of the execution action output by the local path planning model (which may also be referred to as the policy network in the embodiment of the present application) is evaluated, and the value network is usually used for evaluation. The structure of the value network is the same as that of the policy network except for the output layer. The input layer and hidden layer of the value network are the same as those of the policy network. The input can be the first motion state space parameter S of the robot, and the hidden layer includes a spatial feature extraction network and a temporal feature extraction network. The policy network outputs the action that the robot needs to perform, and the value network outputs the value function V(s) used to evaluate the robot state.

[0111] It should be noted that the value function is used to evaluate the value of the current state, and the reward function is the reward obtained in the current state. It can be understood as the accumulation of rewards for the current state evaluated by the value function, but it is not a simple accumulation of rewards, but the accumulation of the current state and possible rewards in the future. The value function and reward function can be used to update the policy network (used to output behavior action information), and both are required by the PPO algorithm.

[0112] It should be noted that the PPO algorithm is used in the embodiment of the present application to train the strategy network and the value network, and the end condition of each round is that the robot reaches the target point or fails to reach the target point within the specified time. The trained strategy network is the pre-trained local path planning model of the lower layer.

[0113] It should be noted that the pre-trained local path planning model can be trained on other devices, and the edge device only deploys the pre-trained local path planning model.

[0114] In an embodiment of the present application, the edge device deploys the trained local path planning model to the edge device end.

[0115] In the embodiment of the present application, during the actual application, it is only necessary to obtain the current motion state space parameters of the robot, input the current motion state space parameters into the pre-trained local path planning model (i.e., the trained local path planning model), and output the action behavior information that the robot currently needs to perform through the pre-trained local path planning model. The edge device sends the current action behavior information to be performed to the robot end, and the robot performs path planning according to the current action behavior information to be performed, such as determining path planning such as continuing to drive forward.

[0116] Based on the above embodiments, since there are often high-density dynamic obstacles (pedestrians, other robots, etc.) in the robot's navigation environment, how to achieve safe and collision-free path planning in a high-density dynamic obstacle environment is a key capability of the robot. In the embodiment of the present application, the global path planning algorithm is used as the upper-level guide, and a global path that reasonably avoids static obstacles and smoothly reaches the global target point is planned in combination with the global map, providing the robot with a general direction of movement. At the same time, the embodiment of the present application decomposes the planned global path into several local target points, and sequentially distributes them to the local path planning algorithm of the lower layer. In view of the problem that the local path planning algorithm in the relevant technology does not consider the scene information sufficiently, the local path planning algorithm of the lower layer in the embodiment of the present application is implemented based on deep reinforcement learning, and a local path planning model combining spatial and temporal features is proposed, which mainly includes a spatial feature extraction network and a temporal feature extraction network, and adjusts the movement direction of the robot in real time, etc., to avoid collisions with static obstacles and dynamic obstacles, thereby ensuring the rationality and safety of robot navigation.

[0117] Based on the above embodiment, when the robot is navigated by the embodiment of the present application, the overall process is as follows: Figure 5 As shown, the following steps may be included:

[0118] S1. Input the robot’s global target navigation position information.

[0119] S2. The robot collects environmental status information and transmits it to the edge device.

[0120] Among them, the environmental status information may include the status information of the robot collected by the robot end, including position information, speed information, direction information, etc.

[0121] S3. Perform global path planning at the upper layer and generate local target point information, and transmit the local target point information to the pre-trained local path planning model at the lower layer.

[0122] For details, please refer to the implementation process of the aforementioned embodiment, which will not be described again here.

[0123] S4. Perform local path planning at the lower level. Specifically, construct the first motion state space parameters according to the local target point information and the collected current environment state information and the motion state information of the obstacle, and generate action behavior information according to the first motion state space parameters and the pre-trained local path planning model. The edge device transmits the target action behavior information to the robot end.

[0124] It should be noted that the training process of the pre-trained local path planning model can refer to the aforementioned embodiment and will not be described in detail here.

[0125] S5. The robot executes the target action behavior according to the action behavior information and plans the driving path.

[0126] S6. Determine whether the robot has reached the end position. If so, end; if not, repeat the above process until the robot reaches the end position safely without collision.

[0127] Based on the foregoing embodiments, the embodiments of the present application also involve a robot navigation system, which includes a robot end, an edge device end, a cloud device and a data transmission module, such as Figure 6 As shown. Among them:

[0128] The robot side consists of a state information acquisition module and a motion control module. The state information acquisition module is used to collect the robot's environmental state information, and the motion control module is used to adjust the robot's movements in real time.

[0129] The edge device side consists of a data storage module and an edge computing module. The data storage module is used to pre-store the global map information in the current environment and receive the environmental status information from the data transmission module. The edge computing module is used to perform operations on the global path planning model and the local path planning model.

[0130] The cloud device consists of multiple virtual cloud server nodes, which are used to store global map information in different environments and realize real-time update of global map information on the edge. The data transmission module is used for two-way data transmission between the robot and the edge device.

[0131] Specifically, the robot is responsible for collecting environmental status information, and then using a router to transmit the environmental status information to the edge device through the Internet; the edge device uses the global map information stored in the hard disk in advance through the edge computing module to perform the upper-level global path planning, decompose the planned global path into several local target point information, and distribute them to the lower-level local path planning model in turn. Among them, the global map information stored in the edge device can interact with the cloud device in real time through the Internet, and regularly update the stored global map information to ensure the correctness of the global map information and the rationality of the global path. The lower-level local path planning model is calculated in the edge device and outputs the optimal action behavior information. The edge device transmits the optimal action behavior information to the robot in real time through the router; the robot calls the motion control module to perform the optimal action to achieve safe and collision-free arrival at the target point (i.e., the end point).

[0132] In the embodiments of the present application, the real-time, reliability and adaptability of robot navigation are improved, and intelligent, automated and efficient services can be provided for all walks of life, such as logistics distribution, security monitoring, inspection and maintenance, etc. The traditional local path planning algorithm represented by the RVO algorithm in the related art is short-term in space and time, and the local path planning algorithm based on deep reinforcement learning has the problem of insufficient extraction of scene features, resulting in unreasonable environment modeling or unrealistic predicted motion trajectory, and thus it is difficult to train the optimal strategy. In view of the above problems, the embodiments of the present application propose a local path planning algorithm for a robot based on deep reinforcement learning, which extracts spatial feature information and temporal feature information in the navigation environment by introducing a multi-layer graph attention network, an additional attention module, a multi-layer parallel GRU network, a PPO algorithm, etc., so that the robot can dynamically learn the interactive relationship with the navigation environment, so that the robot has human-like memory ability, improves the adaptability of the robot to dynamic and complex environments, and finally enables the robot to reach the target point safely and efficiently.

[0133] In order to meet the real-time, reliability and adaptability requirements of robot navigation, the embodiment of the present application adopts an edge computing mode, which distributes the high computing load of global map information and path planning algorithm to the edge device for execution, thereby effectively solving the shortcomings of the cloud computing model in terms of latency and reliability, while also reducing the computing resources and storage consumption on the robot side.

[0134] Compared with the related art, the embodiments of the present application have the following technical advantages:

[0135] The embodiments of the present application utilize the advantages of low latency, high efficiency, and high flexibility of edge computing, and combine global path planning methods with local path planning methods to enable robot navigation in dynamic and complex environments, thereby improving the real-time, reliability, and adaptability of robot navigation.

[0136] Compared with the global path planning method in the related art that needs to rely on fixed map information, the embodiment of the present application stores the global map information on the edge device side and updates it regularly. At the same time, path planning is performed on the edge device side, which can reduce the storage and resource consumption of the robot and improve the reliability and adaptability of navigation.

[0137] Compared with the traditional local path planning method represented by the RVO algorithm, which is short-term in space and time, the local path planning algorithm proposed in the embodiment of the present application fully considers the spatial factors and the temporal factors.

[0138] Compared with the local path planning method in the related art, due to insufficient extraction of scene features, the interaction relationship between moving objects and the long-term dependency relationship between states are not fully and comprehensively considered, resulting in unreasonable environment modeling or unrealistic predicted motion trajectory, and thus difficulty in training the optimal strategy, the embodiment of the present application proposes to extract spatial feature information and temporal feature information in the scene, which can not only enable the robot to dynamically learn the interaction relationship with the navigation environment, but also enable it to have human-like memory ability, thereby improving the robot's adaptability to dynamic and complex environments, and ultimately enabling the robot to reach the target point safely and efficiently.

[0139] Based on the above embodiment, in another embodiment of the present application, an edge device 1 is provided, such as Figure 7 As shown, the edge device 1 includes:

[0140] The acquisition unit 10 is used to acquire the first motion state space parameters of the robot.

[0141] The determination unit 11 is used to input the first motion state space parameter into the pre-trained local path planning model to obtain the first spatial feature information, and determine the first temporal feature information based on the first spatial feature information.

[0142] The determination unit 11 is further used to determine the behavior action information of the robot based on the first time series feature information, and the behavior action information is used to plan the navigation path of the robot.

[0143] In one embodiment, the first motion state space parameter includes a first parameter, a second parameter and a third parameter, and the pre-trained local path planning model includes a spatial feature extraction network.

[0144] In one embodiment, the edge device 1 may further include: a processing unit.

[0145] The processing unit is used to process the first parameter and the second parameter through a spatial feature extraction network to obtain first motion feature information of the robot and second motion feature information of obstacles in the robot's driving path.

[0146] The determination unit 11 is further configured to determine third motion feature information based on the first motion feature information and the second motion feature information.

[0147] The determination unit 11 is further configured to process the third parameter through a spatial feature extraction network to obtain fourth motion feature information between the robot and the obstacle.

[0148] The determination unit 11 is further configured to determine the third motion feature information and the fourth motion feature information as the first spatial feature information.

[0149] In one embodiment, the first spatial feature information includes third motion feature information and fourth motion feature information, and the pre-trained local path planning model includes a temporal feature extraction network.

[0150] In one embodiment, the processing unit is further configured to process the third motion feature information through a temporal feature extraction network to obtain second temporal feature information.

[0151] The processing unit is further used to process the fourth motion feature information through a time series feature extraction network to obtain third time series feature information.

[0152] The determining unit 11 is further configured to determine the first timing characteristic information based on the second timing characteristic information and the third timing characteristic information.

[0153] In one embodiment, the acquisition unit 10 is further used to acquire the first motion state information of the robot, the local target point information in the driving path, and the second motion state information of the obstacle.

[0154] The determination unit 11 is further configured to determine a spatial parameter of the first motion state space based on the first motion state information, the local target point information and the second motion state information.

[0155] In one embodiment, the acquisition unit 10 is further used to acquire the target navigation position information of the robot.

[0156] The acquisition unit 10 is further configured to acquire, based on the first motion state information, global map information corresponding to the first motion state information from the cloud device.

[0157] The determination unit 11 is further used to input the global map information and the target navigation position information into the pre-trained global path planning model to obtain the global path information of the robot.

[0158] The determination unit 11 is further configured to determine local target point information based on the global path information.

[0159] In one embodiment, the edge device 1 may further include: a sending unit.

[0160] The sending unit is used to send the behavior action information to the robot so that the robot can plan a navigation path based on the received behavior action information.

[0161] In one embodiment, the determination unit 11 is further configured to determine a target number of target point information from the global path information at intervals of a preset distance value.

[0162] The determination unit 11 is further used to determine the target number of target point information as local target point information.

[0163] The present application provides an edge device for obtaining a first motion state spatial parameter of a robot; inputting the first motion state spatial parameter into an initial local path planning model to obtain first spatial feature information, and determining first temporal feature information based on the first spatial feature information; determining the robot's behavior action information based on the first temporal feature information, and the behavior action information is used to plan the robot's navigation path. It can be seen that an edge device proposed in an embodiment of the present application can reduce the consumption of computing and storage resources on the robot side by executing the robot's behavior action information confirmation process on the edge device side, and adopts a pre-trained local path planning model on the edge device side, and obtains the spatial feature information and temporal feature information of the robot during the movement process through the robot's first motion state parameter, so that the extracted scene information is richer, so that the robot's behavior action information output by the local path planning model is more in line with the actual scene, so the navigation path planned by the robot through the pre-trained local path planning model is safer.

[0164] Figure 8 A schematic diagram of the structure of an edge device 1 provided in an embodiment of the present application. In practical applications, based on the same disclosed concept of the above embodiments, Figure 8 As shown, the edge device 1 of the embodiment of the present application includes a processor 12 , a memory 13 and a communication bus 14 .

[0165] In the specific implementation process, the acquisition unit 10, the determination unit 11, the processing unit, and the sending unit can be implemented by a processor 12 located on the edge device 1, and the processor 12 can be a specific application integrated circuit (ASIC, Application Specific Integrated Circuit), a digital signal processor (DSP, Digital Signal Processor), a digital signal processing image processing device (DSPD, Digital Signal Processing Device), a programmable logic image processing device (PLD, Programmable Logic Device), a field programmable gate array (FPGA, Field Programmable Gate Array), a CPU, a controller, a microcontroller, and at least one of a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and the embodiments of the present application are not specifically limited.

[0166] In the embodiment of the present application, the communication bus 14 is used to realize the connection and communication between the processor 12 and the memory 13; when the processor 12 executes the running program stored in the memory 13, the following robot navigation method is realized:

[0167] Acquire the first motion state space parameters of the robot; input the first motion state space parameters into the initial local path planning model to obtain first spatial feature information, and determine first temporal feature information based on the first spatial feature information; determine the robot's behavioral action information based on the first temporal feature information, and the behavioral action information is used to plan the robot's navigation path.

[0168] In one embodiment, the first motion state space parameter includes a first parameter, a second parameter and a third parameter, and the pre-trained local path planning model includes a spatial feature extraction network.

[0169] In one embodiment, the processor 13 is further used to process the first parameter and the second parameter through a spatial feature extraction network to obtain first motion feature information of the robot and second motion feature information of obstacles in the robot's driving path; determine third motion feature information based on the first motion feature information and the second motion feature information; process the third parameter through the spatial feature extraction network to obtain fourth motion feature information between the robot and the obstacle; and determine the third motion feature information and the fourth motion feature information as the first spatial feature information.

[0170] In one embodiment, the first spatial feature information includes third motion feature information and fourth motion feature information, and the pre-trained local path planning model includes a temporal feature extraction network.

[0171] In one embodiment, the processor 13 is further used to process the third motion feature information through a timing feature extraction network to obtain second timing feature information; process the fourth motion feature information through a timing feature extraction network to obtain third timing feature information; and determine the first timing feature information based on the second timing feature information and the third timing feature information.

[0172] In one embodiment, the processor 13 is also used to obtain the first motion state information of the robot, the local target point information in the driving path and the second motion state information of the obstacle; and determine the first motion state space parameters based on the first motion state information, the local target point information and the second motion state information.

[0173] In one embodiment, the processor 13 is further used to obtain target navigation position information of the robot; based on the first motion state information, obtain global map information corresponding to the first motion state information from a cloud device; input the global map information and the target navigation position information into a pre-trained global path planning model to obtain the global path information of the robot; based on the global path information, determine the local target point information.

[0174] In one embodiment, the processor 13 is further configured to send the behavior action information to the robot, so that the robot can plan a navigation path based on the received behavior action information.

[0175] In one embodiment, the processor 13 is further configured to determine a target number of target point information from the global path information at intervals of a preset distance value; and determine the target number of target point information as local target point information.

[0176] Based on the above embodiments, an embodiment of the present application provides a storage medium on which a computer program is stored. The above computer-readable storage medium stores one or more programs. The above one or more programs can be executed by one or more processors and applied to edge devices. The computer program implements the robot navigation method as described above.

[0177] Based on the above embodiments, an embodiment of the present application provides a computer program product, including a computer program, which can be executed by one or more processors and applied to an edge device. The computer program implements the robot navigation method as described above.

[0178] It should be noted that in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for an image display device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the method described in each embodiment of the embodiment of the present application.

[0180] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A robot navigation method, characterized in that: Applied to an edge device, the method comprises: Obtaining the first motion state space parameters of the robot; Inputting the first motion state space parameter into a pre-trained local path planning model to obtain first spatial feature information, and determining first temporal feature information based on the first spatial feature information; Based on the first time series feature information, the behavior action information of the robot is determined, and the behavior action information is used to plan a navigation path of the robot.

2. The method according to claim 1, characterized in that The first motion state space parameter includes a first parameter, a second parameter and a third parameter, the pre-trained local path planning model includes a spatial feature extraction network, and the first motion state space parameter is input into the pre-trained local path planning model to obtain the first spatial feature information, including: Processing the first parameter and the second parameter through the spatial feature extraction network to obtain first motion feature information of the robot and second motion feature information of obstacles in the driving path of the robot; Determining third motion feature information based on the first motion feature information and the second motion feature information; Processing the third parameter through the spatial feature extraction network to obtain fourth motion feature information between the robot and the obstacle; The third motion feature information and the fourth motion feature information are determined as the first spatial feature information.

3. The method according to claim 1, characterized in that The first spatial feature information includes third motion feature information and fourth motion feature information, the pre-trained local path planning model includes a temporal feature extraction network, and the determining of the first temporal feature information based on the first spatial feature information includes: Processing the third motion feature information through the time series feature extraction network to obtain second time series feature information; Processing the fourth motion feature information through the time series feature extraction network to obtain third time series feature information; The first timing characteristic information is determined based on the second timing characteristic information and the third timing characteristic information.

4. The method according to claim 1, characterized in that: The obtaining of the first motion state space parameter of the robot comprises: Acquire first motion state information of the robot, local target point information in a driving path, and second motion state information of obstacles; The first motion state space parameter is determined based on the first motion state information, the local target point information and the second motion state information.

5. The method according to claim 4, characterized in that The obtaining of local target point information in the driving path includes: Obtaining target navigation position information of the robot; Based on the first motion state information, obtaining global map information corresponding to the first motion state information from a cloud device; Inputting the global map information and the target navigation position information into a pre-trained global path planning model to obtain the global path information of the robot; Based on the global path information, the local target point information is determined.

6. The method according to claim 1, characterized in that After determining the behavior information of the robot based on the first time series feature information, the method further includes: The behavior action information is sent to the robot so that the robot plans a navigation path based on the received behavior action information.

7. The method according to claim 5, characterized in that The determining the local target point information based on the global path information includes: Determine the target number of target point information from the global path information at intervals of a preset distance value; The target number of target point information is determined as the local target point information.

8. An edge device, characterized in that: The edge device includes: An acquisition unit, used for acquiring a first motion state space parameter of the robot; a determining unit, configured to input the first motion state space parameter into a pre-trained local path planning model to obtain first spatial feature information, and determine first temporal feature information based on the first spatial feature information; The determination unit is further used to determine the behavior action information of the robot based on the first time series feature information, and the behavior action information is used to plan the navigation path of the robot.

9. An edge device, characterized in that: The edge device comprises: a processor and a memory; when the processor executes the running program stored in the memory, the method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 7.

Images