Robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask

Abstract

The invention belongs to the field of autonomous robot trajectory planning and decision making, and particularly discloses a robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask.The method comprises the steps that MDP modeling is conducted on a dynamic obstacle avoidance problem, and a simulation environment is built; designing a sensor data prediction method based on an improved long-short-term memory network, and predicting sensor data in a next period of time by inputting sensor data in a current period of time and sensor data in a past period of time; designing a sensor dynamic data mask based on the spatial attention network, performing data fusion on mask data and original data, and expanding the sensor data; designing a strategy function and a reward function, and training the decision network based on the simulation environment. Through integration of reinforcement learning algorithms, an obstacle avoidance strategy can be continuously optimized, and the navigation and obstacle avoidance capabilities of the robot in a complex environment are improved.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous robot trajectory planning and decision-making, and more specifically, relates to a robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask. Background Art

[0002] Mobile robots are increasingly used in automated logistics, home services, exploration and rescue, and other fields. In these application scenarios, robots need to autonomously navigate and perform tasks in complex and changing environments, and obstacle avoidance is one of its core functions. Traditional obstacle avoidance technologies are mainly based on geometric obstacle avoidance algorithms, such as the artificial potential field method, the convex hull method, and the rapidly exploring random tree (RRT). Existing obstacle avoidance algorithms perform well in static environments, but in dynamic environments, especially when obstacles move quickly or emergencies occur, it is often difficult to make fast and effective obstacle avoidance decisions. In addition, these algorithms usually require the position and shape of obstacles to be defined in advance, and lack sufficient adaptability to unknown or changing obstacles.

[0003] In recent years, with the development of machine learning technology, especially the successful application of reinforcement learning in autonomous driving, games and other fields, people have begun to try to apply it to the obstacle avoidance problem of mobile robots. Reinforcement learning can learn the optimal strategy through interaction with the environment and has good adaptability and generalization ability. However, the existing obstacle avoidance methods based on reinforcement learning still face some challenges in practical applications, such as low training efficiency and slow response to environmental changes. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask, the purpose of which is to improve the robot's navigation and obstacle avoidance capabilities in complex environments.

[0005] To achieve the above object, according to one aspect of the present invention, a robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask is proposed, comprising the following steps:

[0006] Model training phase:

[0007] Convert the robot's dynamic obstacle avoidance problem into a Markov decision process; based on the Markov decision process, build and train a decision network based on reinforcement learning:

[0008] (1) The current observation data obtained by the robot sensor is input into the pre-trained prediction network. The prediction network predicts the sensor prediction data at the next moment based on the current and past observation data;

[0009] (2) Dynamically mask the predicted data based on the self-attention network, and fuse the obtained mask data with the current observation data to expand the predicted data;

[0010] (3) Input the expanded prediction data into the decision network to obtain the robot's execution action. The robot performs the action in the simulation environment and updates the parameters of the self-attention network and the decision network according to the reward function.

[0011] (4) Repeat steps (1) to (3) until the preset termination condition is reached, and the trained self-attention network and decision network are obtained;

[0012] Obtain the obstacle avoidance model based on the trained prediction network, self-attention network and decision network;

[0013] Model usage phase:

[0014] During the robot's movement, the robot's sensors obtain sensor observation data in real time, input the observation data into the obstacle avoidance model, obtain the robot's execution action, and realize the robot's dynamic obstacle avoidance.

[0015] As a further preferred embodiment, the prediction network is pre-trained by a sensor observation data sequence obtained by the robot dynamic obstacle avoidance simulation; the prediction network is represented as:

[0016]

[0017] Among them, f 1 、f 2 represents the long short-term memory network, φ 1 ,φ 2 Represents the pooling layer neural network; o t represents the sensor observation data at the current time t; Δo t It represents the difference between the sensor observation data at time t and time t-1; h t ,c t represents the long short-term memory network f 1 The hidden layer state of , that is, the feature vector of all historical data before time t; represents the long short-term memory network f 2 The hidden layer state of , that is, the feature vector of all historical data differences before time t; φ 1 {f 1 [·]} means predicting the sensor data at time t+1 by using historical sensor data φ 2 {f 2 [·]} indicates the difference between the sensor data at time t+1 and the current sensor data predicted by the historical sensor data difference o t '+1 is the output of the prediction network, that is, the sensor prediction data at time t+1.

[0018] As a further preference, when training the prediction network, a maximum mean error function is used.

[0019] As a further preferred embodiment, the prediction data is expanded, which is specifically expressed as follows:

[0020]

[0021] in, is the expanded sensor data, o t represents the sensor observation value at time t, o t ' +1 is the sensor prediction data at time t+1 output by the prediction network, ψ represents the self-attention network, and φ 3 and φ 4 Represents two data pooling layers, used to achieve data fusion, Represents the addition of two vector matrices.

[0022] As a further preferred embodiment, the strategy function adopted by the decision network is as follows:

[0023]

[0024]

[0025] Among them, a t =(v t ,ω t ) represents the final decision, that is, the robot performs the action, including the speed v t With angular velocity ω t ; is the initial distance between the robot and the target; (x t ,y t ) represents the position coordinates of the robot at time t, represents the coordinates of the obstacle center, d s Safety distance for obstacles; represents the distance between the robot and the obstacle at time t; α, β and γ represent the weights of the decision target.

[0026] As a further preferred embodiment, the weights α, β and γ are adjusted according to an adaptive weight allocation mechanism:

[0027]

[0028] in, represents the safety criterion between the robot and the obstacle, r robRepresents the collision radius of the robot, r obs Represents the collision radius of the obstacle, r t Indicates the preset safety distance threshold between the robot and the obstacle after removing the collision radius.

[0029] As a further preferred embodiment, the reward function is:

[0030] R = a·r tar + b·r obs +c·r θ +d·r step

[0031] Among them, R represents the final reward signal; r tar is the target evaluation reward, which represents the degree to which the robot strategy approaches the target; r obs is the obstacle evaluation reward, which represents the degree to which the robot is away from obstacles; r θ is the corner evaluation reward, which is used to prevent the global path from turning too far; r step is the global step length evaluation reward, which is used to control the global path length to be as small as possible while meeting other objective conditions; a, b, c, d are preset proportional coefficients.

[0032] As a further preferred example, the target evaluation reward r tar The calculation formula is:

[0033]

[0034] in, is the distance from the target point before the robot makes the current decision, The distance from the target point after the robot makes a decision;

[0035] Obstacle Evaluation Reward obs The calculation formula is:

[0036]

[0037]

[0038]

[0039] in, Respectively represent the distance of the robot from the ith obstacle before and after the decision; r vo represents the reward based on the speed obstacle, represents the feasible speed and angular velocity between the robot and the ith obstacle calculated by the speed obstacle method, v r is the robot speed; n is the total number of obstacles, k and m are proportional coefficients;

[0040] Corner Evaluation Rewards θ The calculation formula is:

[0041]

[0042] Among them, θ 0 represents the relative angle of the target point under the initial robot posture, θ j represents the turning point of the j-th decision, and N is the number of decisions;

[0043] Global step evaluation reward r step The calculation formula is:

[0044]

[0045] Among them, l 0 represents the straight-line distance between the initial robot position and the target point, l j Represents the length of the j-th decision execution.

[0046] As a further preferred embodiment, when designing the simulation environment, a domain randomization mechanism is used to increase the complexity of the scene.

[0047] According to another aspect of the present invention, a robot dynamic obstacle avoidance system based on sequence prediction and dynamic mask is provided, comprising a processor, wherein the processor is used to execute the above-mentioned robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask.

[0048] In general, the above technical solution conceived by the present invention has the following technical advantages compared with the prior art:

[0049] 1. The present invention makes predictions based on current and historical data in the sensor data sequence, and combines dynamic masks to obtain dynamic and static fusion data, and then makes robot action decisions, which can improve the robot's dynamic obstacle avoidance ability; based on the formed obstacle avoidance model, it can respond to environmental changes in real time and quickly generate obstacle avoidance paths, and through the integration of reinforcement learning algorithms, it can continuously optimize obstacle avoidance strategies and improve the robot's navigation and obstacle avoidance capabilities in complex environments.

[0050] 2. The present invention adopts a deep learning method to predict the sensor data sequence and data difference, combines the maximum mean error loss function, and solves the mean of the two prediction results to obtain the final result. Compared with the current method of directly predicting data, the prediction accuracy is improved.

[0051] 3. Based on sensor prediction data and combined with the self-attention mechanism, the present invention dynamically masks the static sensor data to obtain dynamic and static fusion data. Compared with the existing direct use of static data or direct use of dynamic data for decision-making, the perception dimension is improved.

[0052] 4. The present invention designs global path length and turning angle constraints to improve learning objectives, optimizes the strategy function, combines the response reward function design with the dynamic learning objective weight allocation mechanism, and dynamically adjusts the learning process to ensure the flexibility of decision-making and at the same time improve the path efficiency of the decision-making process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of a robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to an embodiment of the present invention is provided;

[0054] Figure 2 A schematic diagram of a neural network of a prediction network and a dynamic masking method according to an embodiment of the present invention;

[0055] Figure 3 The figure is a flow chart of decision network training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] The embodiment of the present invention provides a robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask, such as Figure 1 As shown, including:

[0058] S1. Obstacle avoidance problem modeling. The robot dynamic obstacle avoidance problem is described as a Markov decision process and MDP modeling is performed; the problem input is defined as the sensor data carried by the robot itself, and the output is the action performed by the robot.

[0059] Specifically, the sensor data may be laser radar sensor data, depth image data, etc., which indicates the relative distance between the robot and the obstacle. In this embodiment, taking the dynamic obstacle avoidance problem of a four-wheel independent drive mobile robot equipped with a laser radar sensor as an example, the MDP modeling design of the dynamic obstacle avoidance problem is as follows:

[0060] State space: State space refers to all objective characteristics of the robot in the environment, such as position, speed, sensor information, etc., generally represented by S.

[0061] Observation space: Due to the limited sensing range of the sensor, the environment it perceives is different from the actual environment. Therefore, the environment that the mobile robot can perceive is defined as the observation space, which is represented by O:

[0062] O={l h ,lt ,d tar ,θ tar}

[0063] Among them, l h Represents the historical data of LiDAR, l t Represents the current lidar data, d tar Indicates the distance between the current mobile robot and the target point, θ tar Indicates the orientation angle of the target relative to the mobile robot in the current mobile robot posture.

[0064] Action space: The set of actions that a mobile robot can take is the robot's action space, represented by A. Specifically, the formula for A is as follows:

[0065]

[0066] Among them, (v, ω) is the direct output of the neural network, representing the velocity v and angular velocity ω respectively. The value range of the velocity output is (0, 1), which does not represent the direct execution action of the mobile robot, but needs to be adjusted according to the maximum perception range of the sensor. The action formula executed by the real mobile robot is:

[0067]

[0068] Among them, v ture represents the actual execution action of the mobile robot, which is the action v output by the neural network through a value r with the maximum range of the laser radar lidar The linear mapping is obtained by , where 0.2 is the mapping coefficient, which is used to control the step length of each step of the mobile robot so that it is not too long. When the radar range is small, the actual step length will be limited to 0.2 times the maximum range value of the lidar; when the radar range is large enough, the actual step length is the current output value of the neural network. This design improves the versatility of the method to adapt to different types of radars and application scenarios.

[0069] Strategy function: The formula for the target strategy that the robot needs to learn. For dynamic obstacle avoidance tasks, the strategy function is generally expressed as:

[0070]

[0071] in, is the strategy function. The robot's target strategy is the action sampled by the strategy function, which can minimize the distance to the target and avoid collision with the target and other objects after execution. t is the action result obtained by the sampling strategy function; is the initial distance between the robot and the target, is the relative azimuth of the target point in the current posture of the mobile robot; (x t ,y t ) represents the position coordinates of the robot at time t, represents the coordinates of the obstacle center, d s Safety distance for obstacles; It represents the distance between the robot and the obstacle at time t.

[0072] S2. Build a simulation environment for subsequent prediction network and decision network training.

[0073] Specifically, Webots software is used to build a simulation environment. The simulation environment has two main functions: 1) It is used to perform mobile robot simulation modeling, build the robot's perception and motion model, and collect simulation environment data and make decisions; 2) It is used to build a dynamic environment and set up a domain randomization mechanism, so that the robot can cope with a variety of scenarios during simulation training, thereby improving the generalization of the method.

[0074] The simulation environment data can be expressed by the following formula:

[0075] E={S r ,S obs ,Ω|S r ={p r ,v r ,o r ,p tar},S obs = {P obs ,V obs}}

[0076] P obs ={p obs,1 ,p obs,2 ,...,p obs,n},V obs = {v obs,1 ,v obs,2 ,...,v obs,n}

[0077] Among them, E represents the simulation environment data set. The simulation environment data is mainly divided into three categories, including robot data S r , obstacle data S obs and the environmental boundary range Ω. The robot data mainly includes the robot's own coordinates p r , speed v r , own sensor data r And the artificially set target point coordinates p tar ; The obstacle data includes the coordinates P of all obstacles obs With speed V obs , p obs,irepresents the coordinates of the ith obstacle, v obs,i represents the speed of the i-th obstacle.

[0078] Furthermore, the domain randomization mechanism is used to increase the complexity of the scene. Specifically, by changing the obstacles in the simulation environment, the initial positions of the robot and the target point, the initial speed of the obstacles and other simulation environment elements at the beginning of each training process, the training scene is discretized and complicated. The robot strategy trained in such a scene has a certain generalization ability. The formula of the domain randomization mechanism is expressed as follows:

[0079]

[0080] S obs =rand(p obs,1 )+rand(v obs,1 )+rand(p obs,2 )+rand(p obs,2 )+...

[0081] +rand(p obs,n )+rand(p obs,n ),i≠j,p obs,i ≠p obs,j ,i,j=1,2,...,n

[0082] p r =rand(p r ),p tar =rand(p tar )

[0083] The meaning of the above formula is that at the beginning of each training session, all obstacles will be placed at random positions within the boundary of the simulation environment and have random speeds, and the robots and target points will also be randomly placed in the simulation environment without initial speeds, and the positions of all obstacles, robots, and target points are different from each other.

[0084] Furthermore, in order to facilitate the subsequent training of prediction networks and decision networks, a data collection and storage mechanism is designed to provide subsequent high-quality data sets. Data collection and storage include two types of data: the first type is used for prediction network training, and the second type is used for decision network training. The formula is as follows:

[0085] D 1 ={o 0 ,o 1 ,Δo 1 ,o 2 ,Δo 2 ,...,o t ,Δo t}

[0086] D 2,t ={o t ,Δo t+1 ,a t ,r t ,o t+1 ,Δo t+2}

[0087] Among them, D 1 and D 2 They represent the prediction network training data and the decision network training data respectively. In particular, o t represents the observation space data of the mobile robot at time t, Δo t =o t -o t-1 , represents the difference between the observation data of the mobile robot at two consecutive time points. D 1 Data is collected from the beginning of each round and stopped at the end of a round of training, so D 1 The length of each data is different. 2 The data is stored once each time training is performed, and the data stored is of fixed length. 2,t Represents a piece of data obtained by making a decision at time t.

[0088] S3, prediction network pre-training. Through sensor data D 1 The prediction network is pre-trained offline. The prediction network is designed based on an improved long short-term memory network. It predicts the sensor data for the next period of time by inputting the sensor data of the current and past period of time.

[0089] Further, such as Figure 2 As shown, the prediction network expression is as follows:

[0090]

[0091] Among them, f 1 、f 2 represents the long short-term memory network LSTM, φ 1 ,φ 2 represents the pooling layer neural network. 1 and φ 1 For direct access to historical sensor raw data o t With h t ,c t Predict the sensor data for the next moment f 2 and φ 2 The difference Δo between the historical data of the sensor data frames is used t and To predict the difference between the sensor data at the next moment and the current sensor data h t ,c t represents the long short-term memory network f 1 The hidden layer state of , that is, the feature vector of all historical data before time t, represents the long short-term memory network f 2 The hidden layer state of is the feature vector of all historical data differences before time t. In order to reduce the impact of sensor data disturbance on the prediction result, the final sensor data prediction result is obtained by calculating the mean of the above two results. t ' +1 .

[0092] Furthermore, in order to make the prediction network results more accurate, the maximum mean error function is used during training. The specific formula is as follows:

[0093]

[0094] Where N is the sensor data length, d i represents the true data value of each sensor data point, Represents the data value predicted by the neural network. The accuracy of the prediction for each data point is guaranteed by the maximum mean error.

[0095] S4, dynamic mask generation and fusion. According to the output of the prediction network t ' +1 , based on the self-attention network, a sensor dynamic data mask is designed, and the mask data is fused with the original observation data to expand the sensor prediction data.

[0096] Furthermore, the dynamic data mask can extract the dynamic information between different sensor data frames as an extension of the robot observation. Specifically expressed as:

[0097]

[0098] in, is the robot observation value after expanding the observation space, o t is the sensor observation value at time t, o t ' +1 is the data at time t+1 predicted by the trained prediction network based on the original sensor data at time t, ψ represents the self-attention network, and φ 3 and φ 4 Respectively represent two data pooling layers, used to achieve data fusion, Represents the addition of two vector matrices.

[0099] S5. Strategy function design. Optimize the learning objectives during training, design a reinforcement learning strategy function that takes into account the global path efficiency constraints, and perform adaptive weight allocation based on the learning objectives at different learning stages. Specifically:

[0100] Based on the conventional dynamic obstacle avoidance strategy function introduced in S1, the present invention optimizes it. The learning goal optimization introduces global path constraints on the basis of the existing reinforcement learning learning goals, and improves the global path efficiency of the method by setting global path length and global path angle constraints, and improves the flexibility of the method by designing an adaptive weight allocation mechanism. The optimized robot strategy function is shown below:

[0101]

[0102]

[0103] Among them, a t =(v t ,ω t ) represents the final decision of the robot, which mainly includes speed and angular velocity. is the initial distance between the robot and the target; (x t ,y t ) represents the position coordinates of the robot at time t, represents the coordinates of the obstacle center, d s Safety distance for obstacles; It represents the distance between the robot and the obstacle at time t.

[0104] The main goals of decision making are: 1) minimize the distance to the target point when making a decision; 2) reach the target point with the least step length; 3) reach the target point with the smallest turning angle; 4) ensure that the robot is at a certain distance from obstacles during the decision making process. In the process of navigation, facing various possible situations, it is necessary to dynamically adjust the training goals of reinforcement learning in real time, so the learning goals need to be dynamically adjusted in real time according to the learning process.

[0105] α, β and γ represent the weights of the decision-making objectives respectively. The designed dynamic learning strategy objective adaptive weight allocation mechanism needs to adaptively adjust the weights α, β and γ. The adjustment mechanism formula is as follows:

[0106]

[0107] in, represents the safety criterion between the robot and the obstacle, represents the distance between the robot and the obstacle at time t, r rob Represents the collision radius of the robot, r obsRepresents the collision radius of the obstacle, r t It indicates the manually set safety distance threshold between the robot and the obstacle after removing the collision radius.

[0108] S6. Reward function design. Based on the speed obstacle theory, the obstacle avoidance reward function is designed, combined with the target arrival reward, to encourage the robot to generate the optimal collision avoidance strategy; its expression is:

[0109] r = a·r tar + b·r obs +c·r θ +d·r step

[0110] Among them, r represents the final reward signal, which is divided into four parts. a, b, c, and d are the proportional coefficients of the four parts, which are fixed values ​​set artificially.

[0111] The first part is the target evaluation reward r tar , which represents the degree to which the robot strategy is close to the target, and is determined by the distance from the target point before the robot makes the current decision The distance to the target point after making a decision The calculation results are:

[0112]

[0113] The second part is the obstacle evaluation reward r obs , which represents the degree to which the robot is away from obstacles. It is also divided into two parts. The first part is to calculate the distance difference between the robot and the nearest obstacle before and after the decision. get, They represent the distance of the robot from the ith obstacle before and after the decision, respectively. Secondly, the speed obstacle theory is used to determine whether the robot has chosen a feasible speed, thereby obtaining a reward r based on the speed obstacle. vo , It represents the feasible speed and angular velocity between the robot and the ith obstacle calculated by the speed obstacle method. k and m are the proportional coefficients of the two. The formula is as follows:

[0114]

[0115] The third part is the corner evaluation reward r θ , which is used to control the global path angle not to be too large. It is determined by the relative angle θ of the target point under the initial robot posture of the navigation task. 0 and the turning angle θ at step j in the global decision process j The calculation results are:

[0116]

[0117] The fourth part is the global step evaluation reward r step , which is used to control the global path length to be as small as possible while meeting other target conditions. This evaluation is mainly calculated by the straight-line distance between the initial robot position and the target point of the navigation task and the path length of each step:

[0118]

[0119] Among them, N represents the number of decisions in this game, l j represents the length of the j-th step decision execution, l 0 Indicates the straight-line distance between the robot position and the target point at the beginning of the navigation task.

[0120] S7. Based on the above strategy function and reward function, as well as the constructed simulation environment, the decision network is trained, and the training process is monitored in real time to make adjustments. Figure 3 As shown, the specific steps include:

[0121] 1) Initialize the training environment, randomly reset the target point, the position of the obstacle and the robot in the simulation environment, and the initial speed of the obstacle;

[0122] 2) Read the robot's sensor observation data t , input the pre-trained prediction network to obtain the prediction result o t ' +1 , and perform dynamic mask calculation to obtain

[0123] 3) According to Send it to the decision network and calculate the action decision (v t ,ω t );

[0124] 4) In the simulation environment, according to the action decision (v t ,ω t ) Let the robot perform corresponding actions and obtain new sensor data t+1 ;

[0125] 5) Calculate the reward value r according to the designed reward function t ;

[0126] 6) Determine whether the robot has reached the target or collided. If yes, return to step 1); if not, proceed to the subsequent steps;

[0127] 7) The process value of each step of training (o t ,v t ,ω t ,r t ,o t+1 ) is stored in the experience pool;

[0128] 8) Randomly sample the experience pool, calculate network loss, and update network parameters;

[0129] 9) Determine whether a training session exceeds the maximum number of steps. If not, return to step 2); if yes, proceed to the subsequent steps;

[0130] 10) Determine whether the maximum number of training rounds has been exceeded. If not, return to step 1); if yes, end the training.

[0131] Through the above process, the training of the decision network is completed, and the training of the self-attention network for dynamic masking is completed at the same time. Based on the trained prediction network, self-attention network and decision network, an obstacle avoidance model is formed; during the robot movement, the sensor data is obtained in real time through the sensors carried by the robot itself, and the sensor data is input into the obstacle avoidance model to obtain the robot's execution action, thereby realizing the robot's dynamic obstacle avoidance.

[0132] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask, characterized in that: include: Model training phase: Convert the robot dynamic obstacle avoidance problem into a Markov decision process; Based on the Markov decision process, build and train a decision network based on reinforcement learning: (1) The current observation data obtained by the robot sensor is input into the pre-trained prediction network. The prediction network predicts the sensor prediction data at the next moment based on the current and past observation data; (2) Dynamically mask the predicted data based on the self-attention network, and fuse the obtained mask data with the current observation data to expand the predicted data; (3) Input the expanded prediction data into the decision network to obtain the robot's execution action. The robot performs the action in the simulation environment and updates the parameters of the self-attention network and the decision network according to the reward function. (4) Repeat steps (1) to (3) until the preset termination condition is reached, and the trained self-attention network and decision network are obtained; Obtain the obstacle avoidance model based on the trained prediction network, self-attention network and decision network; Model usage phase: During the robot's movement, the robot's sensors obtain sensor observation data in real time, input the observation data into the obstacle avoidance model, obtain the robot's execution action, and realize the robot's dynamic obstacle avoidance.

2. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to claim 1, characterized in that: The prediction network is pre-trained through the sensor observation data sequence obtained by the robot dynamic obstacle avoidance simulation; the prediction network is expressed as: Among them, f1 and f2 represent long short-term memory networks, φ1 and φ2 represent pooling layer neural networks; o t represents the sensor observation data at the current time t; Δo t It represents the difference between the sensor observation data at time t and time t-1; h t ,c t Represents the hidden layer state of the long short-term memory network f1, that is, the feature vector of all historical data before time t; represents the hidden layer state of the long short-term memory network f2, that is, the feature vector of all historical data differences before time t; φ1{f1[·]} represents the prediction of sensor data at time t+1 by historical sensor data φ2{f2[·]} represents the difference between the sensor data at time t+1 and the current sensor data predicted by the historical sensor data difference o t ' +1 is the output of the prediction network, that is, the sensor prediction data at time t+1.

3. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as claimed in claim 2, characterized in that: When training the prediction network, the maximum mean error function is used.

4. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to claim 1, characterized in that: The prediction data is expanded, specifically expressed as follows: in, is the expanded sensor data, o t represents the sensor observation value at time t, o t ' +1 is the sensor prediction data at time t+1 output by the prediction network, ψ represents the self-attention network, φ3 and φ4 represent two data pooling layers for data fusion. Represents the addition of two vector matrices.

5. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to claim 1, characterized in that: The policy function adopted by the decision network as follows: Among them, a t =(v t ,ω t ) represents the final decision, that is, the robot performs the action, including the speed v t With angular velocity ω t ; is the initial distance between the robot and the target; (x t ,y t ) represents the position coordinates of the robot at time t, represents the coordinates of the obstacle center, d s Safety distance for obstacles; represents the distance between the robot and the obstacle at time t; α, β and γ represent the weights of the decision target.

6. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to claim 5, characterized in that: The weights α, β, and γ are adjusted according to the adaptive weight allocation mechanism: in, represents the safety criterion between the robot and the obstacle, r rob Represents the collision radius of the robot, r obs Represents the collision radius of the obstacle, r t Indicates the preset safety distance threshold between the robot and the obstacle after removing the collision radius.

7. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to claim 5, characterized in that: The reward function is: R=a·r tar +b·r obs +c·r θ +d·r step Among them, R represents the final reward signal; r tar is the target evaluation reward, which represents the degree to which the robot strategy approaches the target; r obs is the obstacle evaluation reward, which represents the degree to which the robot is away from obstacles; r θ is the corner evaluation reward, which is used to prevent the global path from turning too far; r step is the global step length evaluation reward, which is used to control the global path length to be as small as possible while meeting other objective conditions; a, b, c, d are preset proportional coefficients.

8. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to claim 7, characterized in that: Target evaluation reward tar The calculation formula is: in, is the distance from the target point before the robot makes the current decision, The distance from the target point after the robot makes a decision; Obstacle Evaluation Reward obs The calculation formula is: in, Respectively represent the distance of the robot from the ith obstacle before and after the decision; r vo represents the reward based on the speed obstacle, represents the feasible speed and angular velocity between the robot and the ith obstacle calculated by the speed obstacle method, v r is the robot speed; n is the total number of obstacles, k and m are proportional coefficients; Corner Evaluation Rewards θ The calculation formula is: Among them, θ0 represents the relative angle of the target point under the initial robot posture, θ j represents the turning point of the j-th decision, and N is the number of decisions; Global step evaluation reward r step The calculation formula is: Among them, l0 represents the straight-line distance between the initial robot position and the target point, l j Represents the length of the j-th decision execution.

9. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask according to any one of claims 1 to 8, characterized in that: When designing the simulation environment, a domain randomization mechanism is used to increase the complexity of the scene.

10. A robot dynamic obstacle avoidance system based on sequence prediction and dynamic mask, characterized in that: It includes a processor, and the processor is used to execute the robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in any one of claims 1-9.

Images