A robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask

Abstract

The application belongs to the field of autonomous robot trajectory planning and decision-making, and specifically discloses a robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask, which comprises the following steps: MDP modeling is performed on the dynamic obstacle avoidance problem itself, and a simulation environment is built; a sensor data prediction method based on an improved long short-term memory network is designed, sensor data in the next period of time is predicted by inputting current and past sensor data; a sensor dynamic data mask is designed based on a spatial attention network, and mask data and original data are fused to expand the sensor data; a strategy function and a reward function are designed, and a decision network is trained based on the simulation environment. Through the integration of the reinforcement learning algorithm, the obstacle avoidance strategy can be continuously optimized, and the navigation and obstacle avoidance ability of the robot in a complex environment can be improved.

Description

Technical Field

[0001] This invention belongs to the field of autonomous robot trajectory planning and decision-making, and more specifically, relates to a robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic masking. Background Technology

[0002] Mobile robots are increasingly used in automated logistics, home services, and rescue operations. In these applications, robots need to navigate and perform tasks autonomously in complex and ever-changing environments, making obstacle avoidance a core function. Traditional obstacle avoidance technologies are primarily based on geometric algorithms, such as artificial potential field methods, convex hull methods, and Rapid Random Tree Exploration (RRT). While existing obstacle avoidance algorithms perform reasonably well in static environments, they often struggle to make fast and effective obstacle avoidance decisions in dynamic environments, especially when obstacles move quickly or unexpected situations arise. Furthermore, these algorithms typically require pre-defining the position and shape of obstacles, lacking sufficient adaptability to unknown or changing obstacles.

[0003] In recent years, with the development of machine learning technology, especially the successful application of reinforcement learning in fields such as autonomous driving and gaming, people have begun to try to apply it to the obstacle avoidance problem of mobile robots. Reinforcement learning can learn optimal strategies through interaction with the environment, and has good adaptability and generalization ability. However, existing reinforcement learning-based obstacle avoidance methods still face some challenges in practical applications, such as low training efficiency and slow response speed to environmental changes. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a robot dynamic obstacle avoidance method and system based on sequence prediction and dynamic mask, the purpose of which is to improve the robot's navigation and obstacle avoidance capabilities in complex environments.

[0005] To achieve the above objectives, according to one aspect of the present invention, a robot dynamic obstacle avoidance method based on sequence prediction and dynamic masking is proposed, comprising the following steps:

[0006] Model training phase:

[0007] The robot dynamic obstacle avoidance problem is transformed into a Markov decision process; based on the Markov decision process, a decision network based on reinforcement learning is constructed and trained.

[0008] (1) Input the current observation data obtained by the robot sensor into the pre-trained prediction network. The prediction network predicts the sensor prediction data for the next moment based on the current and past observation data.

[0009] (2) Dynamic data masking is performed on the prediction data based on the self-attention network, and the obtained mask data is fused with the current observation data to expand the prediction data;

[0010] (3) Input the expanded prediction data into the decision network to obtain the robot's action. In the simulation environment, the robot moves according to the action. Update the parameters of the self-attention network and the decision network according to the reward function.

[0011] (4) Repeat steps (1) to (3) until the preset termination condition is met to obtain the trained self-attention network and decision network;

[0012] Based on the trained prediction network, self-attention network, and decision network, an obstacle avoidance model is obtained.

[0013] Model usage phase:

[0014] During the robot's movement, the robot's sensors acquire sensor observation data in real time, input the observation data into the obstacle avoidance model, and obtain the robot's actions to achieve dynamic obstacle avoidance.

[0015] As a further preferred embodiment, the prediction network is pre-trained using sensor observation data sequences obtained from robot dynamic obstacle avoidance simulation; the prediction network is represented as follows:

[0016]

[0017] Where f1 and f2 represent Long Short-Term Memory networks, and φ1 and φ2 represent pooling layer neural networks; t This represents the sensor observation data at time t; Δo t h represents the difference between the sensor observation data at time t and time t-1; t ,c t This represents the hidden layer state of the Long Short-Term Memory network f1, which is the feature vector of all historical data before time t. The hidden state of the long short-term memory network f2 is represented by the feature vector of the difference between all historical data before time t; φ1{f1[·]} represents the sensor data at time t+1 predicted from historical sensor data. φ2{f2[·]} represents the difference between the sensor data at time t+1 and the current sensor data predicted by the difference of historical sensor data. o t ' +1 The output of the prediction network is the sensor prediction data at time t+1.

[0018] As a further preferred option, the maximum mean error function is used when training the prediction network.

[0019] As a further optimization, the prediction data is expanded, specifically as follows:

[0020]

[0021] in, It is expanded sensor data, o t o represents the sensor observation at time t. t ' +1 This is the sensor prediction data at time t+1, which is the output of the prediction network. ψ represents the self-attention network, and φ3 and φ4 represent two data pooling layers used to achieve data fusion. This indicates the addition of two vector matrices.

[0022] As a further preferred embodiment, the decision network employs a policy function. as follows:

[0023]

[0024]

[0025] Among them, a t =(v t ,ω t () represents the final decision, i.e., the action the robot performs, including velocity v. t With angular velocity ω t ; Let x be the initial distance between the robot and the target; t ,y t () represents the robot's position coordinates at time t. d represents the coordinates of the obstacle's center. s Safety distance from obstacles; Let t represent the distance between the robot and the obstacle; α, β, and γ represent the weights of the decision objective.

[0026] As a further preferred option, the weights α, β, and γ are adjusted according to an adaptive weight allocation mechanism:

[0027]

[0028] in, The safety criterion for the robot and obstacles, r rob The collision radius of the robot, r obs The radius of collision of an obstacle is represented by r. t This represents the preset safe distance threshold between the robot and the obstacle, excluding the collision radius.

[0029] As a further preferred embodiment, the reward function is:

[0030] R = a·rtar +b·r obs +c·r θ +d·r step

[0031] Where R represents the final reward signal; r tar The reward is the evaluation of the target, representing how close the robot's strategy is to the target; r obs The obstacle evaluation reward represents the degree to which the robot stays away from the obstacle; r θ A corner evaluation reward is provided to prevent excessively large corners in the global path; r step The global step length evaluation reward is used to control the global path length to be as small as possible while satisfying other objectives; a, b, c, and d are preset proportional coefficients.

[0032] As a further optimization, the target evaluation reward r tar The formula for calculation is:

[0033]

[0034] in, The distance from the target point before the robot makes its current decision. The distance from the target point after the robot makes a decision;

[0035] Obstacle evaluation reward r obs The formula for calculation is:

[0036]

[0037]

[0038]

[0039] in, r represents the distance from the i-th obstacle before and after the robot makes its decision, respectively; vo Indicates a reward based on speed obstacles. v represents the feasible velocity and angular velocity between the robot and the i-th obstacle, calculated using the velocity-obstacle method. r is the robot speed; n is the total number of obstacles; k and m are proportionality coefficients.

[0040] Corner evaluation reward r θ The formula for calculation is:

[0041]

[0042] Where θ0 represents the relative angle of the target point in the initial robot posture, θ j Let N represent the turning point in the j-th step decision, and N be the number of decisions.

[0043] Global step size evaluation reward r step The formula for calculation is:

[0044]

[0045] Where l0 represents the straight-line distance between the initial robot position and the target point, l j This indicates the length of the decision execution at step j.

[0046] As a further optimization, when designing the simulation environment, a domain randomization mechanism is adopted to increase the complexity of the scenario.

[0047] According to another aspect of the present invention, a robot dynamic obstacle avoidance system based on sequence prediction and dynamic masking is provided, including a processor for executing the above-described robot dynamic obstacle avoidance method based on sequence prediction and dynamic masking.

[0048] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0049] 1. This invention makes predictions based on current and historical data in sensor data sequences and combines them with dynamic masks to obtain dynamic and static fusion data, thereby making robot action decisions and improving the robot's dynamic obstacle avoidance capabilities. Based on the formed obstacle avoidance model, it can respond to environmental changes in real time and quickly generate obstacle avoidance paths. Moreover, through the integration of reinforcement learning algorithms, it can continuously optimize obstacle avoidance strategies and improve the robot's navigation and obstacle avoidance capabilities in complex environments.

[0050] 2. This invention uses deep learning to predict sensor data sequences and data differences. It combines the maximum mean error loss function to calculate the mean of the two prediction results to obtain the final result. Compared with the current method of directly predicting data, this invention improves the prediction accuracy.

[0051] 3. This invention uses sensor prediction data and a self-attention mechanism to dynamically mask static sensor data, resulting in dynamic-static fusion data. Compared to existing methods that directly use static or dynamic data for decision-making, this invention improves the perception dimension.

[0052] 4. This invention improves the learning objective by designing global path length and turning angle constraints, optimizes the policy function, and combines the design of response reward function with a dynamic learning objective weight allocation mechanism to dynamically adjust the learning process, thereby ensuring the flexibility of decision-making and improving the path efficiency of the decision-making process at the same time. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the design of a robot dynamic obstacle avoidance method based on sequence prediction and dynamic masking, as described in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the prediction network and the neural network of the dynamic masking method in an embodiment of the present invention;

[0055] Figure 3 This is a flowchart of the decision network training process according to an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0057] This invention provides a robot dynamic obstacle avoidance method based on sequence prediction and dynamic masking, such as... Figure 1 As shown, it includes:

[0058] S1. Obstacle Avoidance Problem Modeling. The robot's dynamic obstacle avoidance problem is described as a Markov Decision Process (MDP) model; the problem input is defined as the sensor data carried by the robot itself, and the output is the actions performed by the robot.

[0059] Specifically, sensor data can be lidar sensor data, depth image data, etc., representing the relative distance between the robot and obstacles. In this embodiment, taking the dynamic obstacle avoidance problem of a four-wheeled independently driven mobile robot equipped with lidar sensors as an example, the MDP modeling design for the dynamic obstacle avoidance problem is as follows:

[0060] State space: The state space is all the characteristics of the robot in the environment, such as position, velocity, sensor information, etc., and is generally represented by S.

[0061] Observation Space: Due to the limitations of sensor range, the environment perceived by a mobile robot differs from its actual environment. Therefore, the environment that a mobile robot can perceive is defined as its observation space, denoted by O.

[0062] O={l h ,l t ,d tar ,θ tar}

[0063] Among them, l h This represents historical data from the lidar, l t d represents the current lidar data. tar θ represents the current distance between the mobile robot and the target point. tarThis indicates the orientation angle of the target relative to the mobile robot in the current mobile robot posture.

[0064] Action space: The set of actions that a mobile robot can take is called the robot's action space, denoted by A. Specifically, the formula for A is as follows:

[0065]

[0066] Here, (v, ω) are the direct outputs of the neural network, representing velocity v and angular velocity ω, respectively. The velocity output ranges from (0, 1) and does not represent the direct action executed by the mobile robot; rather, it needs to be adjusted based on the maximum sensing range of the sensors. The formula for the actual action executed by the mobile robot is:

[0067]

[0068] Among them, v ture This represents the actual action performed by the mobile robot. It is the action v output by the neural network, processed by a parameter r corresponding to the maximum range value of the LiDAR. lidar The linear mapping is used, where 0.2 is the mapping coefficient, used to control the step size of the mobile robot to prevent it from becoming too long. When the radar range is small, the actual step size is limited to 0.2 times the maximum range value of the lidar; when the radar range is large enough, the actual step size is the current output value of the neural network. This design improves the versatility of the method to adapt to different types of radar and application scenarios.

[0069] Policy function: A formula representing the target policy that the robot needs to learn. For dynamic obstacle avoidance tasks, the policy function is generally expressed as:

[0070]

[0071] in, Let be the policy function. The robot's target policy is the action sampled from the policy function that minimizes the final distance to the target while avoiding collisions with the target and other objects. t It is the action result obtained from the sampling strategy function; The initial distance between the robot and the target. The relative azimuth angle of the target point in the current posture of the mobile robot; (x t ,y t () represents the robot's position coordinates at time t. d represents the coordinates of the obstacle's center. s Safety distance from obstacles; This represents the distance between the robot and the obstacle at time t.

[0072] S2. Set up a simulation environment for subsequent training of prediction and decision networks.

[0073] Specifically, Webots software is used to build a simulation environment, which has two main functions: 1) to perform mobile robot simulation modeling, build the robot's perception and motion model, and collect simulation environment data and decision execution; 2) to build a dynamic environment and set up a domain randomization mechanism so that the robot can cope with diverse scenarios during simulation training, thereby improving the generalization of the method.

[0074] Simulation environment data can be expressed using the following formula:

[0075] E = {S} r ,S obs ,Ω|S r ={p r ,v r ,o r ,p tar},S obs ={P obs V obs}}

[0076] P obs ={p obs,1 ,p obs,2 ,...,p obs,n},V obs ={v obs,1 ,v obs,2 ,...,v obs,n}

[0077] Here, E represents the simulation environment dataset, which is mainly divided into three categories, including robot data S. r Obstacle data S obs And the environmental boundary range Ω. Robot data mainly includes the robot's own coordinates p. r Speed ​​v r Self-sensor data r And the manually set target point coordinates p tar Obstacle data includes the coordinates P of all obstacles. obs With speed V obs p obs,i Let v represent the coordinates of the i-th obstacle. obs,i Let represent the velocity of the i-th obstacle.

[0078] Furthermore, the domain randomization mechanism is used to increase the complexity of the scenario. Specifically, by changing simulation environment elements such as obstacles, the initial positions of the robot and the target point, and the initial velocities of obstacles at the beginning of each training session, the training scenario is discretized and made more complex. Robot policies trained under such scenarios possess a certain degree of generalization ability. The domain randomization mechanism formula is expressed as follows:

[0079]

[0080] S obs =rand(p obs,1 )+rand(v obs,1 )+rand(p obs,2 )+rand(p obs,2 )+...

[0081] +rand(p obs,n )+rand(p obs,n ), i≠j,p obs,i ≠p obs,j i,j=1,2,...,n

[0082] p r =rand(p r ),p tar =rand(p tar )

[0083] The above formula means that at the beginning of each training session, all obstacles will be placed at random locations within the boundary of the simulation environment and have random speeds. The robot and the target point will also be randomly placed in the simulation environment without initial speeds, and the positions of all obstacles, robots and target points will be different from each other.

[0084] Furthermore, to facilitate the subsequent training of the prediction and decision networks, a data acquisition and storage mechanism is designed to provide high-quality datasets. Data acquisition and storage includes two types of data: the first type is used for training the prediction network, and the second type is used for training the decision network. The formulas are as follows:

[0085] D1={o0,o1,Δo1,o2,Δo2,...,o t ,Δo t}

[0086] D 2,t ={o t ,Δo t+1 ,a t ,r t ,o t+1 ,Δo t+2}

[0087] Where D1 and D2 represent the training data for the prediction network and the decision network, respectively. It is particularly important to note that o t Δo represents the spatial observation data of the mobile robot at time t. t =o t -o t-1 , representing the difference in observation data from the mobile robot at two consecutive time points. Data D1 is collected at the beginning of each training session and stops at the end of each session; therefore, each data point in D1 has a different length. Data D2 is stored after each training run, and all stored data is of a fixed length. 2,t This represents a data point obtained from making a decision at time t.

[0088] S3. Pre-training of the prediction network. The prediction network is pre-trained offline using sensor data D1. The prediction network is based on an improved long short-term memory network design, which predicts sensor data for the next time period by taking current and past sensor data as input.

[0089] Furthermore, such as Figure 2 As shown, the prediction network expression is as follows:

[0090]

[0091] Where f1 and f2 represent Long Short-Term Memory (LSTM) networks, and φ1 and φ2 represent pooling layers in a neural network. f1 and φ1 are used to directly process raw historical sensor data. t with h t ,c t Predicting sensor data for the next moment f2 and φ2 are used to measure the historical data difference Δo between sensor data frames. t and To predict the difference between the sensor data at the next moment and the current sensor data. h t ,c t This represents the hidden state of the Long Short-Term Memory (LSM) network f1, which is the feature vector of all historical data up to time t. This represents the hidden state of the Long Short-Term Memory (LSM) network f2, specifically the feature vector representing the differences between all historical data up to time t. To reduce the impact of sensor data perturbations on the prediction results, the final sensor data prediction result is obtained by calculating the average of the two results mentioned above. t ' +1 .

[0092] Furthermore, to make the prediction network's results more accurate, the maximum mean error function is used during training, with the specific formula as follows:

[0093]

[0094] Where N is the sensor data length, d i This represents the actual data value of each sensor data point. This represents the data value predicted by the neural network. The accuracy of the prediction for each data point is ensured by using the maximum mean error.

[0095] S4. Dynamic Mask Generation and Fusion. Based on the output of the prediction network... t ' +1 A sensor dynamic data mask is designed based on a self-attention network, and the mask data is fused with the original observation data to expand the sensor prediction data.

[0096] Furthermore, dynamic data masks can extract dynamic information between data frames from different sensors, serving as an extension of robot observation. Specifically, this is expressed as follows:

[0097]

[0098] in, These are robot observations after expanding the observation space. t It is the sensor observation value at time t, o t ' +1 The data at time t+1 is predicted by the trained prediction network based on the original sensor data at time t. ψ represents the self-attention network, and φ3 and φ4 represent two data pooling layers used to achieve data fusion. This indicates the addition of two vector matrices.

[0099] S5. Policy Function Design. The learning objective during training is optimized by designing a reinforcement learning policy function that considers global path efficiency constraints, and adaptive weight allocation is performed based on the learning objectives at different learning stages. Specifically:

[0100] Based on the conventional dynamic obstacle avoidance strategy function introduced in S1, this invention optimizes it. The learning objective optimization introduces global path constraints on the existing reinforcement learning learning objective. By setting global path length and global path turning angle constraints, the global path efficiency of the method is improved, while the flexibility of the method is enhanced by designing an adaptive weight allocation mechanism. The optimized robot strategy function is shown below:

[0101]

[0102]

[0103] Among them, a t =(v t ,ω t) represents the robot's final decision, which mainly includes speed and angular velocity. Let x be the initial distance between the robot and the target; t ,y t () represents the robot's position coordinates at time t. d represents the coordinates of the obstacle's center. s Safety distance from obstacles; This represents the distance between the robot and the obstacle at time t.

[0104] The main objectives of the decision-making process are: 1) to minimize the distance to the target point during decision-making; 2) to reach the target point with the fewest steps; 3) to reach the target point with the smallest turning angle; and 4) to ensure that the robot maintains a certain distance from obstacles during the decision-making process. During navigation, the training objectives of reinforcement learning need to be dynamically adjusted in real time to address various possible situations. Therefore, the learning objectives need to be dynamically adjusted in real time according to the learning process.

[0105] α, β, and γ represent the weights of the decision objective, respectively. The designed dynamic learning strategy objective adaptive weight allocation mechanism needs to adaptively adjust the weights α, β, and γ. The adjustment mechanism formula is shown below:

[0106]

[0107] in, This indicates the safety criteria for robots and obstacles. r represents the distance between the robot and the obstacle at time t. rob The collision radius of the robot, r obs The radius of collision of an obstacle is represented by r. t This represents a manually set safe distance threshold between the robot and an obstacle, excluding the collision radius.

[0108] S6. Reward Function Design. An obstacle avoidance reward function is designed based on velocity-obstacle theory, incorporating a target arrival reward to encourage the robot to generate the optimal collision avoidance strategy; its expression is:

[0109] r = a·r tar +b·r obs +c·r θ +d·r step

[0110] Here, r represents the final reward signal, which is divided into four parts. a, b, c, and d are the proportional coefficients of the four parts, which are fixed values ​​set by humans.

[0111] The first part is the target evaluation and reward. tarIt represents how close the robot's strategy is to the target, calculated from the distance to the target point before the robot makes its current decision. Distance from the target point after making the decision The calculation yielded:

[0112]

[0113] The second part is obstacle evaluation and reward. obs It represents the degree to which the robot is far from the obstacle, and it is divided into two parts. One part is calculated by measuring the difference in distance between the robot and the nearest obstacle before and after the robot makes a decision. get, The distances to the i-th obstacle before and after the robot's decision are represented by two values, respectively. The second value is the speed-obstacle theory, which determines whether the robot has chosen a feasible speed, thus yielding a speed-obstacle-based reward r. vo , Let represent the feasible velocity and angular velocity between the robot and the i-th obstacle, calculated using the velocity-obstacle method. k and m are the proportionality coefficients between the two, as shown in the following formula:

[0114]

[0115] The third part is the corner evaluation reward. θ This is used to control the global path turning angle to prevent it from becoming too large. It is determined by the relative angle θ0 of the target point in the initial robot posture during the navigation task and the turning angle θ at the j-th step in the global decision-making process. j The calculation yielded:

[0116]

[0117] The fourth part is the global step size evaluation reward r. step This is used to control the global path length to be as small as possible while satisfying other objectives. This evaluation is mainly calculated from the straight-line distance between the robot's initial position and the target point in the navigation task, and the path length at each step.

[0118]

[0119] Where N represents the number of decisions in this round, l j l0 represents the length of the decision execution at step j, and l0 represents the straight-line distance between the robot's position and the target point at the beginning of the navigation task.

[0120] S7. Based on the aforementioned policy function and reward function, and the established simulation environment, train the decision network, and monitor the training process in real time to make adjustments. For example... Figure 3 As shown, the specific steps include the following:

[0121] 1) Initialize the training environment, randomly reset the positions of the target point, obstacles, and robot in the simulation environment, as well as the initial velocity of the obstacles;

[0122] 2) Read the robot's sensor observation data. t Input the pre-trained prediction network to obtain the prediction result. t ' +1 And perform dynamic mask calculations to obtain

[0123] 3) According to The data is fed into a decision network to calculate the action decision (v). t ,ω t );

[0124] 4) In the simulation environment, decisions are made based on action (v) t ,ω t The robot performs corresponding actions to acquire new sensor data. t+1 ;

[0125] 5) Calculate the reward value r based on the designed reward function. t ;

[0126] 6) Determine whether the robot has reached the target or collided with it. If yes, return to step 1); otherwise, proceed to the next step.

[0127] 7) Set the process value (o) for each training step. t ,v t ,ω t ,r t ,o t+1 Store in the experience pool;

[0128] 8) Randomly sample the experience pool, calculate the network loss, and update the network parameters;

[0129] 9) Determine if a training session has exceeded the maximum number of steps. If not, return to step 2); if so, proceed to the next step.

[0130] 10) Determine if the maximum number of training games has been exceeded. If not, return to step 1); if yes, end the training.

[0131] Through the above process, the decision network and the self-attention network for dynamic masking are trained. Based on the trained prediction network, self-attention network, and decision network, an obstacle avoidance model is formed. During robot movement, sensor data is acquired in real time through the robot's onboard sensors, and this sensor data is input into the obstacle avoidance model to obtain the robot's actions, thus achieving dynamic obstacle avoidance.

[0132] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robot dynamic obstacle avoidance method based on sequence prediction and dynamic masking, characterized in that, include: Model training phase: Transform the robot dynamic obstacle avoidance problem into a Markov decision process; Based on Markov decision processes, construct and train a decision network based on reinforcement learning: (1) Input the current observation data obtained by the robot sensor into the pre-trained prediction network. The prediction network predicts the sensor prediction data for the next moment based on the current and past observation data. (2) Dynamic data masking is performed on the prediction data based on the self-attention network, and the obtained mask data is fused with the current observation data to expand the prediction data; (3) Input the expanded prediction data into the decision network to obtain the robot's action. In the simulation environment, the robot moves according to the action. Update the parameters of the self-attention network and the decision network according to the reward function. (4) Repeat steps (1) to (3) until the preset termination condition is met to obtain the trained self-attention network and decision network; Based on the trained prediction network, self-attention network, and decision network, an obstacle avoidance model is obtained. Model usage phase: During the robot's movement, the robot's sensors acquire sensor observation data in real time, input the observation data into the obstacle avoidance model, and obtain the robot's actions to achieve dynamic obstacle avoidance.

2. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 1, characterized in that, The prediction network is pre-trained using sensor observation data sequences obtained from robot dynamic obstacle avoidance simulation; the prediction network is represented as follows: Where f1 and f2 represent Long Short-Term Memory networks, and φ1 and φ2 represent pooling layer neural networks; t This represents the sensor observation data at time t; Δo t h represents the difference between the sensor observation data at time t and time t-1; t ,c t This represents the hidden layer state of the Long Short-Term Memory network f1, which is the feature vector of all historical data before time t. The hidden state of the long short-term memory network f2 is represented by the feature vector of the difference between all historical data before time t; φ1{f1[·]} represents the sensor data at time t+1 predicted from historical sensor data. φ2{f2[·]} represents the difference between the sensor data at time t+1 and the current sensor data predicted by the difference of historical sensor data. o t ' +1 The output of the prediction network is the sensor prediction data at time t+1.

3. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 2, characterized in that, When training the prediction network, the maximum mean error function is used.

4. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 1, characterized in that, The forecast data is expanded, specifically as follows: in, It is expanded sensor data, o t o represents the sensor observation at time t. t ' +1 This is the sensor prediction data at time t+1, which is the output of the prediction network. ψ represents the self-attention network, and φ3 and φ4 represent two data pooling layers used to achieve data fusion. This indicates the addition of two vector matrices.

5. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 1, characterized in that, The decision network employs a policy function. as follows: Among them, a t =(v t ,ω t () represents the final decision, i.e., the action the robot performs, including velocity v. t With angular velocity ω t ; Let x be the initial distance between the robot and the target; t ,y t () represents the robot's position coordinates at time t. d represents the coordinates of the obstacle's center. s Safety distance from obstacles; Let t represent the distance between the robot and the obstacle; α, β, and γ represent the weights of the decision objective.

6. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 5, characterized in that, The weights α, β, and γ are adjusted according to the adaptive weight allocation mechanism: in, The safety criterion for the robot and obstacles, r rob The collision radius of the robot, r obs The radius of collision of an obstacle is represented by r. t This represents the preset safe distance threshold between the robot and the obstacle, excluding the collision radius.

7. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 5, characterized in that, The reward function is: R=a·r tar +b·r obs +c·r θ +d·r step Where R represents the final reward signal; r tar The reward is the evaluation of the target, representing how close the robot's strategy is to the target; r obs The obstacle evaluation reward represents the degree to which the robot stays away from the obstacle; r θ A corner evaluation reward is provided to prevent excessively large corners in the global path; r step The global step length evaluation reward is used to control the global path length to be as small as possible while satisfying other objectives; a, b, c, and d are preset proportional coefficients.

8. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in claim 7, characterized in that, Target evaluation reward r tar The formula for calculation is: in, The distance from the target point before the robot makes its current decision. The distance from the target point after the robot makes a decision; Obstacle evaluation reward r obs The formula for calculation is: in, r represents the distance from the i-th obstacle before and after the robot makes its decision, respectively; vo Indicates a reward based on speed obstacles. v represents the feasible velocity and angular velocity between the robot and the i-th obstacle, calculated using the velocity-obstacle method. r is the robot speed; n is the total number of obstacles; k and m are proportionality coefficients. Corner evaluation reward r θ The formula for calculation is: Where θ0 represents the relative angle of the target point in the initial robot posture, θ j Let N represent the turning point in the j-th step decision, and N be the number of decisions. Global step size evaluation reward r step The formula for calculation is: Where l0 represents the straight-line distance between the initial robot position and the target point, l j This indicates the length of the decision execution at step j.

9. The robot dynamic obstacle avoidance method based on sequence prediction and dynamic mask as described in any one of claims 1-8, characterized in that, When designing the simulation environment, a domain randomization mechanism is adopted to increase the complexity of the scenario.

10. A robot dynamic obstacle avoidance system based on sequence prediction and dynamic masking, characterized in that, Includes a processor for executing the robot dynamic obstacle avoidance method based on sequence prediction and dynamic masking as described in any one of claims 1-9.

Images