Navigation method and device of mobile robot

Abstract

The invention provides a navigation method and device for a mobile robot, and is applied to the technical field of robot navigation, and the method comprises the following steps: obtaining pedestrian position information collected by the mobile robot, inputting the pedestrian position information to a robot navigation model, and obtaining the position information of the pedestrian through a memory pool; storing historical motion information, neighbor motion information and future motion information; determining a predicted pedestrian state of the target pedestrian based on the historical motion information, the neighbor motion information and the future motion information through a state prediction module; taking each pedestrian in the observation time window as a target pedestrian, and determining a predicted pedestrian state of each pedestrian; and acquiring a robot state of the mobile robot through a value estimation module, and determining a target navigation action of the mobile robot based on the robot state and the predicted pedestrian state of each pedestrian. According to the invention, the prediction accuracy of the mobile robot in different scenes can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot navigation, and in particular to a navigation method and device for a mobile robot. Background Art

[0002] Mobile robot navigation refers to the technology that allows mobile robots to perceive the surrounding environment, autonomously plan paths, and move along predetermined or optimized paths.

[0003] For mobile robots, navigating in a crowd is a huge challenge. On the one hand, people in the environment cannot be simply regarded as static or dynamic obstacles, and there may be some complex interactions between people. On the other hand, the movement of pedestrians is random and diverse, making it difficult to accurately model them based on certain rules.

[0004] It can be seen that the mobile robot navigation method in the related art has a technical problem of inaccurate navigation prediction caused by ignoring the interaction relationship between pedestrians. Summary of the invention

[0005] The present invention provides a navigation method and device for a mobile robot, which are used to solve the defect of inaccurate navigation prediction caused by ignoring the interaction relationship between pedestrians in the mobile robot navigation method in the prior art, and improve the prediction accuracy of the mobile robot in different scenarios.

[0006] The present invention provides a navigation method for a mobile robot, comprising the following steps.

[0007] The pedestrian position information collected by the mobile robot is obtained, wherein the pedestrian position information includes: a position information sequence of each pedestrian; the pedestrian position information is input into a robot navigation model to obtain a target navigation action output by the robot navigation model, wherein the robot navigation model includes: a memory pool, a state prediction module and a value estimation module, wherein the following steps are included: through the memory pool, within a current observation time window, the position information sequence of the target pedestrian before the target time step is used as historical motion information; the position information sequence of other pedestrians before the target time step is used as neighbor motion information; the position information sequence of the target pedestrian after the target time step is used as future motion information; through the state prediction module, based on the historical motion information, the neighbor motion information and the future motion information, the predicted pedestrian state of the target pedestrian is determined; each pedestrian within the observation time window is respectively used as the target pedestrian to determine the predicted pedestrian state of each pedestrian; through the value estimation module, the robot state of the mobile robot is obtained, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

[0008] According to a navigation method for a mobile robot provided by the present invention, the memory pool includes: a first memory pool, a second memory pool and a third memory pool; the first memory pool is used to store the robot state and environmental rewards of the robot navigation model at each time step during the training process; the second memory pool is used to store the position information sequence of each pedestrian in the historical observation time window; the third memory pool is used to store the position information sequence of each pedestrian in the current observation time window.

[0009] According to a navigation method of a mobile robot provided by the present invention, the state prediction module includes: a recurrent neural network and a decoder; the predicted pedestrian state of the target pedestrian is determined based on the historical motion information, the neighbor motion information and the future motion information, including: encoding the historical motion information, the neighbor motion information and the future motion information based on the attention mechanism through the recurrent neural network to obtain the social feature vector of the target pedestrian; decoding the social feature vector through the decoder to obtain the predicted pedestrian state of the target pedestrian.

[0010] According to a navigation method for a mobile robot provided by the present invention, the value estimation module includes: a relationship graph network and a value network; the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian, including: through the relationship graph network, based on the robot state and the predicted pedestrian state of each pedestrian, a feature matrix and a relationship matrix are constructed; calling a graph convolutional network to encode based on the feature matrix and the relationship matrix to obtain interactive relationship features; through the value network, the interactive relationship features are estimated based on a preset action value function. The action with the highest value in the action space is determined as the target navigation action.

[0011] According to a navigation method for a mobile robot provided by the present invention, before inputting the pedestrian position information into a robot navigation model to obtain a target navigation action output by the robot navigation model, the method also includes: training a preset robot navigation model based on knowledge distillation to obtain a trained robot navigation model.

[0012] According to a navigation method for a mobile robot provided by the present invention, the training includes: teacher model training and student model training, wherein the preset robot navigation model is trained based on knowledge distillation to obtain a trained robot navigation model, including: based on the preset teacher model, an operation simulation is performed in a target scene based on an optimal mutual collision avoidance strategy to obtain target scene experience data; based on the preset student model, an operation simulation is performed in a source scene to obtain source scene experience data; based on the target scene experience data, a state prediction module and a value estimation module in the preset teacher model are initialized; based on reinforcement learning and supervised learning, the state prediction module and the value estimation module in the preset teacher model after initialization are jointly trained to obtain a trained teacher model; through the trained teacher model, the state prediction module and the value estimation module in the preset student model are initialized based on the source scene experience data; based on reinforcement learning and supervised learning, the state prediction module and the value estimation module in the preset student model after initialization are jointly trained to obtain a trained student model; and the trained student model is used as a trained robot navigation model.

[0013] The present invention also provides a navigation device for a mobile robot, comprising the following modules: an acquisition module, used to acquire pedestrian position information collected by the mobile robot, wherein the pedestrian position information includes: a position information sequence of each pedestrian; a robot navigation module, used to input the pedestrian position information into a robot navigation model, and obtain a target navigation action output by the robot navigation model, wherein the robot navigation model includes: a memory pool, a state prediction module and a value estimation module, wherein the memory pool is used to take the position information sequence of the target pedestrian before the target time step as historical motion information within the current observation time window; the target time step is used as the target movement information; The position information sequence of other pedestrians before is used as the neighbor motion information; the position information sequence of the target pedestrian after the target time step is used as the future motion information; through the state prediction module, the predicted pedestrian state of the target pedestrian is determined based on the historical motion information, the neighbor motion information and the future motion information; each pedestrian in the observation time window is used as the target pedestrian, and the predicted pedestrian state of each pedestrian is determined; through the value estimation module, the robot state of the mobile robot is obtained, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a navigation method for a mobile robot as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the navigation method of the mobile robot as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the navigation method of the mobile robot as described in any one of the above is implemented.

[0017] The navigation method and device of the mobile robot provided by the present invention collect the position information of pedestrians, including the position information sequence of each pedestrian, through the mobile robot; store and process the historical motion information, neighbor motion information and future motion information of the target pedestrian within the current observation time window, so as to more comprehensively understand the motion pattern of the pedestrian; based on the attention mechanism, the predicted pedestrian state of the target pedestrian can be accurately determined; by taking each pedestrian within the observation time window as the target pedestrian, the predicted pedestrian state of each pedestrian can be determined, thereby enhancing the prediction accuracy of the pedestrian's motion trajectory; the robot state of the mobile robot is obtained, and the target navigation action of the mobile robot is determined in combination with the predicted pedestrian state of each pedestrian, thereby enabling the robot to more accurately judge the relative position relationship between itself and the pedestrian, thereby optimizing the navigation action and avoiding collision with the pedestrian; and further, the pedestrian motion information in different scenes can be processed, so that the robot can maintain stable navigation accuracy in different scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of the navigation method of the mobile robot provided by the present invention.

[0020] Figure 2 It is a schematic diagram of the mobile robot navigation strategy provided by the present invention.

[0021] Figure 3 It is a structural schematic diagram of the knowledge distillation training framework provided by the present invention.

[0022] Figure 4 It is a schematic diagram of the module structure of the navigation device of the mobile robot provided by the present invention.

[0023] Figure 5 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] For mobile robots, navigating in a crowd is a huge challenge. On the one hand, people in the environment cannot be simply regarded as static or dynamic obstacles, and there may be some complex interactions between people. On the other hand, the movement of pedestrians is random and diverse, which makes it difficult to accurately model based on certain rules. Robots must observe and predict the movement of people around them in this complex and dynamic environment, and maintain a certain safe distance while avoiding collisions.

[0026] In related technologies, some methods help robots make decisions by making assumptions about human motion behavior, but these assumptions are usually unrealistic. Some methods also avoid modeling pedestrians by directly learning policy networks, but these methods ignore the interactive relationships between pedestrians in various scenarios.

[0027] The models in related technologies are trained in specific scenarios. Although they can achieve good results in the original scenarios, when the scenarios change, there may be a certain degree of deviation between the model's prediction results and the actual state, so the performance of the method will decrease. In fact, most existing works do not discuss the ability of the robot's navigation strategy to be applied to different scenarios, that is, the model's scenario generalization performance, which is crucial for the practical application of robots.

[0028] The present invention provides a navigation method for a mobile robot. On the one hand, a pedestrian state predictor based on the SocialVAE (Social Variational AutoEncoder) method is designed to predict the pedestrian state based on the historical position information of the pedestrian and his neighbors, and to take into account the social interaction between pedestrians, which not only improves the prediction accuracy but also reduces the dependence on the scene. On the other hand, a robot navigation model training method based on knowledge distillation is designed, which distills the trained pedestrian state prediction module in different scenes, so that the navigation model finally trained can learn the information of different scenes.

[0029] Optionally, the navigation method of the mobile robot in the embodiment of the present application can be executed by a server, or by a terminal device (for example, a mobile robot), or can be executed jointly by a server and a terminal device, taking the example of the navigation method of the mobile robot in this embodiment being executed by a server.

[0030] Figure 1 is a flow chart of the navigation method of the mobile robot provided by the present invention, such as Figure 1 As shown, the method includes the following steps.

[0031] Step 101 : Acquire pedestrian position information collected by a mobile robot, wherein the pedestrian position information includes: a position information sequence of each pedestrian.

[0032] In some embodiments, the mobile robot is pre-equipped with multiple hardware devices for collecting pedestrian position information and its own status information.

[0033] For example, laser sensors, sonar sensors, infrared sensors and cameras. Among them, laser sensors are used to obtain high-precision distance information and are suitable for indoor and outdoor environments. Sonar sensors are suitable for indoor environments and can measure the distance to obstacles. Infrared sensors are used for obstacle avoidance and close-range detection and are usually used as auxiliary sensors. Cameras are used for visual navigation and positioning and can capture image information of pedestrians.

[0034] In some embodiments, the mobile robot senses the surrounding environment, including pedestrians, obstacles, etc., in real time through sensors. The sensor data is preprocessed, such as denoising and filtering, to improve the accuracy and reliability of the data. Pedestrians are detected in images captured by the camera using image processing algorithms (such as target detection and tracking algorithms), or three-dimensional detection and tracking of pedestrians are achieved through data fusion of laser sensors and sonar sensors. Based on the results of pedestrian detection and tracking, a position information sequence of each pedestrian is generated, and the position information sequence includes the position coordinates (such as X, Y, Z coordinates), speed, direction, etc. of the pedestrian.

[0035] Step 102: input the pedestrian position information into the robot navigation model to obtain the target navigation action output by the robot navigation model.

[0036] refer to Figure 2 , Figure 2 : is a schematic diagram of the mobile robot navigation strategy provided by the present invention, which includes: environment (Environment), memory pool (Memory), and action space (Action Space), wherein the action space includes: state prediction module (State Predictor) and value estimation module (Value Estimator), Represents a target navigation action.

[0037] The robot navigation model mentioned above includes: a memory pool, a state prediction module and a value estimation module, which include: Step 1021, through the memory pool, within the current observation time window, the position information sequence of the target pedestrian before the target time step is used as the historical motion information; the position information sequence of other pedestrians before the target time step is used as the neighbor motion information; and the position information sequence of the target pedestrian after the target time step is used as the future motion information.

[0038] According to the navigation method of the mobile robot provided by the present invention, the above-mentioned memory pool includes: a first memory pool, a second memory pool and a third memory pool; The first memory pool is used to store the robot state and environment reward at each time step during the training process of the robot navigation model; The second memory pool is used to store the position information sequence of each pedestrian in the historical observation time window; The third memory pool is used to store the position information sequence of each pedestrian in the current observation time window.

[0039] like Figure 2 As shown in the figure, the mobile robot moves and interacts in the environment, and the state of the mobile robot at each moment is and environmental rewards The records go into memory pools. For training and testing, three memory pools are designed.

[0040] First memory pool (Experience Memory) is used to store the status and rewards during training. Second memory pool The Trajectory Memory is used to store the historical trajectories of pedestrians (the location information sequence of each pedestrian in the historical observation time window), that is, to store the location information of humans in time series. These two memory pools are only used and updated during training, and are used to provide training data to the value estimation module and the state prediction module respectively. Horizon Memory is used to store the position information sequence of pedestrians in the latest observation time window of the current round, so the memory pool is kept for a fixed time length. In order to facilitate the prediction of pedestrian status in the simulation step, at the beginning of each round, only pedestrians will be observed, and the robot action decision will be made when the time length of the memory pool reaches the length of the observation time window.

[0041] It should be noted that the collected pedestrian location information includes the location information sequence of all n humans within the past continuous time T0, and the location information sequence of the first T1 time steps of the sequence is used as the historical location information, and the location information sequence of the last T2 time steps is used as the future location information.

[0042] For each target pedestrian i, the target pedestrian's historical motion information is obtained based on the target pedestrian's historical position information of the previous T1 time steps, including motion information such as position, speed, acceleration, etc. The target pedestrian's neighbor motion information is obtained based on the historical position information of the other n-1 humans in the previous T1 time steps, also including motion information such as position, speed, acceleration, etc. The target pedestrian's future motion information is obtained based on the target pedestrian's future position information after T2 time steps. T1 and T2 are pre-set parameters, where T0=T1+T2.

[0043] In the training step, the collected pedestrian position information here comes from the sampling of the second memory pool. The goal is to make predictions based on the historical position information of the previous T1 time steps, and compare the prediction results with the future position information of the next T2 time steps to optimize the parameters of the predictor; in the simulation step, the collected pedestrian position information comes from the third memory pool. At this time, the historical observation data only includes the known position information of T1 time steps. The goal is to predict the future position information of T2 time steps based on the known position information of T1 time steps.

[0044] In some embodiments, the first memory pool (experience memory pool) stores the state and reward of the mobile robot during the training process, which is used for the training value estimation module or the state prediction module. The content stored in the first memory pool includes the state of the mobile robot at each moment and the reward returned by the environment; after each interaction between the mobile robot and the environment, the new experience (robot state and environment reward) is added to the first memory pool.

[0045] The second memory pool (trajectory memory pool) stores the pedestrian's historical trajectory for training the state prediction module or imitation learning. Each entry of the stored content contains a sequence of pedestrian's position information within a certain period of time. This position information may include coordinates, speed, direction, etc. When the pedestrian moves, the new position information is added to the trajectory memory pool.

[0046] The third memory pool (horizontal memory pool) stores the position information sequence of pedestrians in the latest observation time window of the current round, which is used to predict the state of pedestrians in the simulation step. The stored content includes a continuous sequence of pedestrian position information in the current round, and the length of the sequence is equal to the length of the observation time window. At the beginning of each round, only observation is performed without action decision. When the time length in the memory pool reaches the length of the observation time window, the robot starts to make action decisions based on the information in the memory pool. During the observation process, as new human position information arrives, the content in the memory pool is continuously updated, while discarding past information to maintain a fixed time length.

[0047] Through the embodiments of the present invention, rich training data is provided by storing the status and rewards of the mobile robot during the training process; by storing the historical trajectory of humans, the mobile robot can learn the behavior patterns of humans; by storing the position information sequence of humans in the latest observation time window of the current round, the mobile robot can predict the possible position of pedestrians in a short time, thereby adjusting its own navigation path in time to avoid potential collision risks.

[0048] Step 1022, through the state prediction module, based on the historical motion information, the neighbor motion information and the future motion information, the predicted pedestrian state of the target pedestrian is determined; each pedestrian in the observation time window is taken as the target pedestrian, and the predicted pedestrian state of each pedestrian is determined.

[0049] According to the navigation method of the mobile robot provided by the present invention, the state prediction module comprises: a recurrent neural network and a decoder; Based on historical motion information, neighbor motion information, and future motion information, the predicted pedestrian state of the target pedestrian is determined, including: Through the recurrent neural network, the historical motion information, neighbor motion information and future motion information are encoded based on the attention mechanism to obtain the social feature vector of the target pedestrian. The social feature vector is decoded through the decoder to obtain the predicted pedestrian state of the target pedestrian.

[0050] For the state prediction of the mobile robot, it is assumed that the actions of the mobile robot can be perfectly executed and the next state of the mobile robot can be directly calculated from the current state and action.

[0051] For pedestrian state prediction, the embodiment of the present invention designs a pedestrian state predictor based on SocialVAE. The pedestrian state predictor first performs a memory pool or Preprocess the data to obtain the historical motion information in the historical observation , Neighbor movement information And future sports information .

[0052] Among them, SocialVAE (Social Variational AutoEncoder) is a method for pedestrian trajectory prediction. Its core is to use the timewise variational autoencoder architecture combined with stochastic recurrent neural networks for prediction.

[0053] Based on the attention mechanism and recurrent neural network, the above historical motion information, neighbor motion information and future motion information are encoded to extract the motion features of pedestrians, the motion features of surrounding neighbors and the social feature vectors between pedestrians and surrounding neighbors. The encoded social feature vectors are processed by the decoder and finally the predicted pedestrian state is obtained. ,in, It represents the predicted time interval. The predicted pedestrian status includes the pedestrian's position, speed, acceleration, and moving direction, etc., which depends on the application requirements.

[0054] Among them, the social feature vector refers to the feature vector encoded according to the target pedestrian's motion information (i.e., historical motion information and future motion information) and the surrounding neighbor motion information (i.e., neighbor motion information). The social feature vector implicitly describes the relative position and speed relationship between the target pedestrian and the surrounding neighbors.

[0055] Through the embodiments of the present invention, through the attention mechanism and the RNN encoding process, the model can finely extract the motion characteristics of pedestrians and neighbors, as well as the social interaction characteristics between them, which helps the model to more accurately understand the motion patterns and intentions of pedestrians.

[0056] Step 1023, obtaining the robot state of the mobile robot through the value estimation module, and determining the target navigation action of the mobile robot based on the robot state and the predicted pedestrian state of each pedestrian.

[0057] According to the navigation method of the mobile robot provided by the present invention, the above-mentioned value estimation module includes: a relationship graph network and a value network; Based on the robot state and the predicted pedestrian state of each pedestrian, the target navigation action of the mobile robot is determined, including: Through the relationship graph network, the feature matrix and relationship matrix are constructed based on the robot state and the predicted pedestrian state of each pedestrian; The graph convolutional network is called to encode based on the feature matrix and the relationship matrix to obtain the interactive relationship features; Through the value network, the action value of the interactive relationship features is estimated based on the preset action value function, and the action with the highest value in the action space is determined as the target navigation action.

[0058] In an embodiment of the present invention, the value estimation module includes a relationship graph network and a value network.

[0059] The relationship graph is a directed graph. The nodes in the graph represent the states of the agents, and the directed edges between the nodes represent the degree of attention of the agents to another agent. The agents include robots and pedestrians. In the relationship graph network, the robot state and pedestrian state are first encoded into state feature vectors of equal length through different multi-layer perceptrons, and the state feature vectors are combined together to form a feature matrix. The relationship vector is obtained based on the similarity between each two feature vectors (which can be measured by the embedded Gaussian function), and all relationship vectors are combined into a relationship matrix. The relationship graph network infers the feature matrix and the relationship matrix based on the current state (multiple predicted pedestrian states and robot states).

[0060] Among them, the robot state is known information that can be obtained in real time, including position, speed, etc. The action is any action selected from the action space. The above action is the speed. According to the current position and speed, the position at the next moment can be calculated, thereby obtaining the robot state at the next moment.

[0061] Then, the graph convolutional network is used to extract the interaction relationship features between robots and humans and between humans (the interaction relationship features are vectors obtained by further encoding the feature matrix and the relationship matrix. They have no clear meaning, but implicitly represent the relative interaction relationship between intelligent agents, that is, the degree of attention between intelligent agents). The value network is a multi-layer perceptron (MLP) that estimates the action value based on the interaction relationship features extracted from the relationship graph. Using the obtained action value function , select the current optimal navigation action from the action space as the target navigation action.

[0062] In some embodiments, a relational graph is constructed through a relational graph network (RGN), and the nodes in the relational graph represent the states of the agents (robots and pedestrians). Each node contains the current state information of the agent, such as position, speed, etc. The directed edges between nodes represent the degree of attention of an agent to another agent. This degree of attention can be quantified by calculating parameters such as relative position, speed difference, predicted trajectory overlap, etc., and used as the weight of the edge.

[0063] Based on the relationship graph, a feature matrix and a relationship matrix are constructed. The feature matrix is ​​used to store the feature vectors of each node, which describe the state information of the intelligent agent; the relationship matrix stores the interaction relationship characteristics between each pair of nodes, which is usually a two-dimensional matrix, in which each element represents the interaction intensity or attention degree between a pair of nodes.

[0064] The feature matrix and relationship matrix are input into the graph convolutional network. Through the graph convolutional network (GCN), the feature matrix is ​​updated using the relationship matrix to capture the interaction relationship between intelligent agents and output the interaction relationship features.

[0065] Through the value network (multi-layer perceptron), high-level feature representations are extracted based on the interactive relationship features through multiple layers of nonlinear transformations (such as ReLU activation function) and linear transformations; the action value function is output, which represents the expected benefits or costs of performing different actions under a given state; according to the action value function, the action with the highest value is selected from the action space as the current optimal navigation action.

[0066] Through the embodiments of the present invention, through the relationship graph network and GCN, the module can accurately capture the interaction relationship between intelligent agents; through the value network, accurate action value estimation can be performed based on the interaction relationship characteristics to select the optimal navigation action for the robot.

[0067] According to the navigation method of the mobile robot provided by the present invention, before inputting the pedestrian position information into the robot navigation model to obtain the target navigation action output by the robot navigation model, the method further includes: The preset robot navigation model is trained based on knowledge distillation to obtain a trained robot navigation model.

[0068] refer to Figure 3 , Figure 3 It is a structural schematic diagram of the knowledge distillation training framework provided by the present invention, which includes input data (Data), a teacher model (Teacher Model) and a student model (Student Model), wherein the teacher model includes: a first encoder (Encoder1), a second encoder (Encoder2) and a decoder (Decoder); the student model includes: a first encoder (Encoder1), a second encoder (Encoder2) and a decoder (Decoder).

[0069] In an embodiment of the present invention, a preset robot navigation model is trained based on knowledge distillation, aiming to optimize a simpler or more compact student model by extracting knowledge from one or more complex and high-performance teacher models, thereby obtaining a trained robot navigation model.

[0070] The embodiments of the present invention can not only improve the performance of the model, but also reduce the complexity and computing requirements of the model, thereby facilitating deployment in practical applications.

[0071] According to the mobile robot navigation method provided by the present invention, the above-mentioned training includes: teacher model training and student model training, and the preset robot navigation model is trained based on knowledge distillation to obtain the trained robot navigation model, including: Based on the preset teacher model, a simulation is run in the target scenario based on the optimal mutual collision avoidance strategy to obtain the target scenario experience data; Run simulation in the source scene based on the preset student model to obtain source scene experience data; Initialize the state prediction module and value estimation module in the preset teacher model based on the target scene experience data; Based on reinforcement learning and supervised learning, the state prediction module and the value estimation module in the preset teacher model after initialization are jointly trained to obtain a trained teacher model; Through the trained teacher model, the state prediction module and value estimation module in the preset student model are initialized based on the source scene experience data; Based on reinforcement learning and supervised learning, the state prediction module and the value estimation module in the preset student model after initialization are jointly trained to obtain a trained student model; The trained student model is used as the trained robot navigation model.

[0072] refer to Figure 3 ,For the learning of the teacher model, the embodiment of the present invention first uses the ,optimal mutual collision avoidance strategy to perform a simulation demonstration in the ,target scene, and then uses the collected experience and data (memory pool) ) to perform simulation learning and initialize the value estimation module and the state prediction module. Then, we use reinforcement learning and supervised learning to jointly train the value estimation module and the state prediction module. The loss function of the value estimation module is the estimated value and the target value The mean square error (MSE) between . The loss function of the state prediction module is Including the mean square error between the predicted state and the actual state and the posterior distribution and the prior distribution The KL divergence between .

[0073] The loss function of the value estimation module can refer to the following formula (1): (1) in, represents the loss function of the value estimation module of the teacher model, represents the mean square error function, represents the value estimation module of the teacher model after initialization, represents the estimated value of the teacher model, Indicates the target value.

[0074] The loss function of the state prediction module can refer to the following formula (2): (2) in, represents the loss function of the state prediction module of the teacher model, represents the mean square error function, represents the state prediction module of the teacher model after initialization, represents the predicted state of the teacher model (state prediction module), Indicates the real state, represents the posterior distribution of the teacher model and the prior distribution of the teacher model The KL divergence between .

[0075] In some embodiments, Optimal Reciprocal Collision Avoidance (ORCA) aims to ensure that multiple agents (such as robots and pedestrians) can move safely and efficiently in a shared space without collisions.

[0076] During the simulation demonstration, the robot collects the interaction experience between the robot and the environment, including information such as status, action, reward, etc., and stores this information in the memory pool. These experiences will be used for subsequent simulation learning and model training.

[0077] The experience in the memory pool is used to initialize the value estimation module, which is used to estimate the value or expected benefit of performing a specific action in a given state. The experience in the memory pool is also used to initialize the state prediction module, which is used to predict the state of the pedestrian.

[0078] The value estimation module is optimized through reinforcement learning algorithms such as Q-learning, Deep Q-Network, Policy Gradients, etc. In this process, the mobile robot tries different actions and updates its value estimation based on the rewards obtained.

[0079] Use the experience in the memory pool to supervise the training of the state prediction module. This usually involves minimizing the difference between the predicted state and the true state, and ensuring that the posterior distribution of the model is consistent with the prior distribution.

[0080] For student module training, the student model is initialized using the teacher model trained in the previous process, and then the value estimation module and state prediction module are trained in the source scene through reinforcement learning and supervised learning. The value estimation module is still updated in a similar way to formula (1), and the loss function is as follows Formula (3). The state prediction module is updated under the guidance of the teacher model, as Figure 3 As shown: (3) in, represents the loss function of the value estimation module of the student model, represents the mean square error function, represents the value estimation module of the student model after initialization, represents the estimated value of the student model (value estimation module), Indicates the target value.

[0081] Specifically, the hard error of the student model is obtained according to formula (4): : (4) in, represents hard error, represents the mean square error function, represents the predicted state of the student model (state prediction module), Indicates the real state, represents the posterior distribution of the student model and the prior distribution of the student model The KL divergence between .

[0082] According to the student model (state prediction module) The predicted state and teacher model (state prediction module) The mean squared error between the predicted states of , and the prior distribution of the student model and the prior distribution of the teacher model The KL divergence between , refer to the following formula (5) for details: (5) in, represents soft error, represents the mean square error function, represents the predicted state of the student model (state prediction module), represents the predicted state of the teacher model (state prediction module), represents the prior distribution of the student model and the prior distribution of the teacher model The KL divergence between .

[0083] The average of the hard error and the soft error is used as the final error for back propagation, as shown in the following formula (6): (6) in, represents the final error, represents hard error, Indicates soft error.

[0084] In some embodiments, KL divergence is used to measure the difference between two probability distributions.

[0085] In the embodiment of the present invention, both the teacher model and the student model are based on the robot navigation model. The teacher model represents a larger and more complex neural network with stronger performance and accuracy, but usually requires higher computing resources. The student model represents a smaller and simpler neural network, the goal is to obtain performance close to that of the teacher model, while having higher operating efficiency and fewer resource requirements.

[0086] The target scenario is a training environment or data set in which the mobile robot needs to navigate so that the mobile robot exhibits target performance.

[0087] For example, the target scene may contain complex elements such as various obstacles, pedestrians, and traffic signals, which require the mobile robot to accurately perceive and judge. The target scene may involve different time periods (such as day and night), weather conditions (such as sunny days and rainy days), and different geographical locations (such as city streets and rural roads). The navigation tasks in the target scene may be challenging, such as obstacle avoidance, path planning, and pedestrian tracking.

[0088] The source scene is the actual operation scene or data set of the mobile robot in the subsequent operation process.

[0089] Through the embodiments of the present invention, while reducing the dependence of the robot navigation model on the scene, the model's ability to understand information in different scenes is fully explored, and the applicability of the model to various scenes is improved. This can effectively improve the performance of the navigation strategy model in different scenes and ensure that the mobile robot can better cope with crowd navigation tasks in different scenes.

[0090] The navigation device of the mobile robot provided by the present invention is described below. The navigation device of the mobile robot described below and the navigation method of the mobile robot described above can be referred to each other.

[0091] refer to Figure 4 , Figure 4 It is a schematic diagram of the module structure of the navigation device of the mobile robot provided by the present invention.

[0092] The acquisition module 401 is used to acquire the pedestrian position information collected by the mobile robot, wherein the pedestrian position information includes: a position information sequence of each pedestrian; The robot navigation module 402 is used to input the pedestrian position information into the robot navigation model to obtain the target navigation action output by the robot navigation model. The robot navigation model includes: a memory pool, a state prediction module and a value estimation module, which includes: Through the memory pool, within the current observation time window, the position information sequence of the target pedestrian before the target time step is used as the historical motion information; the position information sequence of other pedestrians before the target time step is used as the neighbor motion information; and the position information sequence of the target pedestrian after the target time step is used as the future motion information; Through the state prediction module, the predicted pedestrian state of the target pedestrian is determined based on the historical motion information, the neighbor motion information and the future motion information; each pedestrian in the observation time window is taken as the target pedestrian, and the predicted pedestrian state of each pedestrian is determined; The robot state of the mobile robot is obtained through the value estimation module, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

[0093] Specifically, the navigation device of the mobile robot provided by the present invention can implement all the method steps implemented in the navigation method embodiment of the mobile robot, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0094] Figure 5 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the navigation method of the mobile robot, which includes: obtaining pedestrian position information collected by the mobile robot, wherein the pedestrian position information includes: a position information sequence of each pedestrian; inputting the pedestrian position information into the robot navigation model to obtain the target navigation action output by the robot navigation model, and the robot navigation model includes: a memory pool, a state prediction module and a value estimation module, which includes: through the memory pool, within the current observation time window, the position information sequence of the target pedestrian before the target time step is used as historical motion information; the position information sequence of other pedestrians before the target time step is used as neighbor motion information; the position information sequence of the target pedestrian after the target time step is used as future motion information; through the state prediction module, based on the historical motion information, the neighbor motion information and the future motion information, the predicted pedestrian state of the target pedestrian is determined; each pedestrian in the observation time window is used as a target pedestrian, and the predicted pedestrian state of each pedestrian is determined; through the value estimation module, the robot state of the mobile robot is obtained, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

[0095] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the mobile robot navigation method provided by the above methods, the method comprising: obtaining pedestrian position information collected by the mobile robot, wherein the pedestrian position information comprises: a position information sequence of each pedestrian; inputting the pedestrian position information into a robot navigation model to obtain a target navigation action output by the robot navigation model, the robot navigation model comprising: a memory pool, a state prediction module and a value estimation module, wherein, including: through the memory pool, in the current observation In the measurement time window, the position information sequence of the target pedestrian before the target time step is used as the historical motion information; the position information sequence of other pedestrians before the target time step is used as the neighbor motion information; the position information sequence of the target pedestrian after the target time step is used as the future motion information; through the state prediction module, the predicted pedestrian state of the target pedestrian is determined based on the historical motion information, the neighbor motion information and the future motion information; each pedestrian in the observation time window is taken as the target pedestrian, and the predicted pedestrian state of each pedestrian is determined; through the value estimation module, the robot state of the mobile robot is obtained, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

[0097] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the navigation method of the mobile robot provided by the above-mentioned methods, the method comprising: obtaining pedestrian position information collected by the mobile robot, wherein the pedestrian position information comprises: a position information sequence of each pedestrian; inputting the pedestrian position information into a robot navigation model to obtain a target navigation action output by the robot navigation model, the robot navigation model comprising: a memory pool, a state prediction module and a value estimation module, wherein, including: through the memory pool, within the current observation time window, the target time step is The position information sequence of the target pedestrian before the target time step is used as the historical motion information; the position information sequence of other pedestrians before the target time step is used as the neighbor motion information; the position information sequence of the target pedestrian after the target time step is used as the future motion information; through the state prediction module, the predicted pedestrian state of the target pedestrian is determined based on the historical motion information, the neighbor motion information and the future motion information; each pedestrian in the observation time window is used as the target pedestrian, and the predicted pedestrian state of each pedestrian is determined; through the value estimation module, the robot state of the mobile robot is obtained, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A navigation method for a mobile robot, characterized in that: include: Acquire pedestrian position information collected by the mobile robot, wherein the pedestrian position information includes: a position information sequence of each pedestrian; The pedestrian position information is input into a robot navigation model to obtain a target navigation action output by the robot navigation model, wherein the robot navigation model includes: a memory pool, a state prediction module and a value estimation module, wherein: Through the memory pool, within the current observation time window, the position information sequence of the target pedestrian before the target time step is used as the historical motion information; the position information sequence of other pedestrians before the target time step is used as the neighbor motion information; and the position information sequence of the target pedestrian after the target time step is used as the future motion information; Determine the predicted pedestrian state of the target pedestrian based on the historical motion information, the neighbor motion information and the future motion information through the state prediction module; take each pedestrian in the observation time window as the target pedestrian and determine the predicted pedestrian state of each pedestrian; The robot state of the mobile robot is acquired through the value estimation module, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

2. The navigation method of a mobile robot according to claim 1, characterized in that: The memory pool includes: a first memory pool, a second memory pool and a third memory pool; The first memory pool is used to store the robot state and environment reward of the robot navigation model at each time step during the training process; The second memory pool is used to store the position information sequence of each pedestrian in the historical observation time window; The third memory pool is used to store the position information sequence of each pedestrian in the current observation time window.

3. The navigation method of a mobile robot according to claim 1, characterized in that: The state prediction module includes: a recurrent neural network and a decoder; Determining the predicted pedestrian state of the target pedestrian based on the historical motion information, the neighbor motion information, and the future motion information includes: By means of the recurrent neural network, the historical motion information, the neighbor motion information and the future motion information are encoded based on an attention mechanism to obtain a social feature vector of the target pedestrian; The social feature vector is decoded by the decoder to obtain the predicted pedestrian state of the target pedestrian.

4. The navigation method of a mobile robot according to claim 1, characterized in that: The value estimation module includes: a relationship graph network and a value network; The determining of the target navigation action of the mobile robot based on the robot state and the predicted pedestrian state of each pedestrian includes: Constructing a feature matrix and a relationship matrix based on the robot state and the predicted pedestrian state of each pedestrian through the relationship graph network; Calling a graph convolutional network to encode based on the feature matrix and the relationship matrix to obtain interactive relationship features; Through the value network, the action value of the interactive relationship feature is estimated based on a preset action value function, and the action with the highest value in the action space is determined as the target navigation action.

5. The navigation method of a mobile robot according to claim 1, characterized in that: Before inputting the pedestrian position information into the robot navigation model to obtain the target navigation action output by the robot navigation model, the method further includes: The preset robot navigation model is trained based on knowledge distillation to obtain a trained robot navigation model.

6. The navigation method of the mobile robot according to claim 5, characterized in that: The training includes: teacher model training and student model training. The preset robot navigation model is trained based on knowledge distillation to obtain a trained robot navigation model, including: Based on the preset teacher model, a simulation is run in the target scenario based on the optimal mutual collision avoidance strategy to obtain the target scenario experience data; Run simulation in the source scene based on the preset student model to obtain source scene experience data; Initializing the state prediction module and the value estimation module in the preset teacher model based on the target scene experience data; Based on reinforcement learning and supervised learning, the state prediction module and the value estimation module in the preset teacher model after initialization are jointly trained to obtain a trained teacher model; Initializing the state prediction module and the value estimation module in the preset student model based on the source scene experience data through the trained teacher model; Based on reinforcement learning and supervised learning, the state prediction module and the value estimation module in the preset student model after initialization are jointly trained to obtain a trained student model; The trained student model is used as a trained robot navigation model.

7. A navigation device for a mobile robot, characterized in that: include: An acquisition module, used to acquire pedestrian position information collected by a mobile robot, wherein the pedestrian position information includes: a position information sequence of each pedestrian; A robot navigation module is used to input the pedestrian position information into a robot navigation model to obtain a target navigation action output by the robot navigation model, wherein the robot navigation model includes: a memory pool, a state prediction module and a value estimation module, wherein: Through the memory pool, within the current observation time window, the position information sequence of the target pedestrian before the target time step is used as the historical motion information; the position information sequence of other pedestrians before the target time step is used as the neighbor motion information; and the position information sequence of the target pedestrian after the target time step is used as the future motion information; Determine the predicted pedestrian state of the target pedestrian based on the historical motion information, the neighbor motion information and the future motion information through the state prediction module; take each pedestrian in the observation time window as the target pedestrian and determine the predicted pedestrian state of each pedestrian; The robot state of the mobile robot is acquired through the value estimation module, and the target navigation action of the mobile robot is determined based on the robot state and the predicted pedestrian state of each pedestrian.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the navigation method of the mobile robot according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the navigation method of the mobile robot as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the navigation method of the mobile robot as claimed in any one of claims 1 to 6 is implemented.

Images