A robot social self-adaptive navigation knowledge learning and migration method and system

Abstract

The application provides a robot social self-adaptive navigation knowledge learning and migration method and system, and relates to the field of mobile robot navigation. In view of the problems that the existing path planner lacks time memory and ignores the social intention of pedestrians, a human-machine integration scene is constructed, a training set containing an expert demonstration path is made, and a recursive generation model is input; a recursive neural network embedded RRT is designed to generate an RNN-RRT planner, and historical information and pedestrian convergence probability are fused during training; knowledge migration is realized by loading training parameters for fine-tuning in a new scene, and the final parameters of the RNN are output when the loss converges or a preset number of rounds is reached. The method improves the humanization and generalization ability of the path, and is suitable for complex human-machine interaction scenes.

Description

Technical Field

[0001] This invention relates to the field of mobile robot navigation technology, and in particular to a method and system for robot social adaptive navigation knowledge learning and transfer. Background Technology

[0002] With the widespread application of service robots in dynamic human-machine environments, social adaptive navigation technology has become a research hotspot. This technology integrates principles of robotics and behavioral psychology, enabling robots to learn human navigational behavioral characteristics (such as avoidance rules and maintaining social distance) and transfer knowledge in new scenarios. Current mainstream solutions employ inverse reinforcement learning frameworks, such as the Neural Network Rapidly-exploring RandomTrees (NN-RRT) path planner: first, expert demonstration paths are collected and environmental features are extracted; these features are then input into a fully connected neural network to calculate node costs; iterative optimization is used to make the planned path approximate human behavioral patterns. Furthermore, some studies have introduced generative adversarial networks (GANs) to accelerate knowledge transfer efficiency using game theory mechanisms.

[0003] However, existing technologies have significant limitations: First, fully connected neural networks only process current scene features (such as real-time pedestrian positions and distances to static obstacles), lacking the ability to remember historical dynamic information. This prevents planners from simulating human predictive behavior based on time-series information (such as predicting intersection points based on pedestrian trajectories and avoiding them in advance), easily leading to conflict paths in dynamic scenarios. Second, existing methods simplify pedestrians as moving obstacles, ignoring their social intentions. For example, key features such as the probability of pedestrian convergence are not quantified, causing the planned path to potentially cross areas where people will converge in the future (e.g., Figure 1 (As shown), it interferes with normal social activities. The aforementioned defects limit the robot's human-like navigation and safety in complex human-machine environments.

[0004] Therefore, there is an urgent need for a navigation knowledge learning and transfer method that can integrate historical scene information with pedestrian social intent prediction to solve the problems of temporal memory loss and insufficient behavior modeling in existing path planners. Summary of the Invention

[0005] To address this, embodiments of the present invention provide a method and system for robot social adaptive navigation knowledge learning and transfer, which solves the problems of temporal memory loss and neglect of pedestrian social intent in existing path planners.

[0006] To address the aforementioned problems, embodiments of the present invention provide a method for robot social adaptive navigation knowledge learning and transfer, the method comprising:

[0007] Step S1: Construct multiple human-machine collaborative navigation knowledge learning scenarios;

[0008] Step S2: Based on the human-machine collaborative navigation knowledge learning scenario, create a training set containing multiple expert demonstration paths and corresponding scenario information;

[0009] Step S3: Input the training set into the recursive generative model framework;

[0010] Step S4: Design a recurrent neural network structure and embed the recurrent neural network into the cost function module of the RRT path planner to generate an RNN-RRT path planner;

[0011] Step S5: Using the multiple human-machine collaborative navigation knowledge learning scenarios to form a training loop, start the training of the RNN-RRT path planner. During the training process, calculate the social adaptive navigation feature value of the expert demonstration path to generate social adaptive navigation information for the corresponding scenario, and learn the navigation features of the expert demonstration path based on the recurrent neural network.

[0012] Step S6: In the new human-machine collaborative navigation knowledge learning scenario, load the network parameters of the trained RNN-RRT path planner as the navigation behavior meta-model, and repeat the training process of step S5 to achieve the transfer learning of social adaptive navigation knowledge.

[0013] Step S7: When the loss value of the recurrent neural network converges or reaches the preset number of training rounds, terminate the training and output the final parameters of the recurrent neural network.

[0014] Preferably, in step S2, the process of creating the training set includes:

[0015] In each of the aforementioned human-machine collaborative navigation knowledge learning scenarios, the robot's starting point and target point are set;

[0016] The expert demonstration path is generated by controlling the robot to move from the starting point to the target point along a path that conforms to human navigation behavior habits through a remote control device.

[0017] The robot uses a lidar mounted on it to collect scene data in real time. The scene data includes at least the robot's real-time coordinates, pedestrian coordinates and direction of movement, and obstacle coordinates.

[0018] The scene data is stored using the rosbag tool of the ROS system, and the scene data is associated with the corresponding expert demonstration paths to form the training set.

[0019] Preferably, in step S4, the structure of the recurrent neural network satisfies:

[0020] The recurrent neural network includes an input layer, at least one hidden layer, and an output layer;

[0021] The number of neurons in the input layer is consistent with the dimension of the social adaptive navigation feature value, and is used to receive the social adaptive navigation feature value.

[0022] The hidden layer contains a preset number of neurons, which associate historical navigation feature information with the current social adaptive navigation feature value through the hidden state, thereby realizing the memory and utilization of historical navigation information;

[0023] The output layer has one neuron, and the sigmoid activation function is used to map the output node value to a value range of 0-1.

[0024] Preferably, the social adaptive navigation feature values ​​constitute a 6-dimensional feature vector. The feature components are defined as follows:

[0025] This represents the Euclidean distance from the current path node to the target point.

[0026] The distance feature from the current path node to the nearest obstacle is calculated using a function constructed with preset adjustment parameters and regularization parameters;

[0027] , , The cost features for the pedestrian's front, rear, and right sides are respectively calculated using a Gaussian function based on distance and angle.

[0028] The convergence probability feature of the current path node being in the future intersection area of ​​pedestrians is obtained by traversing the real-time coordinates and directions of pedestrians to determine the pedestrian intersection point, and then calculating the distance from the current path node to the pedestrian intersection point.

[0029] Preferably, in step S4, generating the RNN-RRT path planner further includes a pre-training step of the recurrent neural network:

[0030] Multiple sets of feature vectors corresponding to the social adaptive navigation feature values ​​are randomly generated and used as pre-training input data;

[0031] The pre-trained input data is input into a preset fully connected neural network to calculate the reference cost.

[0032] The recurrent neural network is pre-trained using the pre-training input data as input and the reference cost value as the target output until the weights of the recurrent neural network converge.

[0033] Preferably, in step S5, the training process of the RNN-RRT path planner further includes a parameter iterative update step:

[0034] The planned path generated by the RNN-RRT path planner is discretized into multiple path nodes, and the social adaptive navigation feature value corresponding to each path node is calculated.

[0035] Align the social adaptive navigation feature values ​​of the path nodes with the node values ​​of the expert demonstration path using linear interpolation.

[0036] The loss value between the predicted cost value of the planned path node and the cost value of the expert demonstration path node is calculated using the mean square error loss function.

[0037] The gradient is calculated through backpropagation, and the parameters of the recurrent neural network are updated using the Adam optimizer.

[0038] Preferably, in step S6, the transfer learning of the social adaptive navigation knowledge specifically includes:

[0039] The parameters of the trained recurrent neural network are used as navigation prior knowledge and loaded into the RNN-RRT path planner in the new scene;

[0040] The RNN-RRT path planner generates an initial planned path in the new scenario based on the navigation prior knowledge.

[0041] By calculating the cost difference between the initial planned path and the expert demonstration path in the new scenario, the parameters of the recurrent neural network are fine-tuned to optimize the initial planned path and complete the transfer of social adaptive navigation knowledge.

[0042] This invention also provides a robot social adaptive navigation knowledge learning and transfer system, which is used to implement the robot social adaptive navigation knowledge learning and transfer method described above, specifically including:

[0043] The scenario building module is used to construct multiple human-machine collaborative navigation knowledge learning scenarios;

[0044] The training set acquisition module is used to create a training set containing multiple expert demonstration paths and corresponding scenario information based on the human-machine collaborative navigation knowledge learning scenario.

[0045] A recursive generation model module is used to input the training set into the recursive generation model framework;

[0046] The RNN-RRT planner construction module is used to design a recurrent neural network structure and embed the recurrent neural network into the cost function module of the RRT path planner to generate the RNN-RRT path planner.

[0047] The training optimization module is used to start the training of the RNN-RRT path planner by forming a training loop with the multiple human-machine collaborative navigation knowledge learning scenarios. During the training process, the social adaptive navigation feature value of the expert demonstration path is calculated to generate social adaptive navigation information for the corresponding scenario, and the navigation features of the expert demonstration path are learned based on the recurrent neural network.

[0048] The knowledge transfer module is used to load the network parameters of the trained RNN-RRT path planner as a navigation behavior meta-model in a new human-machine collaborative navigation knowledge learning scenario, and repeat the training process of the optimization module to achieve the transfer learning of social adaptive navigation knowledge.

[0049] The parameter output module is used to terminate training and output the final parameters of the recurrent neural network when the loss value of the recurrent neural network converges or reaches a preset number of training rounds.

[0050] This invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to realize the robot social adaptive navigation knowledge learning and transfer method described above.

[0051] This invention also provides a computer storage medium storing a computer software product, the computer software product including several instructions to cause a computer device to execute the robot social adaptive navigation knowledge learning and transfer method described above.

[0052] As can be seen from the above technical solutions, this invention application has the following beneficial effects:

[0053] (1) The present invention replaces the fully connected neural network of the original NN-RRT planner with a recurrent neural network (RNN). The RNN can associate historical navigation information with current scene features through hidden states, so that the planner can use the scene dynamic data of past moments (such as the past movement trajectory of pedestrians and the historical position of obstacles) to simulate human prediction behavior based on time-series information, avoid dynamic scene conflicts caused by lack of historical information, and improve the rationality of path planning.

[0054] (2) Compared with the existing technology that simplifies pedestrians as dynamic obstacles, the present invention designs a 6-dimensional feature vector, predicts the intersection area by traversing the pedestrian coordinates and the direction of travel, calculates the probability that the path node is in the area, so that the planned path avoids the future intersection area of ​​pedestrians, does not interfere with the normal social activities of pedestrians, conforms to social norms, and improves the social adaptability of navigation.

[0055] (3) The trained RNN parameters are loaded into the new scenario as the navigation behavior meta-model. The initial path is generated with this prior knowledge. Only the parameters need to be fine-tuned to adapt to the new scenario, without the need to train from scratch. For example, the knowledge of 60 training scenarios in the embodiment can be efficiently transferred to 15 verification scenarios, and the feature difference between the path and the demonstration path is lower than the allowable error, which greatly improves the generalization ability of the planner in complex human-machine collaborative scenarios and reduces the adaptation cost of new scenarios. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Referring to the drawings will make the features and advantages of the present invention clearer. The drawings are illustrative and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0057] Figure 1 This is a schematic diagram illustrating the limitations of existing path planning algorithms in the background art of this invention;

[0058] Figure 2 A flowchart of a robot social adaptive navigation knowledge learning and transfer method provided by the present invention;

[0059] Figure 3 The above is an Rviz simulation map of the human-machine collaborative scenario in this invention, where (a) is a scenario containing only static obstacles and robot starting / target points, and (b) is a complex interaction scenario after adding pedestrians;

[0060] Figure 4 This is a demonstration of the remote control expert's path process in this invention;

[0061] Figure 5 This is a diagram of the recurrent neural network structure in the RNN-RRT path planner of this invention;

[0062] Figure 6 This is a schematic diagram of the social adaptive navigation knowledge extraction process in this invention;

[0063] Figure 7 This is a schematic diagram of the path planned by the RNN-RRT path planner for a scene after initialization in this invention;

[0064] Figure 8 This is a schematic diagram of the path planned by the RNN-RRT path planner in this invention after completing navigation knowledge learning and transfer.

[0065] Figure 9 This invention provides a block diagram of a robot social adaptive navigation knowledge learning and transfer system. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1: To address the issues of temporal memory loss and neglect of pedestrian social intent in existing path planners. For example... Figure 2 As shown, this invention proposes a method for robot social adaptive navigation knowledge learning and transfer, which includes:

[0068] Step S1: Construct multiple human-machine collaborative navigation knowledge learning scenarios;

[0069] Step S2: Based on the human-machine collaborative navigation knowledge learning scenario, create a training set containing multiple expert demonstration paths and corresponding scenario information;

[0070] Step S3: Input the training set into the recursive generative model framework;

[0071] Step S4: Design a recurrent neural network structure and embed the recurrent neural network into the cost function module of the RRT path planner to generate the RNN-RRT path planner;

[0072] Step S5: The training loop is formed by multiple human-machine collaborative navigation knowledge learning scenarios. The training of the RNN-RRT path planner is started. During the training process, the social adaptive navigation feature value of the expert demonstration path is calculated to generate the social adaptive navigation information of the corresponding scenario. The navigation features of the expert demonstration path are learned based on the recurrent neural network.

[0073] Step S6: In the new human-machine collaborative navigation knowledge learning scenario, load the network parameters of the trained RNN-RRT path planner as the navigation behavior meta-model, and repeat the training process of step S5 to achieve the transfer learning of social adaptive navigation knowledge.

[0074] Step S7: When the loss value of the recurrent neural network converges or reaches the preset number of training rounds, terminate the training and output the final parameters of the recurrent neural network.

[0075] As can be seen from the above technical solution, this invention proposes a method for robot social adaptive navigation knowledge learning and transfer. First, step S1 constructs multiple human-machine collaborative navigation knowledge learning scenarios, providing a diverse interactive environment covering static obstacles and dynamic pedestrians for subsequent model training, laying the foundation for these scenarios. Next, step S2 creates a training set based on these scenarios, containing multiple expert demonstration paths and corresponding scenario information, providing reference data that conforms to human navigation habits and providing high-quality input for model learning. Then, step S3 recursively generates a model framework by inputting the training set, enabling the framework to acquire the scenario and path data required for learning and initiating the navigation knowledge learning process. Step S4 designs a recurrent neural network structure and embeds the cost function module of the RRT path planner to generate an RNN-RRT path planner, solving the deficiency of traditional planners lacking temporal memory and enabling the planner to associate with historical navigation information. Step S5 trains the planner using multiple scenarios in a training loop, calculating the social adaptive navigation feature values ​​(including pedestrian convergence probability, etc.) of the expert demonstration paths and learning navigation features based on the recurrent neural network, enabling the planner to master social adaptive navigation knowledge that integrates historical information and pedestrian social intentions. Step S6… In the new scenario, the trained planner network parameters are loaded as the navigation behavior meta-model, and repeated training is carried out to achieve knowledge transfer, avoid training from scratch, and improve the adaptation efficiency of the new scenario. Finally, in step S7, the training is terminated and the final parameters are output when the recurrent neural network loss value converges or reaches the preset number of training rounds, ensuring that the planner parameters are stable and can generate human-like navigation paths with high generalization ability in complex human-computer interaction scenarios.

[0076] The robot social adaptive navigation knowledge learning and transfer method in this embodiment is based on the construction of RNN-RRT (Recurrent Neural Network Rapidly-exploring RandomTrees) path planner using a recurrent neural network. Through the process of "scene construction - training set preparation - model design - training optimization - knowledge transfer - parameter output", it realizes efficient learning and cross-scene transfer of navigation knowledge.

[0077] In step S1, multiple human-machine collaborative navigation knowledge learning scenarios are constructed. This embodiment uses a two-dimensional grid map as the basis for scenario construction and employs the RVIZ visualization tool of ROS (Robot Operating System) to display the scenarios. The specific steps are as follows:

[0078] Set map size: Based on common indoor / outdoor human-computer interaction scenarios, set the length and width of the grid map to ensure that the scene covers the complete interactive environment of "obstacles-pedestrians-robot movement space";

[0079] Add static obstacles: Simulate a real scene on the map by adding static obstacles such as walls, pillars, tables and chairs, and record the grid coordinate range of each obstacle to ensure that the position of the obstacles is consistent with the real environment;

[0080] Configure pedestrian information: Based on the static scene, add pedestrians with social interaction intentions (such as 2-5 pedestrians), and set the initial coordinates, movement direction and pedestrian space size of the pedestrians;

[0081] Scene visualization verification: The scene map is loaded using the RVIZ tool to display the real-time positions and movement directions of static obstacles and pedestrians, ensuring the scene meets the interactive requirements of "human-computer co-operation" (e.g., ...). Figure 3 As shown, Figure 3 (a) is a scenario containing only static obstacles and the robot's starting / target point. Figure 3 (b) is a complex interaction scenario after adding pedestrians.

[0082] In step S2, based on the human-machine collaborative navigation knowledge learning scenario of step S1, a training set containing multiple expert demonstration paths and corresponding scenario information is created, including:

[0083] Within each human-robot collaborative navigation knowledge learning scenario, a starting point and a target point for the robot are set. Specifically, in each scenario constructed in step 2, one robot starting point and one target point are set to ensure that the starting point and target point are not obstructed by obstacles and cover different areas of the scenario.

[0084] The robot is controlled remotely to move from the starting point to the target point along a path that conforms to human navigation habits, generating an expert demonstration path. Specifically, a mobile robot equipped with a 360-degree LiDAR is remotely controlled via a ROS handle to move from the starting point to the target point along a smooth path "conforming to human navigation habits" (such as avoiding pedestrian intersections and maintaining a safe distance from obstacles). The process is as follows: Figure 4 As shown.

[0085] The robot's onboard LiDAR collects scene data in real time. This scene data includes at least the robot's real-time coordinates, pedestrian coordinates and direction of movement, and obstacle coordinates. Specifically, scene data is recorded using ROS's rosbag tool to store all data generated during the robot's movement in real time, including:

[0086] Robot real-time coordinates (sampled 10 times per second, recording x / y axis coordinates);

[0087] Pedestrian dynamic information (real-time coordinates and changes in direction of movement of pedestrians);

[0088] Static obstacle information (obstacle coordinate range, used for subsequent collision detection);

[0089] Timestamp information (linking the temporal relationships of various data, providing a foundation for the temporal learning of RNNs).

[0090] The rosbag tool of the ROS system was used to store the scene data, and the scene data was associated with the corresponding expert demonstration paths to form a training set. Specifically, a total of 75 human-computer collaborative scenarios were constructed, of which 60 sets were used as the training set (for parameter training of the RNN-RRT path planner) and 15 sets were used as the validation set (for verifying the knowledge transfer effect). All data were stored in rosbag format to ensure data integrity and reproducibility.

[0091] In this embodiment, in step S3, the training set from step S2 is input into the recursive generative model framework. The recursive generative model framework of this invention uses "RNN as the core + social adaptive features as input" to achieve the fusion of historical navigation information and current scene features. The specific construction process is as follows:

[0092] Training set input processing: The 75 scene rosbag packages prepared in step 3 are used as the frame input. The starting point-target point coordinates, pedestrian dynamic data, and obstacle coordinates of each scene are extracted and transformed into structured data that can be processed by RNN.

[0093] Social adaptive feature vector definition: For each path node in the map, a 6-dimensional feature vector is defined. , with nodes For example, the calculation methods for each feature are as follows:

[0094] : Current path node To the target point The Euclidean distance reflects the goal orientation of the path;

[0095] : Current path node To the nearest obstacle The distance features, where For regularization parameters, , To adjust the parameters and ensure that the impact of obstacle distance on path cost is reasonable;

[0096] Cost characteristics in the direction ahead of the pedestrian. For the current path node To the The distance between pedestrians For the current path node With the The angle between the pedestrians This is a cost calculation function based on the Gaussian function;

[0097] Cost characteristics of pedestrians in the rear direction, calculation logic and They are identical, differing only in the range of direction and angle.

[0098] Cost characteristics of pedestrians on the right, with a direction angle range of 90°±30°, and cost calculation logic is the same as... Consistent;

[0099] : Current path node The convergence probability characteristics of pedestrians in future intersection areas are determined by first traversing the real-time coordinates and directions of movement of all pedestrians, calculating the coordinates of the pedestrian intersection point, and then using the distance from the node to the intersection point. Using the exponential function weight normalization method as input, the convergence probability is calculated to address the shortcomings of existing technologies that ignore pedestrian social intentions.

[0100] In step S4, the recurrent neural network structure is designed, and its structure is as follows: Figure 5 As shown, the RNN structure includes an input layer, a hidden layer, and an output layer, with the following specific parameters:

[0101] Input layer: 6 neurons, corresponding to a 6-dimensional feature vector It receives the social adaptive features of each node;

[0102] Hidden layer: 10 neurons, using the tanh activation function, through hidden states Related historical information ( ),in The hidden layer weight matrix is... For the hidden layer bias, The input layer weight matrix, For input layer bias, (Using the tanh function) to enable memorized navigation knowledge learning;

[0103] Output layer: 1 neuron, using the sigmoid activation function, output node cost ( For output layer weights, (for output layer bias), the cost is mapped to the 0-1 range to facilitate cost comparison;

[0104] Time step: Set to 1 (expandable to 3-5 steps for learning historical information over longer time periods) to ensure a balance between parameter update efficiency and memory effectiveness.

[0105] Furthermore, this invention generates an RNN-RRT planner through "RNN pre-training + fusion with RRT," solving the problem that the original linear cost function of RRT "cannot learn complex navigation knowledge." The specific steps are as follows:

[0106] Determine the RNN structure parameters: Based on the above structure design, fix the structure of the input layer (6 neurons), hidden layer (10 neurons), and output layer (1 neuron), and use the Xavier initialization method to generate the initial weight matrix. , , ) and bias ( , , This ensures that the initial parameters are evenly distributed.

[0107] RNN pre-training: To accelerate the convergence speed of subsequent training, pre-training is performed using the output of the existing fully connected neural network in NN-RRT as the target.

[0108] Multiple sets of feature vectors corresponding to social adaptive navigation feature values ​​are randomly generated and used as pre-training input data. Specifically, 1000 sets of 6-dimensional feature vectors are randomly generated. (Reasonable range of values ​​for coverage features).

[0109] The pre-trained input data is fed into a pre-defined fully connected neural network to calculate the reference value. Specifically, the feature vector is input into the NN-RRT fully connected neural network to calculate the reference value. (The formula is) ), For the weights of the fully connected layer, (for bias)

[0110] Using pre-training input data as input and reference cost value as the target output, the recurrent neural network is pre-trained until the weights of the recurrent neural network converge. Specifically, using randomly generated... For input, With the target output as the objective, train the RNN until the weights converge. At this point, the RNN can approximately simulate the cost computation capability of a fully connected network.

[0111] Fusion of RRT and pre-trained RNN: A pre-trained RNN is embedded into the cost function module of the RRT path planner, replacing the original linear cost function. When selecting a new node, RRT calculates the node cost using the RNN, while also incorporating the historical hidden states. This generates an RNN-RRT path planner with memory.

[0112] In step S5, a training loop is formed by multiple human-machine collaborative navigation knowledge learning scenarios to start the training of the RNN-RRT path planner. During the training process, the social adaptive navigation feature value of the expert demonstration path is calculated to generate the social adaptive navigation information of the corresponding scenario, and the navigation features of the expert demonstration path are learned based on the recurrent neural network.

[0113] Specifically, using 60 training scenarios as a loop, the RNN-RRT path planner learns social adaptive navigation knowledge through an iterative process of path generation, knowledge learning, and parameter updating. The specific steps are as follows:

[0114] Start the training cycle: Set 60 training scenarios as 1 training cycle, initialize the parameters of the RNN-RRT path planner (load pre-trained weights), and enter the single-scenario training process;

[0115] Calculate the feature value of the demonstration path: Traverse all nodes of the expert demonstration path in the current scene, and for each node... Calculate 6-dimensional feature vectors Inputting the pre-trained RNN yields node generation values; summing all node generation values ​​generates social adaptive navigation information for the current scene (e.g., ...). Figure 6 As shown, this is a visualization of the cost values ​​of a demonstration path node.

[0116] Furthermore, the RNN-RRT path planner of the present invention is generated through the following steps. Article (in this embodiment) ) Path planning:

[0117] (1) Random sampling of growth points: One candidate growth point is randomly generated within the map area to ensure that the sampling covers all areas of the scene;

[0118] (2) Collision detection: Determine whether the growth point is located in the space occupied by obstacles or pedestrians. If a collision occurs, resample; otherwise, proceed to the next step.

[0119] (3) Calculate the cost of the growth point: Extract the 6-dimensional feature vector of the growth point. Input an RNN, and combine it with the hidden state from the previous time step. Calculate the cost value (ensuring that the cost value incorporates historical information);

[0120] (4) Selecting a parent node and rewiring: Calculate the "cumulative cost value" of the growth point and the surrounding existing nodes (current node cost value + cumulative cost value from the parent node to the starting point), and select the node with the lowest cumulative cost value as the parent node; at the same time, check whether the surrounding nodes can reduce the cumulative cost value through the current growth point. If so, update the parent node (rewiring operation).

[0121] (5) Determine the endpoint: Calculate the distance between the growth point and the target point. If the distance is less than 0.5m (threshold), it is considered that the endpoint has been reached and the path generation is complete; otherwise, return to (1) to continue expanding.

[0122] The generated initial planned path is as follows Figure 7As shown (the white path is the planned path, and the black path is the expert demonstration path; it can be seen that the initial path did not avoid pedestrian gathering areas).

[0123] Furthermore, this invention learns the navigation features of the demonstration path and aligns the planned path with the demonstration path through RNN training, including the following steps:

[0124] Discrete Path Planning: Discretize the generated planned path into points with a similar number of nodes as the demonstration path (e.g., if the demonstration path has 50 nodes, the planned path is discretized into 50 points), and calculate the feature vector of each discrete point. ;

[0125] Discrete demonstration path: Discretize the expert demonstration path into the same number of points and calculate the feature vector. And input the target value into the RNN. ;

[0126] Data alignment: Due to the incomplete matching of node positions between the planned path and the demonstration path, linear interpolation is used for alignment. and (For example, interpolation based on Euclidean distance of node coordinates to ensure continuity of space and cost).

[0127] RNN training: For input, With the goal of training an RNN, the predicted cost of planning path nodes is increased. Approaching This allows them to learn social adaptive navigation knowledge from the demonstration path (such as avoiding pedestrian congestion areas and maintaining a safe distance).

[0128] Furthermore, the RNN weights are optimized through the following steps:

[0129] Data preprocessing: For all scenarios in the current loop and Perform linear interpolation alignment to obtain the training dataset;

[0130] Forward propagation: after alignment Input an RNN to obtain the predicted cost of each node. ;

[0131] Calculate the loss value: Use PyTorch's built-in mean squared error (MSE) loss function;

[0132] Backpropagation: Calculates the gradient of each layer's weights based on the loss value, and uses gradient clipping to avoid gradient explosion;

[0133] Parameter optimization: Update the weights and biases of the RNN using the Adam optimizer;

[0134] Cyclic iteration judgment: After completing one training cycle, if the MSE loss value of the RNN is below 0.005 for three consecutive cycles (convergence condition) or the number of training cycles reaches 50 (maximum number of cycles), then the knowledge transfer stage is entered; otherwise, the next round of training is started.

[0135] In step S6, in the new human-machine collaborative navigation knowledge learning scenario, the network parameters of the trained RNN-RRT path planner are loaded as the navigation behavior meta-model, and the training process of step S5 is repeated to achieve the transfer learning of social adaptive navigation knowledge.

[0136] Specifically, for 15 sets of verification scenarios (new scenarios), navigation knowledge transfer is achieved by loading prior knowledge and fine-tuning parameters. The specific steps are as follows:

[0137] (1) Loading the navigation behavior meta-model: The RNN parameters after convergence in the training rounds are used as the navigation behavior meta-model and loaded into the RNN-RRT planner. At this time, the planner has learned social adaptive navigation knowledge from 60 training scenarios.

[0138] (2) New scenario initialization: In the verification scenario, the social adaptive navigation information of the demonstration path is recalculated as the target of transfer learning;

[0139] (3) Generate initial migration path: RNN-RRT directly generates the planning path of the new scene based on the loaded meta-model parameters. At this time, the path has a certain anthropomorphic ability (such as avoiding static obstacles).

[0140] (4) Parameter fine-tuning: In response to the differences between the planned path and the demonstration path in the new scenario (such as still being close to the pedestrian gathering area), the RNN parameters are fine-tuned by training with a small batch of data (using only 10% of the data in the new scenario) to learn the personalized navigation knowledge of the new scenario (such as avoidance strategies under different pedestrian densities).

[0141] (5) Recurring transfer verification: Repeat step S5 to complete one round of transfer training for 15 sets of verification scenarios, and determine whether the path meets the anthropomorphism requirements (such as the feature difference between the planned path and the demonstration path). If the value is below 0.05, the migration is successful; otherwise, continue fine-tuning.

[0142] Furthermore, when the transfer learning of all 15 verification scenarios meets the requirement (feature difference E < 0.05), and the RNN loss value converges stably (loss fluctuation less than 0.0001 for 5 consecutive rounds), all training is terminated, and the final parameters of the RNN are output (including...). , , Weight matrix and , , (Bias). At this point, the RNN-RRT path planner can generate human-like paths that take into account both historical information and pedestrian social intentions in new human-machine collaborative scenarios, such as... Figure 8 As shown (the trained path avoids pedestrian converging areas, and its consistency with the demonstration path is significantly improved).

[0143] Example 2: Figure 9 As shown, this invention provides a robot social adaptive navigation knowledge learning and transfer system. This system is used to implement the robot social adaptive navigation knowledge learning and transfer method of Embodiment 1 above, specifically including:

[0144] Scene building module 100 is used to build multiple human-machine collaborative navigation knowledge learning scenarios;

[0145] The training set acquisition module 200 is used to create a training set containing multiple expert demonstration paths and corresponding scenario information based on human-machine collaborative navigation knowledge learning scenarios.

[0146] The recursive generative model module 300 is used to input the training set into the recursive generative model framework;

[0147] RNN-RRT planner building module 400 is used to design the recurrent neural network structure and embed the recurrent neural network into the cost function module of the RRT path planner to generate the RNN-RRT path planner.

[0148] The training optimization module 500 is used to form a training loop with multiple human-machine collaborative navigation knowledge learning scenarios, start the training of the RNN-RRT path planner, calculate the social adaptive navigation feature value of the expert demonstration path during the training process to generate social adaptive navigation information of the corresponding scenario, and learn the navigation features of the expert demonstration path based on the recurrent neural network.

[0149] The knowledge transfer module 600 is used in the new human-machine collaborative navigation knowledge learning scenario to load the network parameters of the trained RNN-RRT path planner as the navigation behavior meta-model and repeat the training process of the optimization module 500 to achieve the transfer learning of social adaptive navigation knowledge.

[0150] The parameter output module 700 is used to terminate training and output the final parameters of the recurrent neural network when the loss value of the recurrent neural network converges or reaches the preset number of training rounds.

[0151] This embodiment provides a robot social adaptive navigation knowledge learning and transfer system to implement the aforementioned robot social adaptive navigation knowledge learning and transfer method. Therefore, the specific implementation of the robot social adaptive navigation knowledge learning and transfer system can be found in the previous section on the implementation of the robot social adaptive navigation knowledge learning and transfer method. For example, the scene construction module 100, training set acquisition module 200, recursive model generation module 300, RNN-RRT planner construction module 400, training optimization module 500, knowledge transfer module 600, and parameter output module 700 are respectively used to implement steps S1, S2, S3, S4, S5, S6, and S7 in the aforementioned robot social adaptive navigation knowledge learning and transfer method. Therefore, its specific implementation can be referred to the descriptions of the corresponding embodiments. To avoid redundancy, further details are omitted here.

[0152] Example 3: This embodiment of the invention provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to realize the above-mentioned robot social adaptive navigation knowledge learning and transfer method.

[0153] Example 4: This embodiment of the invention provides a computer storage medium storing a computer software product. The computer software product includes several instructions to cause a computer device to execute the above-described robot social adaptive navigation knowledge learning and transfer method.

[0154] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for robot social self-adaptive navigation knowledge learning and transfer, characterized in that, The method comprises the following steps: Step S1: constructing multiple human-robot co-adaptive navigation knowledge learning scenarios; Step S2: based on the human-robot co-adaptive navigation knowledge learning scenarios, a training set containing multiple expert demonstration paths and corresponding scene information is prepared; Step S3: the training set is input into a recurrent generation model framework; Step S4: a recurrent neural network structure is designed, and the recurrent neural network is embedded into a cost function module of an RRT path planner to generate an RNN-RRT path planner; Step S5: the multiple human-robot co-adaptive navigation knowledge learning scenarios constitute a training round, and the training of the RNN-RRT path planner is started, in the training process, the social adaptive navigation characteristic values of the expert demonstration paths are calculated to generate the social adaptive navigation information of the corresponding scenes, and the navigation features of the expert demonstration paths are learned based on the recurrent neural network; The social adaptive navigation feature values constitute a 6-dimensional feature vector where T denotes the transpose and each feature component is defined as follows: is the Euclidean distance from the current path node to the goal point; The distance feature from the current path node to the nearest obstacle is calculated by a function constructed by preset adjustment parameters and regularization parameters. , , are respectively the cost features of the front, rear and right side directions of the pedestrian, which are calculated by a Gaussian function based on distance and angle; For the current path node is in the convergence probability characteristics of the future intersection area of pedestrians, the pedestrian intersection point is determined by traversing the real-time coordinates and the direction of travel of pedestrians, and then the distance from the current path node to the pedestrian intersection point is calculated. Step S6: in a new human-robot co-adaptive navigation knowledge learning scenario, the network parameters of the trained RNN-RRT path planner are loaded as a navigation behavior meta-model, and the training process of step S5 is repeated to realize the transfer learning of the social adaptive navigation knowledge; Step S7: when the loss value of the recurrent neural network converges or reaches a preset training round, the training is terminated and the final parameters of the recurrent neural network are output.

2. The method of claim 1, wherein, In step S2, the preparation process of the training set comprises: Setting the starting point and target point of the robot in each human-robot co-adaptive navigation knowledge learning scenario; Controlling the robot to move from the starting point to the target point along a path consistent with human navigation behavior habits through a remote control device to generate the expert demonstration path; Collecting scene data in real time by using the laser radar carried by the robot, wherein the scene data at least includes real-time coordinates of the robot, pedestrian coordinates and motion direction, and obstacle coordinates; Storing the scene data by using the rosbag tool of the ROS system, and associating the scene data with the corresponding expert demonstration path to form the training set.

3. The method of claim 1, wherein, In step S4, the structure of the recurrent neural network satisfies: The recurrent neural network comprises an input layer, at least one hidden layer, and an output layer; The number of neurons of the input layer is consistent with the dimension of the social adaptive navigation characteristic value, and is used to receive the social adaptive navigation characteristic value; The hidden layer contains a preset number of neurons, which associates the navigation feature information at the historical time with the social adaptive navigation characteristic value at the current time through the hidden state to realize the memory and utilization of the historical navigation information; The number of neurons of the output layer is 1, and the sigmoid activation function is used to map the node value of the output to the value range of 0-1.

4. The method of claim 1, wherein, In step S4, the generation of the RNN-RRT path planner further comprises a pre-training step of the recurrent neural network: Randomly generating multiple groups of feature vectors corresponding to the social adaptive navigation characteristic values as pre-training input data; Inputting the pre-training input data into a preset fully connected neural network to calculate a reference cost value; The pre-training input data is inputted, and the reference generation value is taken as the target output, so as to pre-train the recurrent neural network until the weight value of the recurrent neural network converges.

5. The method of claim 1, wherein, The training process of the RNN-RRT path planner in the step S5 further includes a parameter iterative updating step: The planning path generated by the RNN-RRT path planner is discretized into a plurality of path nodes, and the social adaptive navigation feature values corresponding to each path node are calculated; The social adaptive navigation feature values of the path nodes are aligned with the node generation values of the expert demonstration path by using a linear interpolation method; A mean square error loss function is used to calculate the loss value of the predicted generation value of the planning path node and the generation value of the node of the expert demonstration path; The gradient is calculated by back propagation, and the Adam optimizer is used to update the parameters of the recurrent neural network.

6. The method of claim 1, wherein, In the step S6, the transfer learning of the social adaptive navigation knowledge specifically includes: The parameters of the trained recurrent neural network are taken as navigation prior knowledge, and are loaded into the RNN-RRT path planner in a new scene; The RNN-RRT path planner generates an initial planning path in the new scene based on the navigation prior knowledge; The parameters of the recurrent neural network are fine-tuned by calculating the generation value difference between the initial planning path and the expert demonstration path in the new scene, and the initial planning path is optimized, so as to complete the transfer learning of the social adaptive navigation knowledge.

7. A robot social self-adaptive navigation knowledge learning and transfer system, characterized in that, The system is used to implement the robot social adaptive navigation knowledge learning and transfer method of any one of claims 1 to 6, and specifically includes: A scene construction module is configured to construct a plurality of human-robot co-adaptive navigation knowledge learning scenes; A training set acquisition module is configured to make a training set containing a plurality of expert demonstration paths and corresponding scene information based on the human-robot co-adaptive navigation knowledge learning scenes; A recurrent generation model module is configured to input the training set into a recurrent generation model framework; An RNN-RRT planner construction module is configured to design a recurrent neural network structure, and embed the recurrent neural network into a cost function module of an RRT path planner to generate an RNN-RRT path planner; A training optimization module is configured to start the training of the RNN-RRT path planner in a training round constituted by the plurality of human-robot co-adaptive navigation knowledge learning scenes, and calculate the social adaptive navigation feature values of the expert demonstration paths to generate social adaptive navigation information of the corresponding scenes and learn the navigation features of the expert demonstration paths based on the recurrent neural network during the training process; A knowledge transfer module is configured to load the network parameters of the trained RNN-RRT path planner as a navigation behavior meta-model in a new human-robot co-adaptive navigation knowledge learning scene, and repeat the training process of the training optimization module to realize the transfer learning of the social adaptive navigation knowledge; A parameter output module is configured to terminate the training and output the final parameters of the recurrent neural network when the loss value of the recurrent neural network converges or reaches a preset training round number.

8. An electronic device, comprising: The electronic device comprises a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used for storing instructions, and the processor is used for executing the instructions stored in the memory to realize the robot social self-adaptive navigation knowledge learning and migration method in any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores a computer software product, the computer software product comprises a plurality of instructions, and the instructions are used to make a computer device execute the robot social self-adaptive navigation knowledge learning and migration method in any one of claims 1 to 6.

Images