Social sensing robot navigation method, device, equipment, medium and product

Abstract

The invention discloses a social perception robot navigation method, device and equipment, a medium and a product. The method comprises the following steps: acquiring grid map data, robot position data and target object voice data; converting the grid map data into an image information vector by using a first preprocessing network, converting the robot position data into a relative position information vector, and converting the target object voice data into a condition code vector by using a second preprocessing network; connecting to obtain an aggregation state; and inputting the aggregation state into a pre-trained HSAC-LLM model to obtain a target action, and generating voice prompt information according to the target action, so that the target object receives the voice prompt information. According to the embodiment of the invention, bidirectional natural language interaction is realized in the navigation process between the robot and the pedestrian, real-time communication is promoted, navigation safety is enhanced in a dynamic environment, intelligent obstacle avoidance is realized, and interaction experience is provided at the same time.

Description

Technical Field

[0001] The present application belongs to the field of robotics technology, and in particular, relates to a social perception robot navigation method, device, equipment, medium and product. Background Art

[0002] Robot navigation enables robots to move around in an environment while avoiding collisions and reaching a designated destination. It is widely used in various fields, such as autonomous vehicles, drones, warehouse robots, etc. However, traditional robot navigation systems often prioritize efficiency and obstacle avoidance, lacking a detailed understanding of social cues and human intentions, which are essential for harmonious coexistence in shared spaces.

[0003] With the remarkable progress in the field of robotics in recent years, people are paying more and more attention to giving robots the ability to navigate and interact in complex real-world environments. Socially aware navigation has become a key research area, aiming to enable robots to have seamless navigation skills while respecting social norms and participating in natural human interactions. However, related technologies either predict the movement of pedestrians or only send warning signals to pedestrians, failing to promote real interaction between humans and robots. Summary of the invention

[0004] The embodiments of the present application provide a social perception robot navigation method, device, equipment, medium and product to at least solve the problem in related technologies that true interaction between humans and robots cannot be achieved.

[0005] In a first aspect, an embodiment of the present application provides a social perception robot navigation method, which is applied to a robot, and the method includes:

[0006] Obtain grid map data, robot position data, and target object voice data;

[0007] Using a first preprocessing network to convert the grid map data into an image information vector, converting the robot position data into a relative position information vector, and using a second preprocessing network to convert the target object voice data into a conditional code vector;

[0008] Connecting the image information vector, the relative position information vector and the conditional code vector to obtain an aggregated state;

[0009] The aggregated state is input into a pre-trained HSAC-LLM model to obtain a target action, and voice prompt information is generated according to the target action so that the target object receives the voice prompt information.

[0010] In a second aspect, an embodiment of the present application provides a social perception robot navigation device, which is applied to a robot, and the device includes:

[0011] An acquisition module, used to acquire grid map data, robot position data and target object voice data;

[0012] a conversion module, configured to convert the grid map data into an image information vector using a first preprocessing network, convert the robot position data into a relative position information vector, and convert the target object voice data into a conditional code vector using a second preprocessing network;

[0013] A connection module, used for connecting the image information vector, the relative position information vector and the conditional code vector to obtain an aggregated state;

[0014] The input module is used to input the aggregation state into the pre-trained HSAC-LLM model to obtain the target action, and generate voice prompt information according to the target action so that the target object receives the voice prompt information.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the social-perception robot navigation method as described in any one of the embodiments of the first aspect are implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the steps of the social-perception robot navigation method as described in any one of the embodiments of the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the social-perception robot navigation method provided in the first aspect of the embodiment of the present application.

[0018] The social perception robot navigation method, device, equipment, medium and product of the embodiment of the present application obtains raw environmental data, i.e., image and voice data in the environment, etc., and processes the raw environmental data using a preprocessing network to capture the current state, and inputs the state vector into a pre-trained HSAC-LLM model to derive its subsequent action and speed. Once the appropriate action is determined, the action vector output from the model is converted into a coherent natural language interaction with the target object (i.e., pedestrian) to provide voice feedback to the target object, promote real-time communication and enhance navigation safety in a dynamic environment. According to the embodiment of the present application, two-way natural language interaction is achieved during the navigation process between the robot and the pedestrian, achieving intelligent obstacle avoidance while providing a comfortable and interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 It is a flowchart of a social perception robot navigation method provided in an embodiment of the present application;

[0021] Figure 2 is a flowchart of another social perception robot navigation method provided in an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of navigation trajectory visualization of the baseline model and the HSAC-LLM model provided in the embodiments of the present application in the square, corridor and crosswalk scenes;

[0023] Figure 4 is a schematic diagram of the interaction between a robot and a pedestrian in a "crosswalk" scene in a Gazebo environment provided by an embodiment of the present application;

[0024] Figure 5 is a structural schematic diagram of a social perception robot navigation device provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0026] Reference numerals:

[0027] Social perception robot navigation device 500, acquisition module 501, conversion module 502, connection module 503, input module 504,

[0028] Electronic device 600 , processor 601 , memory 602 , communication interface 603 , bus 610 . DETAILED DESCRIPTION

[0029] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0030] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0031] Robot navigation enables robots to move around in an environment while avoiding collisions and reaching a designated destination. It is widely used in various fields, such as autonomous vehicles, drones, warehouse robots, etc. However, traditional robot navigation systems often prioritize efficiency and obstacle avoidance, lacking a detailed understanding of social cues and human intentions, which are essential for harmonious coexistence in shared spaces.

[0032] With the remarkable progress in the field of robotics in recent years, there is a growing emphasis on giving robots the ability to navigate and interact in complex real-world environments. Socially-aware navigation has become a key research area that aims to equip robots with seamless navigation skills while respecting social norms and engaging in natural human interactions. In addition to traditional navigation capabilities, robots are also required to understand human social norms, predict human behavior, and prevent actions that may cause discomfort or misunderstandings. For example, robots with socially-aware navigation capabilities can be deployed to navigate within international airports. Using advanced language understanding capabilities and a deep understanding of social norms, these robots are able to communicate in natural language with passengers from different cultural backgrounds while maintaining appropriate social distance.

[0033] Early research often aims to model or predict human motion so that robots can develop safe paths. Work such as social force models (SFM), social LSTM, and social GAN ​​focuses on predicting human motion trends, enabling robots to plan safer trajectories. In addition, some studies recognize human gestures or gaze directions to predict pedestrian motion and adjust navigation actions accordingly. However, these methods still mainly focus on how robots can avoid pedestrians, often resulting in longer detour distances or even collisions due to inaccurate pedestrian motion predictions.

[0034] To promote meaningful interactions between robots and individuals around them, recent studies have utilized “beeps,” “voice reminders,” and “nudges” as warning mechanisms for pedestrians. However, these approaches only provide simple one-way alerts, which not only fail to achieve a truly comfortable and interactive experience, but also run the risk of ambiguity and misunderstanding, especially in noisy or crowded environments where the intent or source of the alert may not be clear. In addition, repetitive or loud alerts may become intrusive or irritating in environments that require quietness or etiquette, negatively affecting the user experience and acceptance of robot assistance.

[0035] It can be seen that the relevant technologies either predict the movement of pedestrians or only send warning signals to pedestrians, failing to promote real interaction between humans and robots.

[0036] In order to solve the problems of related technologies, the embodiments of the present application provide a social perception robot navigation method, device, equipment, medium and product.

[0037] The social perception robot navigation method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in combination with the accompanying drawings.

[0038] It should be noted that the social perception robot navigation method in the embodiment of the present application is applied to a robot, and the target object in this embodiment refers to a pedestrian within a preset area with the robot as a reference.

[0039] It should be noted that the standard soft actor-critic (SAC) method was originally introduced to model the continuous actions of robot tasks. In the embodiment of the present application, a hybrid SAC model is designed specifically for the current navigation task, emphasizing the interaction between humans and robots.

[0040] In an embodiment of the present application, a hybrid soft actor-critic (HSAC-LLM) combined with a large language model seamlessly integrates deep reinforcement learning and a large language model, enabling it to predict continuous and discrete actions for navigation. It is worth noting that the HSAC-LLM model implements two-way interaction with a pedestrian model based on natural language. When a possible collision with a pedestrian is detected, the robot can initiate or respond to communication with the pedestrian, obtain and execute subsequent avoidance strategies. In addition, experimental results in 2D simulations, Gazebo environments, and real environments show that the HSAC-LLM model not only efficiently implements interaction with humans, but also performs better than the most advanced deep reinforcement learning algorithms in terms of navigation and obstacle avoidance, that is, the model outperforms the current most advanced DRL algorithms in performance.

[0041] Figure 1 The flowchart of the social perception robot navigation method of the embodiment of the present application is shown. Figure 1As shown, the social perception robot navigation method may specifically include the following steps:

[0042] S101, obtaining grid map data, robot position data and target object voice data;

[0043] S102, using a first preprocessing network to convert the grid map data into an image information vector, converting the robot position data into a relative position information vector, and using a second preprocessing network to convert the target object voice data into a conditional code vector;

[0044] S103, connecting the image information vector, the relative position information vector and the conditional code vector to obtain an aggregated state;

[0045] S104: input the aggregation state into a pre-trained HSAC-LLM model to obtain a target action, and generate voice prompt information according to the target action so that the target object receives the voice prompt information.

[0046] Thus, by acquiring the original environmental data, i.e., the image and voice data in the environment, etc., and processing the original environmental data with a preprocessing network to capture the current state, the state vector is input into the pre-trained HSAC-LLM model to derive its subsequent action and speed, and once the appropriate action is determined, the action vector output from the model is converted into a coherent natural language interaction with the target object (i.e., the pedestrian) to provide voice feedback to the target object, promote real-time communication and enhance navigation safety in a dynamic environment. According to the embodiment of the present application, a two-way natural language interaction is realized during the navigation process between the robot and the pedestrian, achieving intelligent obstacle avoidance while providing a comfortable and interactive experience.

[0047] The specific implementation methods of the above steps are introduced below.

[0048] In some embodiments, in S101, the grid map data includes a radar scanning grid map and a three-channel target object grid map, the radar scanning grid map represents the spatial configuration information of the robot, and the three-channel target object grid map represents the position information and speed information of the target object within a preset area, for example, a 180-degree 2D laser scanning grid map and a three-channel pedestrian grid map; the robot position data includes the target position information and target direction information of the robot within three time steps, for example, it can be represented as a 9-dimensional relative position vector; the target object voice data is generated by initiating or responding to voice interaction with the target object when a potential collision with the target object is perceived.

[0049] In this way, the grid map data, the robot position data and the target object voice data constitute the input source of the preset model in the embodiment of this application. However, it should be noted that this embodiment does not directly integrate these raw environmental data into the pre-trained HSAC-LLM model, but uses two pre-processing networks to convert the image and voice data in the environment into corresponding state vectors, that is, execute S102.

[0050] In the specific implementation, the first preprocessing network Img_PreNet uses three convolutional layers and three maximum pooling layers to extract features on the grid map data, and generates a 512-dimensional image information vector S through a fully connected layer. i .

[0051] In the specific implementation, the target object voice data is input into the second preprocessing network Lang_PreNet network to generate a state vector S for the robot strategy condition. c .

[0052] Furthermore, in some embodiments, in S103, the image information vector S i , relative position information vector S p and the condition code vector S c is connected and passed to State_PreNet, which contains a fully connected layer with 512 units, forming a 512-dimensional vector S, which is used as the environment state of the HSAC-LLM model and can be defined as the aggregate state S.

[0053] It should be noted that the pre-trained HSAC-LLM model includes a hybrid soft actor-critic HSAC module and a Lang_GenNet module. Specifically, HSAC models the continuous and discrete actions of the robot, while Lang_GenNet complements the model, interpreting the discrete outputs and conveying them to pedestrians through voice communication, ensuring a seamless and safe co-navigation experience. It can be seen that the HSAC-LLM model enables two-way communication between the robot and the pedestrian to coordinate trajectories and prevent collisions.

[0054] Furthermore, it should be understood that in the robot navigation process, the behavior involves continuous and discrete elements, and the target action includes continuous actions and discrete actions.

[0055] Therefore, the specific implementation of S104 includes: using the actor network to generate a shared hidden state representation h, which is used to generate a discrete distribution π and a mean μ and a standard deviation σ of the continuous component, and a discrete action a c Sampling from a discrete distribution π, i.e., a discrete action a cTo satisfy the discrete distribution π and randomly generate actions; calculate the continuous action of linear velocity v and angular velocity ω; use the critic network to take the aggregate state S, the mean μ and standard deviation σ of the continuous component as input, and predict the Q value of all discrete actions at its output layer; use the Lang_GenNet module to generate voice prompt information according to the target action.

[0056] In specific implementation, the continuous motion of the linear velocity v and the angular velocity ω is calculated according to the following formula (1):

[0057] [v,ω]=tanh(μ+εσ); (1)

[0058] Where v represents the linear velocity, ω represents the angular velocity, μ represents the mean of the continuous component, σ represents the standard deviation of the continuous component, and ε represents a random variable of the standard normal distribution.

[0059] In specific implementation, the optimization strategy π is implemented according to the following formula (2), so that the objective function [alogπ(a|[S i ,S p ],S c )-Q(S,μ,ω,a c )] reaches its minimum expected value:

[0060]

[0061] Among them, π * represents the discrete distribution of updates, E represents the expected value of the distribution π with respect to action a, and a represents the action that satisfies π.

[0062] In specific implementation, the Lang_GenNet module is based on the architecture of large language models (LLMs), which is carefully designed to convert the target actions output from the HSAC model into coherent natural language interactions with pedestrians. This module is instantiated by leveraging the capabilities of the GPT-4 model, demonstrating a sophisticated integration of advanced language understanding and generation techniques.

[0063] In this way, by classifying the robot's motion control (angular velocity and linear speed) into continuous actions, the decision-making after interaction into discrete actions, and once the appropriate action is determined, the robot utilizes Lang_GenNet to provide voice feedback to the pedestrian, facilitating real-time communication and enhancing navigation safety in dynamic environments.

[0064] As an optional embodiment, the reward function used for HSAC-LLM model training is:

[0065]

[0066]

[0067] Among them, r t represents the reward of the robot at time step t; the number of steps taken is used as the negative penalty

[0068] If the robot successfully reaches its target position and the minimum distance to the target object The safety threshold d is not exceeded during the entire trip safe , then the robot will receive a positive reward r arrive ; If the robot reaches its target position but the minimum distance to the target object at any time Below the safety threshold d safe , then r arrive Apply a penalty proportional to the severity of the violation; in the event of a collision, Will be penalized collide In other cases, is 0; ∈1 represents the first hyperparameter;

[0069] By using the robot's current position p t and target location Encourage the robot to move towards its target position, ∈2 represents the second hyperparameter;

[0070] Initiate a conversation request in the robot And the request is consistent with the trajectory of the target object. Reward: Initiate a conversation request with the target object And the robot will give a Rewards; in other cases is 0.

[0071] In another embodiment, a specific implementation of another social perception robot navigation method can be referred to Figure 2 .

[0072] like Figure 2 As shown, another social perception robot navigation method provided by an embodiment of the present application may include the following steps.

[0073] Radar scan and pedestrian information are first input into the Img_PreNet network to obtain the state S i At the same time, the robot’s position information is captured as state S p When a potential collision risk is detected, the robot initiates or responds to voice interactions with humans, and these interactions are processed through the Lang_PreNet network to generate conditional states S cCombining these states, the robot infers its subsequent actions and velocities by inputting the aggregated state S into the trained HSAC-LLM model. Crucially, the model exploits both states and conditions in its decision making, denoted as π(a|[S i ,S p ],S c ). Once the appropriate action is determined, the robot leverages Lang_GenNet to provide voice feedback to the pedestrian, facilitating real-time communication and enhancing navigation safety in dynamic environments.

[0074] Specifically, the raw environment data is first processed by the PreNet network to capture the current state, where the voice state input is derived from the dialogue that the robot actively or passively conducts with the pedestrian when a potential collision is detected. Then, these states are input into the pre-trained HSAC-LLM model, which simultaneously models the distribution of the robot's continuous actions (linear and angular velocities) and discrete actions (interaction codes). In addition, the robot communicates its final action with the pedestrian by generating the corresponding voice notification, using the LLM model.

[0075] Optionally, train in a 2D simulator using the OpenCV framework.

[0076] Optionally, define the robot’s reward at time step t as r t , which consists of four components:

[0077]

[0078] for If the robot successfully reaches its destination and its minimum distance to the pedestrian The safety threshold d is not exceeded during the entire trip safe , then the robot will receive a positive reward r arrive However, if the robot reaches its goal but is at any time within the minimum distance of a pedestrian Below the safety threshold d safe , then r arrive A penalty proportional to the severity of the violation will be applied. will be penalized negatively, denoted as r collide In all other cases, Set to 0. Then it is formulated as follows:

[0079]

[0080] Among them, ∈1 represents the first hyperparameter.

[0081] By using the robot's current position p t and the target position pg , motivating the robot to move towards its target position. It is defined as follows:

[0082]

[0083] Among them, ∈2 represents the second hyperparameter.

[0084] The number of steps taken is used as a negative penalty This encourages the robot to choose shorter paths, thus making its movement more efficient.

[0085] In addition, interactive rewards are introduced Designed to encourage appropriate interaction between robots and humans. If the request is consistent with the current pedestrian trajectory, Reward. On the contrary, if a pedestrian initiates a conversation request If the path planning generated by the robot according to the model is consistent with the pedestrian's request, then In all other cases, Set to 0. The formula is as follows:

[0086]

[0087] Therefore, the final reward r t It can be calculated as:

[0088]

[0089] Furthermore, the performance of the social perception robot navigation method of the present application embodiment is evaluated in three different environments (2D simulation environment, Gazebo simulation environment and real world environment). The specific evaluation process and results are as follows.

[0090] Experiments in the 2D simulator:

[0091] In the 2D simulator, all pedestrians are controlled by a customized version of the Reciprocal Velocity Obstacle (RVO) algorithm, which enables them to respond to the robot's avoidance requests by performing stop and edge actions. We call this enhanced model Interactive Reciprocal Velocity Obstacle (IRVO). The experiments focus on scenarios involving a single pedestrian, which is very similar to real-world conditions in voice-based human-robot interaction. For all algorithms, the action space of linear and angular velocities is set to v∈[0,0.6]m / s and ω∈[-0.9,0.9]rad / s, respectively; the speed of the pedestrian is fixed at 0.5m / s. OpenAI's GPT-4 model is selected as the LLM of the HSAC-LLM model. A four-dimensional discrete action space is used to represent the action code, denoted as a c∈{none,stop,margin-right,margin-left}, where “none” means no interaction, “stop” means signaling pedestrians to stop, and “margin” means commanding pedestrians to move left or right to create space.

[0092] Considering the complexity of the real-world navigation environment, three different scenarios were designed in the simulator: square, corridor, and crosswalk. Figure 3 A schematic diagram showing the visualization of the navigation trajectories of the baseline model and the HSAC-LLM model in the square, corridor, and crosswalk scenarios.

[0093] like Figure 3 As shown, in order to simulate different navigation challenges, the starting and ending positions of the robot and pedestrian in each scenario follow a specific pattern: in the square, the robot and pedestrian are randomly placed at locations within the map; in the corridor, the robot and pedestrian start from random starting points at both ends of the corridor and cross to the other side; in the crosswalk, the robot and pedestrian start their paths from random locations at the ends of different corridors with the goal of reaching the other side.

[0094] The robot's main goal is to navigate to the destination, giving priority to pedestrian safety and collision avoidance. The parameters of the reward function are set as follows: rarrive = 500, rcollide = -500, rstep = -5, dsafe = 1.0, ε1 = -50 and ε2 = 200.

[0095] The following metrics are selected to evaluate navigation performance: success, the ratio of the number of times the robot safely reaches the goal to the total number of attempts; collision, the ratio of the number of collisions with walls or pedestrians to the total number of attempts; step number, the average number of steps the robot takes to successfully reach the goal; training time, the time required to train the model; collision with humans, the ratio of the number of collisions with humans to the total number of attempts; minimum distance to humans, the shortest distance between the robot and humans during successful navigation to the goal; and comfort space compliance, the ratio of the number of cases where the minimum distance exceeds the comfort distance threshold to the total number of attempts.

[0096] State-of-the-art deep reinforcement learning (DRL) algorithms designed for continuous action spaces are adopted as our baseline methods, including Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), and Double-Delayed Deep Deterministic Policy Gradient (TD3).

[0097] All models were trained for 128,000 steps on an i7-10750H CPU and NVIDIA RTX 2070 GPU. Figure 3 Trajectories of different agents are depicted. During testing, a reverse request probability pr = 0.5 was used to ensure that both the robot and the pedestrian have equal chances to initiate an avoidance request.

[0098] Table 1 below shows the results of 500 tests for each scenario, that is, the performance comparison between the HSAC-LLM model of the embodiment of the present application and different baseline models in three scenarios. Among them, MDH represents the minimum distance to humans, and CSC represents the comfort space compliance.

[0099] Table 1

[0100]

[0101] As can be seen from Table 1, the HSAC-LLM algorithm of the embodiment of the present application outperforms the existing deep reinforcement learning (DRL) algorithm in all three scenarios, achieving the highest success rate, the lowest collision rate and the shortest number of steps. Specifically, in the square scene, a success rate of 0.91, a collision rate of 0.08, a minimum distance from humans of 0.31, and a comfortable space compliance of 0.85 were achieved. Compared with other methods, the significantly lower collision rate highlights the ability of the model of the embodiment of the present application to greatly reduce collisions through natural language-based interactions. In more complex corridor and crosswalk scenarios, where collisions between humans and robots are more likely to occur, the model of the embodiment of the present application still maintains a collision rate of only 0.07 and 0.09. This is in contrast to other methods, which rely only on one-way detours to avoid collisions. This strategy often leads to collisions in slightly more complex environments due to insufficient escape time. The HSAC-LLM model of the embodiment of the present application flexibly communicates with pedestrians based on natural language before a potential collision occurs, fully understands their intentions, and eliminates ambiguity in communication, thereby effectively preventing any impending collisions.

[0102] In order to more intuitively demonstrate the navigation performance, examples from three scenarios are randomly selected and Figure 3 The navigation trajectories of the baseline model and the model of the embodiment of the present application are shown in . It can be observed that the navigation paths of these baseline models are quite tortuous, especially the PPO model, which collided with pedestrians in both the square and the crosswalk scenes. In contrast, the model of the embodiment of the present application promotes two-way natural language dialogue between the robot and pedestrians, and can dynamically adjust the traffic priority according to the specific situation. Therefore, it reaches the destination efficiently through a shorter path while avoiding collisions.

[0103] Gazebo and real-world experiments:

[0104] To enhance the practicality of the algorithm, realistic 3D scenarios were created in the Gazebo simulation environment. In these scenarios, the speed of pedestrians is no longer fixed at 0.5m / s, but is estimated through the leg tracking frame, allowing the robot to dynamically decide the appropriate distance to initiate or respond to pedestrian conversations. The pedestrian's voice input and the robot's response are simulated using the computer's microphone and speakers. Extensive evaluations in three scenarios showed that the robot consistently demonstrated a high degree of responsiveness, navigating according to the pedestrian's preferences, either actively or passively.

[0105] Figure 4 Schematic diagram of the interaction between the robot and pedestrians in the "crosswalk" scene in the Gazebo environment. (a) and (b) respectively show the explicit and implicit requests of the pedestrians, demonstrating the ability of the model in this application to understand natural language and respond accordingly.

[0106] like Figure 4 As shown, the robot obeys the pedestrian's stop request in the crosswalk scenario, allowing the pedestrian to pass safely. It is worth noting that the model of this application is good at understanding explicit requests such as "Can you let me pass first?" and implicit requests such as "I'm in a hurry", so as to respond appropriately. This shows that the model of this application is able to understand the deeper nuances of human natural language, not just literal interpretation. By deeply understanding human needs, the model of this application can plan reasonable navigation routes to meet these needs.

[0107] In addition, real-world environment tests were conducted using the TurtleBot3 robot to evaluate the navigation capabilities of the model of this application. Specifically, the Burger model was selected, equipped with an RPLIDAR A12D lidar, an Intel D435i depth camera, and a NUC11PHKi7C computer. These tests show that even in practical settings, the model of this application still exhibits superior navigation performance. For example, when pedestrians explicitly indicated that they wanted to pass from the left, the robot strategically planned to move to the right and successfully performed the necessary operations. This further emphasizes the effectiveness of the model of this application in interacting with humans and implementing reasonable path planning and actions in real-world scenarios.

[0108] Ablation studies:

[0109] Furthermore, to evaluate the impact of the discrete action space dimension, the navigation performance of the HSAC-LLM model using 2D and 4D discrete action spaces is compared. Specifically, a common 2D discrete action space is used, defined as ac∈{none,stop}.

[0110] Table 2 below shows the navigation results of the HSAC-LLM model using these two configurations, i.e., the performance of HSAC-LLM trained in 2D and 4D discrete action spaces (with the best values ​​in bold). As shown in Table 2, despite requiring slightly longer training time, the HSAC-LLM model using 4D discrete action space outperforms its 2D counterpart in all metrics except training time. This improvement is particularly evident in scenarios where the robot and pedestrian are close and need to adjust their positions to avoid collisions.

[0111] Table 2

[0112]

[0113] Furthermore, in order to verify the effectiveness of the LLM component in the HSAC-LLM model, a comparative experiment was conducted to remove the LLM module responsible for natural language two-way interaction with pedestrians from the HSAC-LLM model. Specifically, when the robot detected a potential collision risk, it only issued an auditory warning to the pedestrians without the ability to receive their voice feedback.

[0114] Table 3 below shows the comparison between the HSAC model with the LLM module removed and the original HSAC-LLM model in three scenarios (the performance of the HSAC-LLM trained in the one-way and two-way interaction scenarios, where the best value is shown in bold). According to the results in Table 3, with the addition of the LLM module, the success rate of all three scenarios has improved, while the collision rate and the number of steps required to reach the target have decreased. It is worth noting that the improvement in the collision rate is more significant, especially in the square and crosswalk scenarios, which are reduced by 42.86% and 30.77%, respectively. This shows that the integration of LLM enables the robot to better understand natural language and communicate with pedestrians, thereby eliminating collisions that may arise from insufficient communication. The addition of the LLM module not only significantly improves navigation safety and reduces potential collisions, but also significantly enhances the robot's ability in social navigation. By enabling more detailed natural language communication with pedestrians, this development emphasizes the key role of advanced language processing in human-robot interaction in rich and dynamic environments. This progress in social navigation demonstrates the great potential of integrating complex language models into robotic systems, paving the way for more intuitive and effective human-robot collaboration, especially in complex social environments.

[0115] Table 3

[0116]

[0117] It can be seen that the social perception robot navigation method of the embodiment of the present application is compared with two existing navigation interaction methods, non-interactive strategy (detour) and one-way voice interaction (beep), through natural language dialogue. According to the comparison results, the social perception robot navigation method of the embodiment of the present application enables the robot to understand human intentions more intelligently and clearly in a complex environment.

[0118] Therefore, based on the pre-trained hybrid soft actor-critic combined with a large language model (HSAC-LLM), deep reinforcement learning (DRL) is combined with a large language model (LLM) for social-aware robot navigation to achieve natural language dialogue with humans; the social-aware robot navigation method implemented using this model not only improves the safety and efficiency of robot navigation, but also promotes the harmonious coexistence of humans and robots in complex social scenarios, paving the way for efficient and socially-aware human-robot interaction in dynamic environments.

[0119] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0120] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a social perception robot navigation device 500.

[0121] like Figure 5 As shown, the social perception robot navigation device 500 may include:

[0122] An acquisition module 501 is used to acquire grid map data, robot position data and target object voice data;

[0123] A conversion module 502, configured to convert the grid map data into an image information vector using a first preprocessing network, convert the robot position data into a relative position information vector, and convert the target object voice data into a conditional code vector using a second preprocessing network;

[0124] A connection module 503, used to connect the image information vector, the relative position information vector and the condition code vector to obtain an aggregated state;

[0125] The input module 504 is used to input the aggregation state into the pre-trained HSAC-LLM model to obtain the target action, and generate voice prompt information according to the target action so that the target object receives the voice prompt information.

[0126] Optionally, the grid map data includes a radar scanning grid map and a three-channel target object grid map, the radar scanning grid map represents the spatial configuration information of the robot, and the three-channel target object grid map represents the position information and speed information of the target object within a preset area; the robot position data includes the target position information and target direction information of the robot within three time steps; the target object voice data is generated by initiating or responding to voice interaction with the target object when a potential collision with the target object is sensed.

[0127] Optionally, the HSAC-LLM model includes a hybrid soft actor-critic HSAC module and a Lang_GenNet module, and the target action includes a continuous action and a discrete action.

[0128] In some embodiments, the input module 504 is specifically used to generate a shared hidden state representation using an actor network, to generate a discrete distribution and a mean and a standard deviation of continuous components, and discrete actions are sampled from the discrete distribution; to calculate the continuous actions of linear velocity and angular velocity; to use the critic network to take the aggregate state, the mean and the standard deviation of the continuous components as input, and to predict the Q value of all discrete actions at its output layer; and to use the Lang_GenNet module to generate voice prompt information according to the target action.

[0129] Specifically, the continuous motion of linear velocity and angular velocity is calculated according to the following formula:

[0130] [v,ω]=tanh(μ+εσ);

[0131] Where v represents the linear velocity, ω represents the angular velocity, μ represents the mean of the continuous component, σ represents the standard deviation of the continuous component, and ε represents a random variable of the standard normal distribution.

[0132] In some embodiments, the conversion module 502 is specifically used to perform feature extraction on the grid map data using three convolutional layers and three maximum pooling layers, and generate a 512-dimensional image information vector through a fully connected layer.

[0133] In some optional embodiments, the reward function used for HSAC-LLM model training is:

[0134]

[0135] Among them, r trepresents the reward of the robot at time step t; the number of steps taken is used as the negative penalty

[0136] If the robot successfully reaches its target position and the minimum distance to the target object The safety threshold d is not exceeded during the entire trip safe , then the robot will receive a positive reward r arrive ; If the robot reaches its target position but the minimum distance to the target object at any time Below the safety threshold d safe , then r arrive Apply a penalty proportional to the severity of the violation; in the event of a collision, Will be penalized collide In other cases, is 0; ∈1 represents the first hyperparameter;

[0137] By using the robot's current position p t and target location Encourage the robot to move towards its target position, ∈2 represents the second hyperparameter;

[0138] Initiate a conversation request in the robot And the request is consistent with the trajectory of the target object. Reward: Initiate a conversation request with the target object And the robot will give a Rewards; in other cases is 0.

[0139] It should be noted that, for the convenience of description, the above device is described in various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0140] The device of the above-mentioned embodiment is used to implement the corresponding social perception robot navigation method in any of the aforementioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0141] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device.

[0142] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided by this embodiment is shown.

[0143] The electronic device 600 may include a processor 601 and a memory 602 storing computer program instructions.

[0144] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0145] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0146] In certain embodiments, the memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0147] The processor 601 implements any one of the social-aware robot navigation methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .

[0148] In some examples, the electronic device 600 may further include a communication interface 603 and a bus 610. Figure 6 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.

[0149] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0150] Bus 610 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus 610 may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 610 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0151] Exemplarily, the electronic device 600 may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA).

[0152] Based on the same technical concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the social perception robot navigation methods in the above-mentioned embodiments is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0153] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the social-aware robot navigation method. Corresponding to the execution subject corresponding to each step in each embodiment of the social-aware robot navigation method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0154] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0155] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0156] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0157] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0158] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A social perception robot navigation method, characterized in that: Applied to a robot, the method comprises: Obtain grid map data, robot position data, and target object voice data; Using a first preprocessing network to convert the grid map data into an image information vector, converting the robot position data into a relative position information vector, and using a second preprocessing network to convert the target object voice data into a conditional code vector; Connecting the image information vector, the relative position information vector and the conditional code vector to obtain an aggregated state; The aggregated state is input into a pre-trained HSAC-LLM model to obtain a target action, and voice prompt information is generated according to the target action so that the target object receives the voice prompt information.

2. The method according to claim 1, characterized in that The grid map data includes a radar scanning grid map and a three-channel target object grid map, wherein the radar scanning grid map represents the spatial configuration information of the robot, and the three-channel target object grid map represents the position information and speed information of the target object within a preset area; The robot position data includes target position information and target direction information of the robot within three time steps; The target object voice data is generated when a potential collision with the target object is sensed and a voice interaction with the target object is initiated or responded to.

3. The method according to claim 1, characterized in that The HSAC-LLM model includes a hybrid soft actor-critic HSAC module and a Lang_GenNet module, and the target action includes a continuous action and a discrete action; The step of inputting the aggregated state into a pre-trained HSAC-LLM model to obtain a target action, and generating voice prompt information according to the target action includes: The actor network is used to generate a shared hidden state representation that is used to generate a discrete distribution as well as the mean and standard deviation of the continuous component. Discrete actions are sampled from the discrete distribution. Calculate linear and angular velocities for continuous motion; A critic network takes as input the aggregated state, the mean and standard deviation of the continuous components, and predicts the Q-values ​​of all discrete actions at its output layer; The Lang_GenNet module is used to generate voice prompt information according to the target action.

4. The method according to claim 3, characterized in that The continuous action of calculating the linear velocity and the angular velocity includes: The linear and angular velocities of continuous motion are calculated according to the following formulas: [v,ω]=tanh(μ+εσ); Where v represents the linear velocity, ω represents the angular velocity, μ represents the mean of the continuous component, σ represents the standard deviation of the continuous component, and ε represents a random variable of the standard normal distribution.

5. The method according to claim 1, characterized in that The step of converting the grid map data into an image information vector by using a first preprocessing network comprises: Three convolutional layers and three maximum pooling layers are used to extract features on the grid map data, and a 512-dimensional image information vector is generated through a fully connected layer.

6. The method according to claim 1, characterized in that The reward function used for HSAC-LLM model training is: Among them, r t represents the reward of the robot at time step t; the number of steps taken is used as the negative penalty If the robot successfully reaches its target position and the minimum distance to the target object The safety threshold d is not exceeded during the entire trip safe , then the robot will receive a positive reward r arrive ; If the robot reaches its target position but the minimum distance to the target object at any time Below the safety threshold d safe , then r arrive Apply a penalty proportional to the severity of the violation; in the event of a collision, Will be penalized collide In other cases, is 0; ∈1 represents the first hyperparameter; By using the robot's current position p t and target location Encourage the robot to move towards its target position, ∈2 represents the second hyperparameter; Initiate a conversation request in the robot And the request is consistent with the trajectory of the target object. Reward: Initiate a conversation request with the target object And the robot will give a Rewards; in other cases is 0.

7. A social perception robot navigation device, characterized in that: Applied to a robot, the device comprises: An acquisition module, used to acquire grid map data, robot position data and target object voice data; a conversion module, configured to convert the grid map data into an image information vector using a first preprocessing network, convert the robot position data into a relative position information vector, and convert the target object voice data into a conditional code vector using a second preprocessing network; A connection module, used for connecting the image information vector, the relative position information vector and the conditional code vector to obtain an aggregated state; The input module is used to input the aggregation state into a pre-trained HSAC-LLM model to obtain a target action, and generate voice prompt information according to the target action so that the target object receives the voice prompt information.

8. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor calls the computer program instructions, it implements the social perception robot navigation method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when called by a processor, implement the social-aware robot navigation method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the social-aware robot navigation method as described in any one of claims 1-6.

Images