Robot navigation method and system, terminal and storage medium

Abstract

The invention discloses a robot navigation method and system, a terminal and a storage medium, and relates to the technical field of navigation control, and the method comprises the following steps: obtaining robot state data, and depth image data and color image data corresponding to a target area; according to the depth image data and the color image data, obtaining a bird's-eye view feature map corresponding to the target area, and obtaining scene occupation features corresponding to the target area based on the bird's-eye view feature map; obtaining pedestrian three-dimensional attitude data in the target area according to the color image data, and determining intention features corresponding to pedestrians in the target area based on the pedestrian three-dimensional attitude data; obtaining interaction representation data according to the intention features and the robot state data; and according to the scene occupation characteristics, the interaction representation data and the robot state data, acquiring a navigation action for the robot through a deep reinforcement learning network, and performing motion control on the robot according to the navigation action. Therefore, the accuracy of robot navigation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of navigation control, and particularly relates to a robot navigation method and system, a terminal and a storage medium. BACKGROUND

[0002] With the development of science and technology, the application of mobile robots is more and more extensive, and the navigation requirements for robots are also higher and higher.

[0003] At present, robot navigation usually relies on visual perception technology, determines the position of obstacles in the region based on the corresponding image of the region, and avoids obstacles. The problem of the prior art is that only the avoidance of static obstacles cannot fully consider the social situation and the intention of pedestrians and realize navigation conforming to social norms in a complex crowded environment, for example, when there are movable pedestrians in the region, which is not conducive to improving the accuracy of robot navigation.

[0004] Therefore, the related technology still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a robot navigation method and system, a terminal and a storage medium, which aims to solve the technical problem that only the avoidance of static obstacles cannot fully consider the social situation and the intention of pedestrians and realize navigation conforming to social norms in a complex crowded environment, for example, when there are movable pedestrians in the region, which is not conducive to improving the accuracy of robot navigation.

[0006] In order to achieve the above purpose, the first aspect of the present application provides a robot navigation method, wherein the method comprises: obtaining robot state data, and depth image data and color image data corresponding to a target region; obtaining a bird's eye view feature map corresponding to the target region according to the depth image data and the color image data, and obtaining scene occupancy features corresponding to the target region based on the bird's eye view feature map; obtaining pedestrian three-dimensional pose data in the target region according to the color image data, and determining intention features corresponding to pedestrians in the target region based on the pedestrian three-dimensional pose data; obtaining interaction representation data according to the intention features and the robot state data, wherein the interaction representation data is used to represent interaction information between pedestrians in the target region and the robot; obtaining navigation actions for the robot through a deep reinforcement learning network according to the scene occupancy features, the interaction representation data and the robot state data, and performing motion control on the robot according to the navigation actions.

[0007] Optionally, the robot state data comprises a target direction, a target distance and a robot speed, wherein the target direction is a direction of a target position of the robot movement, and the target distance is a distance between the robot and the target position.

[0008] Optionally, the obtaining of the bird's eye view perspective feature map corresponding to the target region according to the depth image data and the color image data comprises: obtaining a bird's eye view perspective feature map corresponding to the target region according to the depth image data and the color image data; encoding the bird's eye view perspective feature map through a time sequence encoder to obtain a space-time bird's eye view feature; encoding the space-time bird's eye view feature through a bird's eye view encoder to obtain the scene occupancy feature corresponding to the target region.

[0009] Optionally, the obtaining of the pedestrian three-dimensional pose data in the target region according to the color image data comprises: detecting a pose based on the color image data to obtain a two-dimensional human body pose of a pedestrian in the target region; obtaining a three-dimensional human body pose according to the two-dimensional human body pose, the depth image data and a preset camera intrinsic parameter; converting the three-dimensional human body pose into a coordinate system corresponding to the robot according to a preset camera extrinsic parameter to obtain the pedestrian three-dimensional pose data.

[0010] Optionally, the determining of the intention feature corresponding to the pedestrian in the target region based on the pedestrian three-dimensional pose data comprises: determining the intention feature corresponding to the pedestrian in the target region through an intention transformer according to the pedestrian three-dimensional pose data, wherein the intention feature is used to represent a behavior intention of the pedestrian.

[0011] Optionally, the obtaining of the interaction representation data according to the intention feature and the robot state data comprises: encoding the robot state data to obtain a robot state embedding; obtaining the interaction representation data through an interaction transformer according to the intention feature and the robot state embedding.

[0012] Optionally, the obtaining of the navigation action for the robot through a deep reinforcement learning network according to the scene occupancy feature, the interaction representation data and the robot state data comprises: encoding the robot state data to obtain state encoding data; By concatenating the above-mentioned scene occupancy features, the above-mentioned interaction representation data, and the above-mentioned state encoding data, the concatenated features are obtained. The spliced ​​features are fused using a fusion network to obtain the state embedding. The above state is embedded into the deep reinforcement learning network to obtain the navigation action output by the deep reinforcement learning network.

[0013] A second aspect of this application provides a robot navigation system, wherein the system includes: The data acquisition module is used to acquire robot status data, as well as depth image data and color image data corresponding to the target area; The scene occupancy feature acquisition module is used to acquire the bird's-eye view feature map corresponding to the target area based on the depth image data and the color image data, and to acquire the scene occupancy feature corresponding to the target area based on the bird's-eye view feature map. The intent feature acquisition module is used to acquire pedestrian three-dimensional pose data in the target area based on the above-mentioned color image data, and to determine the intent features of the pedestrians in the target area based on the above-mentioned pedestrian three-dimensional pose data. The interaction representation data acquisition module is used to acquire interaction representation data based on the above-mentioned intention features and the above-mentioned robot state data, wherein the interaction representation data is used to characterize the interaction information between the pedestrian and the robot in the above-mentioned target area; The navigation module is used to obtain navigation actions for the robot through a deep reinforcement learning network based on the scene occupancy characteristics, the interaction representation data, and the robot state data, and to perform motion control on the robot based on the navigation actions.

[0014] A third aspect of this application provides a terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements any of the steps of the robot navigation method described above.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described robot navigation methods.

[0016] As can be seen from the above, in this application, robot state data, as well as depth image data and color image data corresponding to the target area are obtained; based on the depth image data and the color image data, a bird's-eye view feature map corresponding to the target area is obtained, and scene occupancy features corresponding to the target area are obtained based on the bird's-eye view feature map; based on the color image data, pedestrian three-dimensional pose data within the target area is obtained, and intention features corresponding to the pedestrian within the target area are determined based on the pedestrian three-dimensional pose data; interaction representation data is obtained based on the intention features and the robot state data, wherein the interaction representation data is used to characterize the interaction information between the pedestrian and the robot within the target area; based on the scene occupancy features, the interaction representation data, and the robot state data, navigation actions for the robot are obtained through a deep reinforcement learning network, and motion control of the robot is performed based on the navigation actions.

[0017] Thus, during robot navigation, a bird's-eye view feature map is used to represent the scene occupancy characteristics of the target area. Pedestrians within the target area are also considered, and their intention features are determined based on their 3D pose data. Furthermore, interaction representation data, which characterizes the interaction between pedestrians and the robot within the target area, is obtained based on these intention features and the robot's state data. Finally, the scene occupancy characteristics, interaction representation data, and robot state data are combined to determine the final navigation action. It is evident that this approach considers not only static obstacles within the target area but also movable pedestrians, specifically taking into account their interaction intentions towards the robot. This allows for navigation that fully considers the social context and pedestrian intentions, achieving socially compliant navigation and improving its accuracy. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a robot navigation method provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a robot navigation method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the data processing flow of a feature extraction module provided in an embodiment of this application; Figure 4 This is a schematic diagram of the constituent modules of a robot navigation system provided in an embodiment of this application; Figure 5 This is a block diagram illustrating the internal structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0021] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to classification." Similarly, the phrases "if determined" or "if classified to [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once classified to [the described condition or event]," or "in response to classification to [the described condition or event]."

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0027] With the development of visual perception technology and intelligent planning, the application of autonomous navigation is becoming increasingly widespread. However, achieving socially compliant navigation for robots operating in dynamic and unstructured environments with dense pedestrian traffic, relying solely on egocentric vision, remains an unsolved challenge. For example, service robots performing delivery tasks in unfamiliar shopping malls. While this task is simple for humans, robots face significant difficulties in moving safely and efficiently in these highly dynamic environments. This requires a comprehensive understanding of the social context from visual input to make appropriate and forward-thinking decisions that conform to social norms.

[0028] Currently, robot navigation typically relies on visual perception technology, which determines the location of obstacles within a region based on corresponding images and then avoids them. However, in complex and crowded environments, such as when there are moving pedestrians in the area, it is difficult to fully consider the social context and pedestrian intentions and achieve navigation that conforms to social norms, which is detrimental to improving the accuracy of robot navigation.

[0029] To move safely and naturally among pedestrians, navigation strategies that conform to social norms should align with human expectations and contextual appropriateness, rather than solely focusing on obstacle avoidance. In some applications, deep reinforcement learning (DRL)-based methods can be used to learn complex social interaction patterns that are difficult to model explicitly. However, the effectiveness of such methods depends on the design of the state space. Through training on empirical data, the learned state transitions can approximate the underlying environmental dynamics of interactions between humans, robots, and their surroundings. Therefore, state embeddings should be decision-sufficient, meaning they should retain social cues that are both expressive and sufficiently accurate to make optimal decisions. Consequently, there is a need to focus on learning scene representations in a first-person perspective to better capture social cues used for training deep reinforcement learning-based policies. Existing methods often oversimplify scene representations to low-level pedestrian states, including position and velocity on a two-dimensional plane. However, these methods ignore the rich social cues in the original images, assuming that social interactions are only related to the pedestrian's trajectory. Specifically, this low-level scene representation ignores subtle but crucial human behaviors that express intent (e.g., human gestures or gaze), as well as the geometric features of the environment and pedestrians. These factors are particularly important in real-world scenarios. Therefore, low-level scenario representations are limited in the laboratory environment of specific participants and differ from real-world scenarios.

[0030] It is evident that robots operating in environments with pedestrian activity need to possess "socially adaptive navigation" capabilities. However, current methods rely on limited scene representations of pedestrian planar trajectories. These methods oversimplify crucial social and environmental contexts, resulting in poor generalization ability to complex and dynamic scenarios in real-world situations.

[0031] To address at least one of the aforementioned technical problems, this application proposes a robot navigation method. The method involves: acquiring robot state data, as well as depth image data and color image data corresponding to a target area; obtaining a bird's-eye view feature map corresponding to the target area based on the depth image data and the color image data, and obtaining scene occupancy features corresponding to the target area based on the bird's-eye view feature map; acquiring pedestrian 3D pose data within the target area based on the color image data, and determining the pedestrian's intent features within the target area based on the pedestrian's 3D pose data; obtaining interaction representation data based on the intent features and the robot state data, wherein the interaction representation data characterizes the interaction information between the pedestrian and the robot within the target area; and obtaining navigation actions for the robot using a deep reinforcement learning network based on the scene occupancy features, the interaction representation data, and the robot state data, and performing motion control on the robot based on the navigation actions.

[0032] Thus, during robot navigation, a bird's-eye view feature map is used to represent the scene occupancy characteristics of the target area. Pedestrians within the target area are also considered, and their intention features are determined based on their 3D pose data. Furthermore, interaction representation data, which characterizes the interaction between pedestrians and the robot within the target area, is obtained based on these intention features and the robot's state data. Finally, the scene occupancy characteristics, interaction representation data, and robot state data are combined to determine the final navigation action. It is evident that this approach considers not only static obstacles within the target area but also movable pedestrians, specifically taking into account their interaction intentions towards the robot. This allows for navigation that fully considers the social context and pedestrian intentions, achieving socially compliant navigation and improving its accuracy.

[0033] Specifically, this application introduces a novel intent-aware scene representation method, namely, using a scene occupancy feature encoded by a Social-Bird's-Eye View (BEV) encoder to comprehensively encode the social context from egocentric visual observation. In this embodiment, data processing is based on two key components: a spatiotemporal encoder for extracting scene occupancy features based on a bird's-eye view (BEV); and an intent interaction transformer (…). The intent interaction transformer infers the pedestrian's intent from their 3D pose to simulate human-computer interaction. These features are unified into a compact state embedding, supporting effective deep reinforcement learning policy training. This solution can be deployed on mobile robot platforms without requiring mapping, enabling socially compliant navigation using only visual sensors, even in unfamiliar and crowded scenarios.

[0034] like Figure 1 As shown in the figure, this application provides a robot navigation method, which specifically includes the following steps: Step S100: Obtain robot status data, as well as depth image data and color image data corresponding to the target area; Step S200: Based on the above depth image data and the above color image data, obtain the bird's-eye view feature map corresponding to the above target area, and obtain the scene occupancy feature corresponding to the above target area based on the above bird's-eye view feature map. Step S300: Based on the above color image data, obtain the three-dimensional pose data of pedestrians in the above target area, and determine the intention features of pedestrians in the above target area based on the above three-dimensional pose data of pedestrians. Step S400: Obtain interaction representation data based on the above-mentioned intention features and robot state data, wherein the interaction representation data is used to characterize the interaction information between pedestrians and the robot in the above-mentioned target area; In step S500, based on the scene occupancy features, the interaction representation data, and the robot state data, a deep reinforcement learning network is used to obtain navigation actions for the robot, and motion control is performed on the robot based on the navigation actions.

[0035] The robot state data includes target direction, target distance, and robot speed. The target direction is the direction in which the robot moves to its target position, and the target distance is the distance between the robot and the target position.

[0036] It should be noted that the steps of the above robot navigation method can be implemented based on a pre-trained robot navigation model to solve the navigation problem of vision-based socially compatible robots in crowded environments. Figure 2 This is a schematic flowchart illustrating a robot navigation method provided in an embodiment of this application. Figure 2 The robot navigation method shown is based on a pre-trained robot navigation model, such as... Figure 2As shown, the robot navigation model described above includes a feature extraction module, a feature fusion module, and a DRL network. In the feature extraction module, scene occupancy features are extracted based on a spatiotemporal encoder, and intent-related features are extracted from the postures of surrounding pedestrians based on an intent interaction transformer. These features are then concatenated to form a scene representation with intent-aware capabilities. These representations are subsequently combined with the robot's state features to form a state embedding for the deep reinforcement learning network. Finally, the deep reinforcement learning network predicts navigation actions to achieve collision-free navigation in dynamic environments.

[0037] Specifically, based on the aforementioned depth image data and color image data, a bird's-eye view feature map corresponding to the target region is obtained, and based on the aforementioned bird's-eye view feature map, scene occupancy features corresponding to the target region are obtained, including: Based on the aforementioned depth image data and color image data, obtain the bird's-eye view feature map corresponding to the aforementioned target area; The above-mentioned bird's-eye view feature map is encoded by a temporal encoder to obtain spatiotemporal bird's-eye view features; The above-mentioned spatiotemporal bird's-eye view features are encoded by a bird's-eye view encoder to obtain the scene occupancy features corresponding to the target area.

[0038] Furthermore, the acquisition of pedestrian three-dimensional pose data within the target area based on the aforementioned color image data includes: Based on the above color image data, pose detection is performed to obtain the two-dimensional human pose of pedestrians in the above target area. Based on the above two-dimensional human pose, the above depth image data, and the preset camera intrinsics, the three-dimensional human pose is obtained. Based on the preset camera extrinsic parameters, the above three-dimensional human pose is transformed into the coordinate system corresponding to the robot to obtain the above three-dimensional pose data of the pedestrian.

[0039] The determination of the pedestrian's intent features within the target area based on the aforementioned pedestrian three-dimensional pose data includes: Based on the aforementioned pedestrian 3D pose data, the intent features corresponding to the pedestrians within the aforementioned target area are determined by an intent transformer, wherein the aforementioned intent features are used to characterize the behavioral intent of the aforementioned pedestrians.

[0040] like Figure 2 As shown, the feature extraction module consists of two key parts: a spatiotemporal encoder and an intent-interaction transformer. Figure 3 This is a schematic diagram of the data processing flow of a feature extraction module provided in an embodiment of this application, such as... Figure 3 As shown, in some application scenarios, the feature extraction module needs to obtain RGB-D data at time t (i.e., data representing color images and depth information). Pedestrian pose obtained from RGB-D data (i.e., two-dimensional human posture) and robot state data The data is then processed to obtain scene occupancy features and interaction representation data. It should be noted that, in this embodiment, the aforementioned RGB-D data can be acquired by a multi-view image acquisition device (e.g., a left camera and a right camera) mounted on the robot, but this is not intended as a specific limitation.

[0041] In order to achieve global perception of the surrounding environment, the embodiments of this application convert the visual observation results from multiple perspectives into BEV feature maps as an intermediate representation for downstream encoding, thereby providing a spatially consistent representation that helps to reason about nearby pedestrians and obstacles in complex environments. Figure 3 Demonstrates the extraction of spatiotemporal BEV features The process of (i.e., spatiotemporal bird's-eye view features) and encoding them into a BEV representation using a 2D convolutional BEV encoder. (i.e., the process of scene occupancy characteristics). Specifically, the RGB-D input is represented as... This includes information from The camera is in continuous Data within a time step, Color image data representing time t, The depth image data represents time t. At each time step, in this embodiment, image features are extracted from all cameras, and then these features are up-dimensioned and stitched together to form a local map to obtain BEV features. (i.e., bird's-eye view feature map). Specifically, a pre-trained ResNet-18 is used as the backbone network. To extract RGB features : ; At the same time, depth image Downsampling is performed to match the spatial resolution of the RGB features: ; Using known camera intrinsic and extrinsic parameters, and downsampled depth images RGB features The robot is transformed from 2D space into a common reference coordinate system (i.e., the robot's own center of inertia at time t) in 3D space. Then, the resulting 3D features are summed and aggregated along the vertical direction to form the BEV features. Subsequently, BEV features from consecutive time steps are correlated to capture spatial-temporal features. First, the features from multiple time steps are aligned using the robot's own motion, and then input into the temporal encoder. In order to obtain space-time BEV characteristics Timing encoder It is implemented by a 3D convolutional network and pre-trained on the nuScenes dataset. Finally, the spatial-temporal BEV features are fed into the BEV encoder to extract scene representations. (i.e., scene occupancy characteristics): ; in, It is a two-dimensional convolutional network with residual connections and two-dimensional pooling layers.

[0042] Furthermore, the above-mentioned acquisition of interaction representation data based on the aforementioned intent features and robot state data includes: The robot state data is encoded to obtain the robot state embedding; Based on the aforementioned intent features and robot state embedding, the aforementioned interaction representation data is obtained through the interaction transformer.

[0043] In order to capture the behavioral intentions conveyed by human posture, this application embodiment sets up... The module, its overall process is as follows Figure 3 As shown, this module requires 3D human pose data (i.e., pedestrian 3D pose data) from surrounding pedestrians. To acquire this data, YOLO is first used to detect 2D human poses from RGB images, and then these are converted into 3D human poses using camera intrinsics and depth images. Finally, camera extrinsic parameters are used to transform the 3D human poses into the robot's coordinate system. It consists of four parts: attitude encoder, A state encoder, intent transformer, and interaction transformer are used. Since each joint is initially represented only by three-dimensional coordinates, this embodiment employs a pose encoder to generate a higher-dimensional embedding for each joint. This also enables the model to better represent undetected joints and handle incomplete poses more robustly. ; in, It is pose embedding (i.e., human 3D pose data). It is an attitude encoder, which is a multilayer perceptron with ReLU activation function, and its parameters are expressed as follows: Human posture reflects a pedestrian's next action. By modeling the relationships between all joints, behavioral intentions such as moving forward, turning, or yielding can be inferred. To this end, an IntentFormer is implemented, utilizing Multi-Head Self-Attention (MHSA) to capture these relationships, enabling the model to understand implicit behavioral intentions. Furthermore, it includes a Feed-Forward Network (FFN), residual connections, and Layer Normalization (LN). Finally, Attention Pooling (AttnPool) is applied to adaptively weigh the importance of different joints, improving the model's understanding of human actions and intentions. The IntentFormer outputs implicit intention features. Its expression can be written as: ; .

[0044] Furthermore, based on the aforementioned scene occupancy characteristics, interaction representation data, and robot state data, a deep reinforcement learning network is used to obtain navigation actions for the robot, including: The robot state data is encoded to obtain state encoded data; By concatenating the above-mentioned scene occupancy features, the above-mentioned interaction representation data, and the above-mentioned state encoding data, the concatenated features are obtained. The spliced ​​features are fused using a fusion network to obtain the state embedding. The above state is embedded into the deep reinforcement learning network to obtain the navigation action output by the deep reinforcement learning network.

[0045] Specifically, after acquiring the intent features of each pedestrian, a module is needed to associate the robot with these pedestrians to capture the robot's interactions with them. To this end, an InteractFormer is set up based on Multi-Head Cross-Attention (MHCA), where the robot's state embedding serves as a query to focus on the features of surrounding pedestrians, enabling the robot to form a global understanding of their behavioral intentions. The InteractFormer outputs interaction representation data. , can be represented as: ; ; in, , indicating robot state embedding. yes The state encoder can be implemented using a multilayer perceptron with a ReLU activation function, and its parameters are... express.

[0046] This application embodiment fuses an intent-aware scene representation with the robot's state to obtain the state embedding of the DRL. First, the robot's state... It will be encoded into state-coded data by a state encoder. : ; ; in, Indicates the direction of the target; This refers to the target distance, which is the distance between the robot and the target location. The target location is the destination of the robot's movement and can be determined according to actual needs. It's the robot's speed. It is a state encoder, implemented using a multilayer perceptron with ReLU activation function, and its parameters are... express.

[0047] Then, by combining the intent-aware scene representation with the encoded robot state... By merging them, we obtain the splicing features. As a state embedding in DRL: ; in, The fusion network is represented by a multilayer perceptron with ReLU activation function, and its parameters are... express.

[0048] like Figure 2 As shown, by inputting the above-mentioned spliced ​​features into the DRL network, navigation actions can be obtained, and robot navigation control can be performed based on these actions. It should be noted that... Figure 2 The learning process for the DRL network in the middle only occurs during model training. In some application scenarios, online training can be performed based on the Proximal Policy Optimization (PPO) algorithm. The RGB backbone network of the spatiotemporal encoder. and timing encoder The modules are pre-trained on an external dataset and remain frozen during deep reinforcement learning training. The remaining modules are optimized together with the deep reinforcement learning policy. This helps to enhance the robustness of the obtained scene representation in unknown environments.

[0049] Thus, during robot navigation, a bird's-eye view feature map is used to represent the scene occupancy characteristics of the target area. Pedestrians within the target area are also considered, and their intention features are determined based on their 3D pose data. Furthermore, interaction representation data, which characterizes the interaction between pedestrians and the robot within the target area, is obtained based on these intention features and the robot's state data. Finally, the scene occupancy characteristics, interaction representation data, and robot state data are combined to determine the final navigation action. It is evident that this approach considers not only static obstacles within the target area but also movable pedestrians, specifically taking into account their interaction intentions towards the robot. This allows for navigation that fully considers the social context and pedestrian intentions, achieving socially compliant navigation and improving its accuracy.

[0050] Specifically, this application incorporates a novel visual encoder into a socially compliant navigation strategy utilizing a camera. It is trained end-to-end in a simulated environment and deployed with zero samples in unseen crowded scenarios. The intent-aware scene representation proposed in this application implicitly captures behavior and environmental context. Encoding the BEV representation densely constructs scene occupancy features, and using an attention mechanism to infer human intent from 3D position prioritizes human-computer interaction, thus improving robot navigation performance.

[0051] like Figure 4 As shown in the figure, corresponding to the above-described robot navigation method, this application embodiment also provides a robot navigation system, which includes: The data acquisition module 410 is used to acquire robot status data, as well as depth image data and color image data corresponding to the target area; The scene occupancy feature acquisition module 420 is used to acquire the bird's-eye view feature map corresponding to the target area based on the depth image data and the color image data, and to acquire the scene occupancy feature corresponding to the target area based on the bird's-eye view feature map. The intent feature acquisition module 430 is used to acquire pedestrian three-dimensional pose data in the target area based on the above-mentioned color image data, and determine the intent features corresponding to the pedestrian in the target area based on the above-mentioned pedestrian three-dimensional pose data. The interaction representation data acquisition module 440 is used to acquire interaction representation data based on the above-mentioned intention features and the above-mentioned robot state data, wherein the interaction representation data is used to characterize the interaction information between the pedestrian and the robot in the above-mentioned target area. The navigation module 450 is used to obtain navigation actions for the robot through a deep reinforcement learning network based on the scene occupancy characteristics, the interaction representation data, and the robot state data, and to perform motion control on the robot based on the navigation actions.

[0052] Thus, during robot navigation, a bird's-eye view feature map is used to represent the scene occupancy characteristics of the target area. Pedestrians within the target area are also considered, and their intention features are determined based on their 3D pose data. Furthermore, interaction representation data, which characterizes the interaction between pedestrians and the robot within the target area, is obtained based on these intention features and the robot's state data. Finally, the scene occupancy characteristics, interaction representation data, and robot state data are combined to determine the final navigation action. It is evident that this approach considers not only static obstacles within the target area but also movable pedestrians, specifically taking into account their interaction intentions towards the robot. This allows for navigation that fully considers the social context and pedestrian intentions, achieving socially compliant navigation and improving its accuracy.

[0053] It should be noted that the specific structure and implementation of the above-mentioned robot navigation system and its various modules or units can be referred to the corresponding descriptions in the above method embodiments, and will not be repeated here.

[0054] It should be noted that the division of the various modules of the above-mentioned robot navigation system is not unique and is not intended as a specific limitation.

[0055] Based on the above embodiments, this application also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown. The terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps of any of the robot navigation methods described above. The display screen can be a liquid crystal display (LCD) or an e-ink display.

[0056] Those skilled in the art will understand that Figure 5 The block diagram shown is only a partial structural diagram related to the solution of this application and does not constitute a limitation on the terminal on which the solution of this application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0057] In one embodiment, a terminal is provided, the terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any robot navigation method provided in the embodiments of this application.

[0058] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any robot navigation method provided in this application.

[0059] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0060] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0061] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0062] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0063] In the embodiments provided in this application, it should be understood that the disclosed systems / terminal devices and methods can be implemented in other ways. For example, the system / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0064] If the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0065] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions are not in essence a departure from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of robot navigation, characterized by, The method comprises: obtaining robot state data, and depth image data and color image data corresponding to a target region; obtaining an aerial view perspective feature map corresponding to the target region according to the depth image data and the color image data, and obtaining scene occupancy features corresponding to the target region based on the aerial view perspective feature map; obtaining pedestrian three-dimensional pose data in the target region according to the color image data, and determining intention features corresponding to pedestrians in the target region based on the pedestrian three-dimensional pose data; obtaining interaction representation data according to the intention features and the robot state data, wherein the interaction representation data is used to represent interaction information between pedestrians in the target region and the robot; obtaining a navigation action for the robot through a deep reinforcement learning network according to the scene occupancy features, the interaction representation data and the robot state data, and performing motion control on the robot according to the navigation action.

2. The method of claim 1, wherein, The robot state data comprises a target direction, a target distance and a robot speed, wherein the target direction is a direction of a target position of the robot motion, and the target distance is a distance between the robot and the target position.

3. The method of Claim 1, wherein, The method comprises: obtaining an aerial view perspective feature map corresponding to the target region according to the depth image data and the color image data; encoding the aerial view perspective feature map through a time sequence encoder to obtain a space-time aerial view feature; encoding the space-time aerial view feature through an aerial view encoder to obtain the scene occupancy features corresponding to the target region.

4. The method of Claim 1, wherein, The method comprises: performing pose detection based on the color image data to obtain a two-dimensional human body pose of pedestrians in the target region; obtaining a three-dimensional human body pose according to the two-dimensional human body pose, the depth image data and a preset camera intrinsic parameter; converting the three-dimensional human body pose into a coordinate system corresponding to the robot according to a preset camera extrinsic parameter to obtain the pedestrian three-dimensional pose data.

5. The method of robot navigation of claim 4, wherein, The method comprises: determining the intention features corresponding to the pedestrians in the target region through an intention transformer according to the pedestrian three-dimensional pose data, wherein the intention features are used to represent behavior intentions corresponding to the pedestrians.

6. The method of Claim 1, wherein, The method comprises: encoding the robot state data to obtain a robot state embedding; obtaining the interaction representation data through an interaction transformer according to the intention features and the robot state embedding.

7. The method of robot navigation according to any one of claims 1 to 6, characterized in that, The method comprises: Encode the robot state data to obtain state encoding data; Splice the scene occupancy feature, the interaction representation data and the state encoding data to obtain spliced features; Perform feature fusion processing on the spliced features through a fusion network to obtain state embedding; Input the state embedding into the deep reinforcement learning network to obtain a navigation action output by the deep reinforcement learning network.

8. A robot navigation system characterized by, The system comprises: A data acquisition module configured to acquire robot state data, and depth image data and color image data corresponding to a target region; A scene occupancy feature acquisition module configured to acquire an overhead view feature map corresponding to the target region based on the depth image data and the color image data, and acquire scene occupancy features corresponding to the target region based on the overhead view feature map; An intention feature acquisition module configured to acquire pedestrian three-dimensional pose data in the target region based on the color image data, and determine intention features corresponding to pedestrians in the target region based on the pedestrian three-dimensional pose data; An interaction representation data acquisition module configured to acquire interaction representation data based on the intention features and the robot state data, wherein the interaction representation data is used to represent interaction information between pedestrians in the target region and the robot; A navigation module configured to acquire a navigation action for the robot through a deep reinforcement learning network based on the scene occupancy features, the interaction representation data and the robot state data, and perform motion control on the robot based on the navigation action.

9. A terminal, characterized by comprising: The terminal comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the robot navigation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the robot navigation method according to any one of claims 1 to 7.

Images