Robot space navigation method, robot space navigation device, and robot

Abstract

The application provides a robot space navigation method, a robot space navigation device and a robot, and relates to the technical field of robot vision navigation. The method comprises the following steps: acquiring a navigation instruction of a robot in a three-dimensional indoor scene for a target object category and a current semantic map of the three-dimensional indoor scene; determining a candidate target point for the target object category from the current semantic map; if the robot is in a collision state at a current long-term target point, determining a target point from the current semantic map; determining the target point as a next long-term target point; generating a short-term target point of the robot at a next time step by using a preset local strategy according to the next long-term target point; and generating an action instruction of the robot at the next time step according to the short-term target point, so as to control the robot to navigate to a target object corresponding to the target object category. The application can effectively break through the obstruction of a collision trap to navigation exploration and guarantee the continuous advancement of a navigation task.

Description

Technical Field

[0001] This application relates to the field of robot vision navigation technology, and more specifically, to a robot space navigation method, a robot space navigation device, and a robot. Background Technology

[0002] Embodied AI, an important research direction in the field of artificial intelligence, aims to enable intelligent agents to autonomously perceive, make decisions, and execute tasks in physical or virtual environments. Zero-Shot Object Navigation (ZSO) is one of the core challenges in this field. ZSO requires an intelligent agent to accurately locate and navigate to a designated target object in an unknown environment without specific environmental training or additional training data. This technology is widely applicable to service robots, autonomous navigation devices, and other scenarios, and has extremely high practical value.

[0003] Currently, research on zero-shot target navigation technology mainly revolves around three core aspects: semantic map construction, frontier selection, and large language model (LLM) decision-making, resulting in several baseline methods. For example, the L3MVN method constructs a semantic map and combines it with a large language model to score and select frontier regions to determine the navigation target point; the VLFM method proposes a visual-language frontier map, which guides the exploration direction by evaluating the semantic value of frontier regions; in addition, some methods attempt to improve navigation performance by integrating techniques such as vision-language model (VLM), reinforcement learning, or imitation learning. These existing technologies have promoted the development of zero-shot target navigation to some extent, but they still face many key problems in practical applications, resulting in poor navigation performance in complex environments and failing to meet actual needs.

[0004] For example, existing technologies are prone to causing agents to fall into collision traps. In unknown environments, agents often become trapped in confined spaces or areas with dense obstacles due to improper long-term goal selection, resulting in repeated collisions. Simultaneously, when the local path planner is unable to calculate a reachable path from the current position to the long-term goal, it hinders the agent's further exploration and severely impacts the progress of the navigation task. These shortcomings make it difficult for existing zero-shot target navigation methods to achieve stable and efficient target navigation in multi-story indoor environments with complex obstacle distributions, limiting the practical application of zero-shot target navigation technology. Summary of the Invention

[0005] This application addresses the shortcomings of the prior art by providing a robot space navigation method, a robot space navigation device, and a robot, in order to solve the problems existing in the prior art.

[0006] The technical solution adopted in the embodiments of this application is as follows: In a first aspect, embodiments of this application provide a robot space navigation method, including: The robot acquires navigation instructions for target object categories in a 3D indoor scene and the current semantic map of the 3D indoor scene; the current semantic map is the semantic map constructed by the robot at the current time step. Based on the navigation instructions for the target object category, a preset visual language model is used to determine candidate target points for the target object category from the current semantic map; Based on the robot's current state at the current time step, a collision assist is used to determine whether the robot is in a collision state at the current long-term target point. If the robot is in a collision state at the current long-term target point, the target point is determined from the current semantic map. A preset global strategy is adopted to determine the target point as the next long-term target point; Based on the next long-term target point, a preset local strategy is used to generate the robot's short-term target point at the next time step; Based on the short-term target point, the robot's action command is generated at the next time step to control the robot to navigate to the target object corresponding to the target object category.

[0007] In one embodiment, determining candidate target points for the target object category from the current semantic map using a preset visual language model based on navigation instructions for the target object category includes: Based on the navigation instructions for the target object category, the target search module in the preset visual language model is used to determine the current optimal boundary target point from the current semantic map; The detection aid in the preset visual language model is used to determine whether the target object category is in the current semantic map. If the target object category is in the current semantic map, the current optimal boundary target point is determined as the candidate target point.

[0008] In one embodiment, determining the target point from the current semantic map includes: Obtain the largest connected region of the navigable area in the current semantic map; The centroid of the largest connected region is determined as the target point.

[0009] In one embodiment, obtaining the maximum connected region of the navigable area in the current semantic map includes: The navigable region in the current semantic map is decomposed to obtain multiple connected regions, and the size of each connected region is determined. The largest connected region is determined from the plurality of connected regions based on the size of each of the connected regions.

[0010] In one embodiment, the method further includes: Obtain the distance between the robot's current position at the current time step and the next long-term target point; If the distance is less than the preset sleep threshold, and the current optimal boundary target point is the same as the optimal boundary target point of the previous time step, then a preset global strategy is adopted to control the preset visual language model to be in a sleep state for multiple sleep time steps, so that the robot can freely explore the three-dimensional indoor scene.

[0011] In one embodiment, before acquiring the robot's navigation instructions for the target object category in a 3D indoor scene and the current semantic map of the 3D indoor scene, the method further includes: Acquire the visual data and current pose data collected by the robot at the current time step; The current semantic map is constructed using the visual data and the current pose data.

[0012] In one embodiment, the current pose data includes: the robot's current position and past positions at the current time step; constructing the current semantic map using the visual data and the current pose data includes: Using the visual data, a map of the current obstacles, a map of the currently explored area, and semantic maps of various preset object categories are constructed respectively. The current obstacle map, the currently explored area map, the semantic map of various preset object categories, the robot's current position, and past positions are stored in multiple channels of a preset image to obtain the current semantic map.

[0013] In one embodiment, the method further includes: Add a fake target semantic map to the current semantic map.

[0014] Secondly, embodiments of this application provide a robot space navigation device, comprising: The acquisition module is used to acquire the robot's navigation instructions for the target object category in the 3D indoor scene and the current semantic map of the 3D indoor scene; the current semantic map is the semantic map constructed by the robot at the current time step; The first determining module is used to determine candidate target points for the target object category from the current semantic map based on the navigation instructions for the target object category and using a preset visual language model; The collision detection module is used to determine whether the robot is in a collision state at the current long-term target point based on the robot's current state at the current time step and using a collision assist. If the robot is in a collision state at the current long-term target point, the target point is determined from the current semantic map. The second determining module is used to determine the target point as the next long-term target point by adopting a preset global strategy. The first generation module is used to generate the robot's short-term target point at the next time step based on the next long-term target point and using a preset local strategy. The second generation module is used to generate the robot's action instructions at the next time step based on the short-term target point, so as to control the robot to navigate to the target object corresponding to the target object category.

[0015] Thirdly, embodiments of this application provide a robot, which includes at least: a robot body and a controller disposed within the robot body, the controller being used to execute the robot space navigation method described in any of the above embodiments.

[0016] The beneficial effects of this application are: it provides a robot spatial navigation method that uses a collision assist device to determine the robot's current state and the reachability of the current long-term goal point in real time, accurately identifying two types of collision trap scenarios: the agent getting stuck in a narrow / obstacle-dense area due to improper long-term goal selection, and the local path planner being unable to calculate a reachable path. It also triggers the operation of re-determining the goal point from the semantic map, completely avoiding the problem of the agent getting stuck due to the mistake in goal point selection or the failure of path planning. It effectively eliminates the obstacle of collision traps to navigation exploration and ensures the continuous progress of the navigation task. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the overall architecture of an embodiment of this application; Figure 2 One of the flowcharts of the robot space navigation method provided in the embodiments of this application; Figure 3 A second schematic flowchart illustrating the robot spatial navigation method provided in this application embodiment; Figure 4 The third schematic flowchart of the robot space navigation method provided in the embodiments of this application; Figure 5 The fourth schematic flowchart of the robot space navigation method provided in the embodiments of this application; Figure 6 Fifth schematic flowchart of the robot space navigation method provided in the embodiments of this application; Figure 7 Sixth schematic flowchart of the robot space navigation method provided in the embodiments of this application; Figure 8 Seventh schematic flowchart of the robot space navigation method provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the robot space navigation device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the controller provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0023] This application provides a robot space navigation method, which can be generated by any controller with computing and processing capabilities. Figure 1 This is a schematic diagram of the overall architecture of an embodiment of this application. Figure 1 This paper presents a closed-loop framework for zero-shot navigation of robots: The robot first collects visual data of a 3D indoor scene and its own pose data through sensors. After semantic segmentation to extract object information, it combines this with navigation instructions based on the target object category. The target search module of the Visual Language Model (VLM) locates potential targets. Simultaneously, this data is integrated into a semantic map containing scene, object, and robot position information. Then, an exploration aid driven by a Large Language Model (LLM), a collision aid, and a detection aid driven by VLM are used to address the issues of exploration range, collision traps, and target misidentification, respectively. Finally, a global strategy determines a long-term target point based on the semantic map and aid outputs, while local strategies break it down into short-term targets and generate action instructions. The robot continuously collects new data while executing these instructions, cyclically completing the navigation process. The following section combines... Figure 1 The robot space navigation method provided in this application is illustrated with specific examples through multiple accompanying figures and examples.

[0024] Figure 2 This is one of the flowcharts illustrating the robot space navigation method provided in the embodiments of this application, such as... Figure 2 As shown, the method includes: S101. Obtain the robot's navigation instructions for the target object category in the 3D indoor scene and the current semantic map of the 3D indoor scene.

[0025] The system acquires navigation commands for the robot within a 3D indoor scene, targeting specific object categories, as well as the current semantic map of the 3D indoor scene. This current semantic map is a multi-channel semantic map constructed by the robot at the current time step, based on visual data from the 3D indoor scene and its own pose data. It includes core information such as obstacle maps, explored area maps, semantic maps of various object categories, and the robot's position. Target object categories can be, for example, one of the common objects in the 3D indoor scene: a table, chair, bed, or sofa.

[0026] S102. Based on the navigation instructions for the target object category, a preset visual language model is used to determine candidate target points for the target object category from the current semantic map.

[0027] Based on the navigation instructions for the target object category, a preset visual language model (VLM) is used to filter and determine candidate target points for the target object category from the current semantic map. The candidate target points are potential navigation points that have the possibility of the target object existing.

[0028] S103. Based on the robot's current state at the current time step, a collision assist is used to determine whether the robot is in a collision state at the current long-term target point. If the robot is in a collision state at the current long-term target point, the target point is determined from the current semantic map.

[0029] The robot's motion state and path planning state at the current time step are collected as the current state. The current state is then input into the collision assistant, which determines whether the robot is in a collision state at the current candidate target point.

[0030] Collision states include two situations: the robot is trapped in a fixed position and collides with obstacles multiple times, and the local path planner is unable to calculate a reachable path from the current position to the candidate target point. If a collision state is determined, a new target point is re-determined from the navigable area of ​​the current semantic map.

[0031] S104. Adopt a preset global strategy to determine the target point as the next long-term target point.

[0032] The target point, which is redefined in the collision state, is input into the preset global strategy. The global strategy determines the target point as the robot's next long-term target point in the next time step based on the exploration needs and target localization needs of the 3D indoor scene.

[0033] S105. Based on the next long-term target point, a preset local strategy is adopted to generate the robot's short-term target point for the next time step.

[0034] Using the next long-term target point as the navigation endpoint and the robot's current position as the navigation starting point, a preset local strategy is used for point-to-point path planning to generate the robot's short-term target point in the next time step. The short-term target point is a stage execution point on the navigation path between the navigation starting point and the long-term target point.

[0035] S106. Based on the short-term target point, generate the robot's action instructions for the next time step to control the robot to navigate to the target object corresponding to the target object category.

[0036] Based on the short-term target point and combined with the robot's motion space (move forward, turn left, turn right, look up, look down, stop), the robot's motion command for the next time step is generated, and the robot is controlled to perform motion operations according to the motion command, gradually navigating to the target object corresponding to the target object category.

[0037] In summary, this embodiment provides a robot spatial navigation method that uses a collision assist device to determine the robot's current state and the reachability of its current long-term target point in real time. It accurately identifies two types of collision trap scenarios: the agent getting stuck in narrow / obstacle-dense areas due to improper long-term target selection, and the local path planner being unable to calculate a reachable path. It then triggers the operation of re-determining the target point from the semantic map, completely avoiding the problem of the agent getting trapped due to target point selection errors or path planning failures. This effectively eliminates the obstacles that collision traps pose to navigation exploration and ensures the continuous progress of the navigation task.

[0038] Figure 3 This is a second schematic flowchart of the robot space navigation method provided in the embodiments of this application, as shown below. Figure 3 As shown, step S102, which involves determining candidate target points for a specific target object category from the current semantic map using a preset visual language model based on navigation instructions for that category, includes: S201. Based on the navigation instructions for the target object category, the target search module in the preset visual language model is used to determine the current optimal boundary target point from the current semantic map.

[0039] Based on the navigation instructions for the target object category, the target search module integrated in the preset visual language model is invoked. The target search module subtracts the obstacle map from the explored area map of the current semantic map, highlights the potential next exploration area, identifies and removes small areas in the map, and retains only important areas as potential exploration targets. The center of the remaining area is the set of boundary points.

[0040] These boundary point sets are scored and filtered to determine the boundary point with the highest score as the current optimal boundary target point. The scoring and filtering process could involve evaluating each boundary point in the set, balancing the cost of reaching the target with the expected utility, determining its feasibility as an exploration destination, and obtaining a score for each boundary point set.

[0041] S202. Using the detection assistant in the preset visual language model, determine whether the target object category is in the current semantic map. If the target object category is in the current semantic map, determine the current optimal boundary target point as a candidate target point.

[0042] The detection assistant in the preset visual language model is activated. The detection assistant extracts the semantic mask of objects and regional scene information in the current semantic map. Through natural language prompt verification (for example, based on all objects and room information in the image, determine whether there is a table in the image, and only output 'yes' or 'no'), it is determined whether the target object category exists in the 3D indoor scene region corresponding to the current semantic map.

[0043] If the detection aid determines that the target object category is in the current semantic map, then the current optimal boundary target point determined in step S201 will be used as a candidate target point for the target object category.

[0044] This embodiment relies on the VLM target search module to filter the optimal boundary target points. By combining the region and object association information of the semantic map, it improves the matching degree between candidate target points and navigation targets, narrows the exploration range, and reduces invalid exploration behavior. Furthermore, the VLM detection assistant performs secondary verification of the target object category. By utilizing the semantic understanding capability of visual-language fusion, it effectively filters semantic segmentation misjudgment information, reduces the probability of invalid candidate points caused by false targets, and improves the effectiveness and accuracy of candidate target points.

[0045] Figure 4 This is the third flowchart illustrating the robot space navigation method provided in the embodiments of this application, as shown below. Figure 4 As shown, S103, determining the target point from the current semantic map, includes: S301. Obtain the largest connected region of the navigable area in the current semantic map.

[0046] Extract channel data representing the navigable area map from the current semantic map, and determine the largest connected region of the navigable area in the current semantic map based on the channel data. Specifically, decompose and filter the navigable area to obtain the navigable connected region with the largest area.

[0047] Specifically, Figure 5 This is the fourth flowchart illustrating the robot space navigation method provided in the embodiments of this application, as shown below. Figure 5 As shown, S301 includes: S401. Decompose the navigable region in the current semantic map to obtain multiple connected regions, and determine the size of each connected region.

[0048] The channel tensor data of the navigable region is extracted from the current semantic map. The navigable region is then decomposed into multiple independent connected regions using a clustering algorithm. At the same time, the pixel area or spatial area of ​​each connected region is calculated to obtain the size parameters of each connected region.

[0049] S402. Determine the largest connected region from among the multiple connected regions based on the size of each connected region.

[0050] The size parameters of each connected region are compared and sorted, and the connected region with the largest size parameter is selected as the largest connected region of the navigable area in the current semantic map.

[0051] S302. Determine the centroid of the largest connected region as the target point.

[0052] The centroid calculation method is used to solve for the geometric centroid of the largest connected region. The coordinate position of the centroid is determined as the target point of the robot in the collision state, so as to guide the robot to move to an open navigable area and escape the collision state.

[0053] This embodiment accurately extracts the largest connected region of the navigable area from the semantic map, and uses the centroid of the largest connected region as the target point to ensure that the robot moves to an open navigable space, avoiding getting stuck in a narrow local area again. This provides a better spatial basis for subsequent long-term goal planning and ensures the continuous progress of the navigation task.

[0054] Figure 6 This is the fifth flowchart illustrating the robot space navigation method provided in the embodiments of this application, as shown below. Figure 6 As shown, the method of this application further includes: S501. Obtain the distance between the robot's current position at the current time step and the next long-term target point.

[0055] Obtain the robot's current position coordinates at the current time step, and the coordinates of the next long-term target point determined in step S104. Use the exploration assistant to calculate the spatial distance between the two according to the Euclidean distance calculation formula.

[0056] S502. If the distance is less than the preset sleep threshold, and the current optimal boundary target point is the same as the optimal boundary target point of the previous time step, then the preset global strategy is adopted to control the preset visual language model to be in a sleep state for multiple sleep time steps, so that the robot can freely explore in the three-dimensional indoor scene.

[0057] The spatial distance is compared with a preset sleep threshold, and it is determined whether the current optimal boundary target point is the same as the optimal boundary target point determined by the robot in the previous time step. If the spatial distance is less than the preset sleep threshold and the optimal boundary target points in the previous and next time steps are the same, a sleep control command is sent by a preset global strategy to control the preset visual language model to be in a sleep state for a preset number of sleep time steps (e.g., 20 time steps), pausing the target point selection operation dominated by the visual language model, so that the robot can freely explore the three-dimensional indoor scene in a way without a fixed target, thus expanding the scene exploration range.

[0058] This embodiment controls the VLM to hibernate and allows the robot to explore freely, breaking the exploration range limitations caused by long-term goal stagnation, fully covering unknown three-dimensional indoor scenes, improving the comprehensiveness and efficiency of exploration, and increasing the probability of discovering target objects.

[0059] Figure 7 This is the sixth flowchart illustrating the robot space navigation method provided in the embodiments of this application, as shown below. Figure 7As shown, before executing S101 to obtain the robot's navigation instructions for the target object category and the current semantic map of the 3D indoor scene, the method of this application further includes: S601. Acquire the visual data and current pose data collected by the robot at the current time step.

[0060] The robot uses an RGB-D camera to collect visual data of the 3D indoor scene at the current time step. The visual data includes color RGB information and depth information. At the same time, the robot uses an odometry sensor to collect its current pose data at the current time step. The pose data includes the robot's position and attitude information.

[0061] S602. Construct the current semantic map using visual data and current pose data.

[0062] Visual data and current pose data are input into the semantic map construction module. The construction module combines geometric mapping and channel fusion technology to fuse the visual data and pose information, thereby constructing the current semantic map of the robot at the current time step.

[0063] Current pose data can include, for example, the robot's current position and past positions at the current time step, such as... Figure 8 As shown, S602 specifically includes: S701. Using visual data, construct a map of the current obstacles, a map of the currently explored area, and semantic maps of various preset object categories.

[0064] The collected RGB-D visual data is processed by semantic segmentation. Obstacles, explored areas, and multiple preset object categories in the visual data are identified by the semantic segmentation model. Corresponding semantic maps of the current obstacle, the current explored area, and multiple preset object categories are constructed. Each map is stored in the form of tensor channels.

[0065] S702. Store the current obstacle map, the current explored area map, the semantic map of various preset object categories, the robot's current position and past positions into multiple channels of the preset image to obtain the current semantic map.

[0066] A three-dimensional tensor is preset as the basic carrier of the semantic map. The current obstacle map, the current explored area map, and semantic maps of various preset object categories are stored in different channels of the basic carrier. At the same time, the current position and past position information in the robot's current pose data are extracted, and after coordinate mapping, they are stored in the designated channel of the basic carrier. The current semantic map is obtained by fusing multi-channel data.

[0067] Optionally, an independent channel can be added to the basic carrier to construct and store a false target semantic map. The false target semantic map is used to record the target object information misidentified during the semantic segmentation process for reference in subsequent navigation decisions.

[0068] This embodiment uses a multi-channel tensor carrier to store various types of information, realizing structured and modular management of data. This not only improves the efficiency of information extraction but also has good scalability, allowing for the flexible addition of new information channels.

[0069] This application also provides a robot, which includes at least a robot body and a controller disposed within the robot body. The controller is used to execute the robot space navigation method provided in any of the above embodiments.

[0070] The following will continue to explain the apparatus, device and storage medium for implementing the robot space navigation method provided in any of the above embodiments of this application. The specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For the sake of brevity, the parts not mentioned in the following embodiments can be referred to the corresponding content in the method embodiments.

[0071] Figure 9 This is a schematic diagram of the structure of the robot space navigation device provided in the embodiments of this application, as shown below. Figure 9 As shown, this application also provides a robot space navigation device, including: The acquisition module 10 is used to acquire the navigation instructions of the robot for the target object category in the three-dimensional indoor scene and the current semantic map of the three-dimensional indoor scene; the current semantic map is the semantic map constructed by the robot at the current time step.

[0072] The first determining module 20 is used to determine candidate target points for the target object category from the current semantic map based on the navigation instructions for the target object category and using a preset visual language model.

[0073] The collision detection module 30 is used to determine whether the robot is in a collision state at the current long-term target point based on the robot's current state at the current time step using a collision assist. If the robot is in a collision state at the current long-term target point, the target point is determined from the current semantic map.

[0074] The second determining module 40 is used to determine the target point as the next long-term target point by adopting a preset global strategy.

[0075] The first generation module 50 is used to generate the robot's short-term target point in the next time step by adopting a preset local strategy based on the next long-term target point.

[0076] The second generation module 60 is used to generate the robot's action instructions at the next time step based on the short-term target point, so as to control the robot to navigate to the target object corresponding to the target object category.

[0077] Optionally, the first determining module 20 is configured to determine the current optimal boundary target point from the current semantic map using the target search module in the preset visual language model according to the navigation instruction of the target object category; and to determine whether the target object category is in the current semantic map using the detection assistant in the preset visual language model. If the target object category is in the current semantic map, the current optimal boundary target point is determined as the candidate target point.

[0078] Optionally, the collision detection module 30 is used to obtain the largest connected region in the navigable area of ​​the current semantic map; and determine the centroid of the largest connected region as the target point.

[0079] Optionally, the collision detection module 30 is used to decompose the navigable region in the current semantic map to obtain multiple connected regions and determine the size of each connected region; and determine the largest connected region from the multiple connected regions based on the size of each connected region.

[0080] Optionally, the device further includes a hibernation module for obtaining the distance between the robot's current position at the current time step and the next long-term target point; if the distance is less than a preset hibernation threshold and the current optimal boundary target point is the same as the optimal boundary target point of the previous time step, a preset global strategy is adopted to control the preset visual language model to be in a hibernation state for multiple hibernation time steps, so that the robot can freely explore in the three-dimensional indoor scene.

[0081] Optionally, the acquisition module 10 is used to acquire the visual data and current pose data collected by the robot at the current time step; and to construct the current semantic map using the visual data and the current pose data.

[0082] Optionally, the current pose data includes: the robot's current position and past positions at the current time step. The acquisition module 10 is used to construct a current obstacle map, a currently explored area map, and semantic maps for multiple preset object categories using the visual data; and to store the current obstacle map, the currently explored area map, the semantic maps for multiple preset object categories, the robot's current position, and past positions into multiple channels of a preset image to obtain the current semantic map.

[0083] Optionally, the acquisition module 10 is also used to add a false target semantic map to the current semantic map.

[0084] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0085] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0086] Figure 10 This is a schematic diagram of the controller provided in the embodiments of this application, such as... Figure 10 As shown, this application also provides a controller, including a processor 100, a storage medium 200 and a bus 300. The storage medium stores program instructions executable by the processor. When the controller is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the robot space navigation method described in any of the above embodiments.

[0087] This application also provides a readable storage medium storing program instructions, which, when executed by a processor, implement the robot space navigation method described in any of the above embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0089] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0091] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot space navigation method, characterized in that, include: The robot acquires navigation instructions for target object categories in a 3D indoor scene and the current semantic map of the 3D indoor scene; the current semantic map is the semantic map constructed by the robot at the current time step. Based on the navigation instructions for the target object category, a preset visual language model is used to determine candidate target points for the target object category from the current semantic map; Based on the robot's current state at the current time step, a collision assist is used to determine whether the robot is in a collision state at the current long-term target point. If the robot is in a collision state at the current long-term target point, the target point is determined from the current semantic map. A preset global strategy is adopted to determine the target point as the next long-term target point; Based on the next long-term target point, a preset local strategy is used to generate the robot's short-term target point at the next time step; Based on the short-term target point, the robot's action command is generated at the next time step to control the robot to navigate to the target object corresponding to the target object category.

2. The method according to claim 1, characterized in that, The step of determining candidate target points for the target object category from the current semantic map using a preset visual language model based on navigation instructions for the target object category includes: Based on the navigation instructions for the target object category, the target search module in the preset visual language model is used to determine the current optimal boundary target point from the current semantic map; The detection aid in the preset visual language model is used to determine whether the target object category is in the current semantic map. If the target object category is in the current semantic map, the current optimal boundary target point is determined as the candidate target point.

3. The method according to claim 1, characterized in that, Determining the target point from the current semantic map includes: Obtain the largest connected region of the navigable area in the current semantic map; The centroid of the largest connected region is determined as the target point.

4. The method according to claim 3, characterized in that, The step of obtaining the maximum connected region of the navigable area in the current semantic map includes: The navigable region in the current semantic map is decomposed to obtain multiple connected regions, and the size of each connected region is determined. The largest connected region is determined from the plurality of connected regions based on the size of each of the connected regions.

5. The method according to claim 2, characterized in that, The method further includes: Obtain the distance between the robot's current position at the current time step and the next long-term target point; If the distance is less than the preset sleep threshold, and the current optimal boundary target point is the same as the optimal boundary target point of the previous time step, then a preset global strategy is adopted to control the preset visual language model to be in a sleep state for multiple sleep time steps, so that the robot can freely explore the three-dimensional indoor scene.

6. The method according to claim 1, characterized in that, Before acquiring the robot's navigation instructions for the target object category in the 3D indoor scene and the current semantic map of the 3D indoor scene, the method further includes: Acquire the visual data and current pose data collected by the robot at the current time step; The current semantic map is constructed using the visual data and the current pose data.

7. The method according to claim 6, characterized in that, The current pose data includes: the robot's current position and past positions at the current time step; the construction of the current semantic map using the visual data and the current pose data includes: Using the visual data, a map of the current obstacles, a map of the currently explored area, and semantic maps of various preset object categories are constructed respectively. The current obstacle map, the currently explored area map, the semantic map of various preset object categories, the robot's current position, and past positions are stored in multiple channels of a preset image to obtain the current semantic map.

8. The method according to claim 7, characterized in that, The method further includes: Add a fake target semantic map to the current semantic map.

9. A robot space navigation device, characterized in that, include: The acquisition module is used to acquire the robot's navigation instructions for the target object category in the 3D indoor scene and the current semantic map of the 3D indoor scene; the current semantic map is the semantic map constructed by the robot at the current time step; The first determining module is used to determine candidate target points for the target object category from the current semantic map based on the navigation instructions for the target object category and using a preset visual language model; The collision detection module is used to determine whether the robot is in a collision state at the current long-term target point based on the robot's current state at the current time step using a collision assist. If the robot is in a collision state at the current long-term target point, the target point is determined from the current semantic map. The second determining module is used to determine the target point as the next long-term target point by adopting a preset global strategy. The first generation module is used to generate the robot's short-term target point at the next time step based on the next long-term target point and using a preset local strategy. The second generation module is used to generate the robot's action instructions at the next time step based on the short-term target point, so as to control the robot to navigate to the target object corresponding to the target object category.

10. A robot, characterized in that, At least including: The robot body and the controller disposed within the robot body, the controller being used to execute the robot space navigation method according to any one of claims 1-8.

Images