Building scene robot navigation method based on large model technology

Abstract

The application discloses a building scene robot navigation method based on a large model technology, and relates to the technical field of intelligent robot navigation. The application comprises at least the following steps: S1: topology and geometric relations of a building scene are extracted by using a BIM model to form a topology graph, and further, a large language model is used to describe the topology relations by using words, which are used as the basis for reasoning in the next step; S2: according to a given starting point and an ending point, a shortest path is found by using a path search algorithm on a three-dimensional map of the BIM model, and observation images and topology information of each position on the path are recorded. The visual-language navigation method provided by the application can generate navigation instructions directly to guide autonomous navigation and positioning in an indoor environment without establishing a map, by using prior information of a navigation path in a BIM scene, and at the same time, the navigation information is verified by combining the BIM prior information, the accuracy of navigation is improved, and the process automation of SLAM artificial mapping can be realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot navigation technology, specifically to a robot navigation method for architectural scenes based on large model technology. Background Technology

[0002] Currently, in architectural scenarios, robot navigation technology is mainly based on SLAM technology. SLAM technology requires manual traversal of the environment to achieve mapping. At the same time, since the actual environment is in a real-time dynamic process, SLAM-based methods require manual intervention and reconfiguration to adapt to the dynamic environmental changes in architectural scenarios.

[0003] Therefore, SLAM-based navigation methods lack a comprehensive understanding of the built environment and the ability to interact with people.

[0004] Visual-language navigation methods based on large language models can align with human needs, driving navigation by providing specific requirements. By understanding human needs and intentions, the robot's navigation behavior better meets user expectations. However, existing navigation technologies based on large language models lack effective utilization of architectural scene information, hindering accurate and efficient navigation.

[0005] Therefore, a new solution is needed to address the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a robot navigation method for building scenes based on large model technology, so as to solve the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a robot navigation method for building scenes based on large model technology, comprising at least the following steps:

[0008] S1: Use the BIM model to extract the topological and geometric relationships of the building scene, construct a topological map, and further combine it with the large language model to use text to describe the topological relationships, which will serve as the basis for the next step of reasoning.

[0009] S2: Based on the given starting point and ending point, use the path search algorithm to find the shortest path through the 3D map of the BIM model, and record the observation image and its topological information at each location on the path;

[0010] S3: Use the large model to parse out the topologically feasible path and form step-by-step path instructions with textual descriptions, and build a large model navigation mind chain;

[0011] S4: In the initial stage, the robot constructs a voxel map using depth images and camera poses, exports a grid map and an obstacle avoidance map, and determines whether it has reached the target node based on historical images and map attributes. It uses the obstacle avoidance map to plan the path and perform navigation, while continuously updating the grid and obstacle avoidance maps to optimize the path.

[0012] S5: After the task is completed, match each map node in the execution plan with the topology map to ensure the task is completed.

[0013] Furthermore, the topological graph in S1 is represented as G = {V, E, T};

[0014] Where: V represents a map node representing scene semantics, T represents information related to the map node, and E represents an edge formed based on the reachability between specific locations;

[0015] The specific locations include at least doors, windows, and rooms;

[0016] The information associated with the map node includes at least its geometric dimensions, center coordinates, and the number of doors and windows.

[0017] Furthermore, S2 includes at least the following steps:

[0018] Given a prompt word, determine the geometric centers of the map nodes where the start and end points are located, and denote the coordinates of the start point as p1 and the coordinates of the end point as p. N The prompt words include at least a start point and an end point;

[0019] Based on the 3D map corresponding to the BIM model, a path search algorithm is used to find the shortest path P = (p1,…,p...) in the 3D map. i ,…,p N N is the number of path coordinates;

[0020] In the 3D map corresponding to the BIM model, along the search path, the observed image is recorded at each map location, and the topological information of the current location is obtained using the multimodal large model.

[0021] Furthermore, the large model navigation thought chain in S3 includes path description information, map information, and path awareness information. The path information represents the format of the path text description, the map information includes at least the path location and topology map, and the path awareness information includes at least the image corresponding to the path location.

[0022] Furthermore, S4 includes at least the following steps:

[0023] In the initial stage, the robot's position state is set as the initial map node;

[0024] The robot's depth images and camera poses are used to construct a voxel map, and a two-dimensional raster map G is exported. o and obstacle avoidance map G e ;

[0025] During the search process to reach the next map node, a multimodal large model is used to check the robot's historical images and compare the map node attributes to determine whether to reach the next map node (if it is a door / window, it is determined based on the distance to the door / window).

[0026] The next path navigation point is obtained based on the algorithm for reaching the next path navigation point;

[0027] Use G e The robot performs trajectory planning; if it reaches its destination, the robot's position state is changed, and the next path instruction is executed.

[0028] Furthermore, the algorithm for reaching the next path navigation point includes at least the following steps:

[0029] G o The locations are recorded as A according to whether they have been explored, are obstacles, or have not been explored. e A o A u A is obtained through depth-first search. e With A u Boundary point A p The robot from A p The system selects the centers of the M points closest to the direction of the navigation command as the next target point, based on G. e The robot searches for and plans a trajectory path in its configuration space, which is then executed by the robot's underlying controller.

[0030] The grid map G is continuously updated during the process of reaching the next path navigation point. o With obstacle avoidance map G e :

[0031] G o =G os ∪G o

[0032] G e =G es ∪G e

[0033] Among them G os and G es These are the current sensor's grid map and Euclid signature field map, respectively.

[0034] Search G sub The boundary point is A′ p Then the updated boundary points are:

[0035] A p =A p ∪A′ p -A in

[0036] Where A in For G o Boundary points in the middle.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] The visual-language navigation method proposed in this invention can directly generate navigation commands to guide autonomous navigation and positioning in indoor environments without the need for map creation, by utilizing prior information about navigation paths in BIM scenarios. At the same time, it combines BIM prior information to verify navigation information, thereby improving navigation accuracy and automating the SLAM manual mapping process. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the overall BIM model of the present invention;

[0041] Figure 2 This is a partial topological map of the present invention;

[0042] Figure 3 This is a schematic diagram of the node attributes of the present invention;

[0043] Figure 4 This is a schematic diagram of the segmentation and recognition results of the starting position in this invention;

[0044] Figure 5 This is a schematic diagram of the local navigation algorithm of the present invention;

[0045] Figure 6 This is a top view of the robot in motion according to the present invention;

[0046] Figure 7 This is a schematic diagram illustrating the robot's position confirmation according to the present invention;

[0047] Figure 8 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0048] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0049] See Figure 8 A robot navigation method for architectural scenes based on large model technology includes at least the following steps:

[0050] S1: Utilize the BIM model to extract the topological and geometric relationships of the building scene, forming a topological map G = {V, E, T} containing the locations of doors, windows, rooms, etc., where V represents map nodes indicating scene semantics, T represents information related to map nodes (such as geometric dimensions, center coordinates, and the number of doors and windows), and E represents edges formed based on the reachability between doors, windows, and rooms. Further, combine this with a large language model to describe the topological relationships using text, which serves as the basis for the next step of reasoning. For example, ROOM1 connects to ROOM3 by door2.

[0051] (See also) Figures 1-3 )

[0052] S2: Given a prompt (including start and end points), find the geometric centers of the map nodes where the start and end points are located, and denote the coordinates of the start point as p1 and the coordinates of the end point as p. N Based on the 3D map corresponding to the BIM model, a path search algorithm is used to find the shortest path P = (p1, ..., p) in the 3D map. i ,…,p N N represents the number of path coordinates. In the 3D map corresponding to the BIM model, along the search path, the observed image is recorded at each map location, and the topological information of the current location is obtained using a multimodal large model. (See...) Figure 4 )

[0053] S3: Use the large model to parse the topologically feasible path and generate step-by-step path instructions with textual descriptions. Construct a large model navigation thought chain, including path description information, map information, and path-aware information. The path information represents the format of the path's textual description, the map information includes the path location and a topological map, and the path-aware information includes the image corresponding to the path location.

[0054] S4: In the initial stage, the robot's position state is set as the initial map node; the robot's depth image and camera pose are used to construct a voxel map, and a 2D grid map G is exported. o and obstacle avoidance map G eDuring the search to reach the next map node, a multimodal large model is used to examine the robot's historical images, comparing map node attributes to determine whether the next map node has been reached (if it is a door / window, the distance to the door / window is used to determine this); the next path navigation point is obtained according to the following algorithm, using G... e The robot performs trajectory planning; if it reaches its destination, the robot's position state is changed, and the next path instruction is executed.

[0055] The algorithm for reaching the next path navigation point is as follows:

[0056] G o The locations are recorded as A according to whether they have been explored, are obstacles, or have not been explored. e A o A u A is obtained through depth-first search. e With A u Boundary point A p The robot from A p The system selects the centers of the M points closest to the direction of the navigation command as the next target point, based on G. e The robot searches for a trajectory plan in its configuration space and then entrusts it to the robot's underlying controller for execution.

[0057] The grid map G is continuously updated during the process of reaching the next path navigation point. o With obstacle avoidance map G e :

[0058] G o =G os ∪G o

[0059] G e =G es ∪G e

[0060] Among them G os and G es For the current sensor's grid map and Euclid signature field map;

[0061] Search G sub The boundary point is A′ p Then the updated boundary points are:

[0062] A p =A p ∪A′ p -A in

[0063] Where A in For G o Boundary points in the middle. (See also) Figures 5-7 )

[0064] The fifth step is to match each map node in the execution plan with the topology map after the task is completed to ensure the task is completed.

[0065] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A robot navigation method for architectural scenes based on large model technology, characterized in that: At least the following steps are included: S1: Use the BIM model to extract the topological and geometric relationships of the building scene, construct a topological map, and further combine it with the large language model to use text to describe the topological relationships, which will serve as the basis for the next step of reasoning. S2: Based on the given starting point and ending point, use the path search algorithm to find the shortest path through the 3D map of the BIM model, and record the observation image and its topological information at each location on the path; S3: Use the large model to parse out the topologically feasible path and form step-by-step path instructions with textual descriptions, and build a large model navigation mind chain; S4: In the initial stage, the robot constructs a voxel map using depth images and camera poses, exports a grid map and an obstacle avoidance map, and determines whether it has reached the target node based on historical images and map attributes. It uses the obstacle avoidance map to plan the path and perform navigation, while continuously updating the grid and obstacle avoidance maps to optimize the path. The S4 includes at least the following steps: In the initial stage, the robot's position state is set as the initial map node; The robot's depth images and camera poses are used to construct a voxel map, and a 2D raster map is exported. and obstacle avoidance map ; During the search process to reach the next map node, a multimodal large model is used to check the robot's historical images and compare the map node attributes to determine whether the next map node has been reached. The next path navigation point is obtained based on the algorithm for reaching the next path navigation point; use Perform trajectory planning for the robot; if the target is reached, change the robot's position state and execute the next path instruction. The algorithm for reaching the next path navigation point includes at least the following steps: Will The locations are recorded as explored, obstacles, and unexplored. And obtained through depth-first search and boundary points The robot from The system selects the centers of the M points closest to the direction of the navigation command as the next target points, based on... The robot searches for and plans a trajectory path in its configuration space, which is then executed by the robot's underlying controller. The raster map is continuously updated during the journey to the next waypoint. With obstacle avoidance map : in and These are the current sensor's grid map and Euclid signature field map, respectively. Search The boundary point is Then the updated boundary points are: in In order to be in Boundary points in; S5: After the task is completed, match each map node in the execution plan with the topology map to ensure the task is completed.

2. The building scene robot navigation method based on large model technology according to claim 1, characterized in that: The topology graph in S1 is represented as follows: ; in: Map nodes representing scene semantics. This indicates information related to map nodes. Edges are formed based on the reachability between specific locations; The specific locations include at least doors, windows, and rooms; The information associated with the map node includes at least its geometric dimensions, center coordinates, and the number of doors and windows.

3. The building scene robot navigation method based on large model technology according to claim 1, characterized in that: S2 includes at least the following steps: Given a prompt word, determine the geometric center of the map nodes where the start and end points are located, and denote the coordinates of the start point as . The endpoint coordinates are The prompt words include at least a start point and an end point; Based on the 3D map corresponding to the BIM model, a path search algorithm is used to find the shortest path in the 3D map. , The number of path coordinates; In the 3D map corresponding to the BIM model, along the search path, the observed image is recorded at each map location, and the topological information of the current location is obtained using the multimodal large model.

4. The building scene robot navigation method based on large model technology according to claim 1, characterized in that: The large model navigation thought chain in S3 includes path description information, map information, and path awareness information. The path description information represents the format of the path text description. The map information includes at least the path location and topology map. The path awareness information includes at least the image corresponding to the path location.

Images