Weak texture environment semantic mapping method and navigation method based on cognitive inspiration

Abstract

The invention discloses a weak texture environment semantic mapping method and navigation method based on cognitive inspiration, and the method comprises the steps: collecting a panoramic image of an external environment of a robot in real time, judging whether the current panoramic image is a corridor environment or not through a visual language model, and if not, continuing to judge whether the next panoramic image is the corridor environment or not; by constructing a topological graph framework, traditional geometric mapping depending on dense feature points or three-dimensional point cloud is replaced; meanwhile, only key semantic nodes such as corridors and rooms are reserved, only necessary information such as house numbers is stored for corridor nodes, only necessary information such as image features and two-dimensional point clouds is stored for room nodes, the storage overhead of environment characterization is remarkably reduced, and the long-term operation capacity of the robot on a resource-limited platform is improved; the technical problems that in the prior art, a weak texture environment semantic mapping method depends on dense geometric storage in a weak texture environment, so that the storage cost is high, and the semantic reasoning ability is weak are solved.

Description

Technical Field

[0001] This invention belongs to the fields of cognitive computing and semantic mapping, and relates to a mapping method, specifically a cognitively inspired semantic mapping method and navigation method for weakly textured environments. Background Technology

[0002] With the widespread application of embodied robots in ubiquitous computing and the Internet of Things (IoT) ecosystem, their efficient interaction and navigation capabilities with the physical environment have become a key foundation for realizing various intelligent services. Environmental mapping technology provides robots with support for environmental cognition and task planning by constructing spatial representations; however, existing high-precision mapping methods generally rely on the storage and processing of dense geometric or visual information, resulting in high storage overhead and poor environmental adaptability, especially unsuitable for resource-constrained robot platforms. Furthermore, such methods are less robust to environments with weak texture, low features, or dynamic changes, and cannot effectively mimic the efficient spatial cognition mechanisms of humans based on semantic and topological relationships.

[0003] Humans can construct lightweight, structured cognitive maps through abstraction and reasoning, storing only key spatial relationships and semantic information. This allows for efficient understanding and navigation of complex environments while avoiding redundant storage. For example, in indoor corridors lacking visual features, humans can navigate reliably using high-level semantic cues such as room numbers and signs, without relying on detailed low-level visual feature extraction. In recent years, the rapid development of Large Language Models (LLMs) and Vision-Language Models (VLMs) has demonstrated powerful semantic understanding and reasoning capabilities, enabling them to extract and integrate high-level semantic information from the environment. This provides a technological possibility for constructing human-like topological environmental representations.

[0004] Therefore, there is an urgent need to propose a cognitive-inspired semantic mapping and navigation method for weakly textured environments to overcome the shortcomings of existing technologies, such as high storage costs and insufficient environmental adaptability, and to achieve efficient and reliable navigation for robots in complex scenarios with weak textures and dynamic changes. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a cognitive-inspired semantic mapping and navigation method for weakly textured environments, thereby solving the technical problems of poor adaptability, high storage costs, and weak semantic reasoning capabilities caused by the reliance on dense geometric storage in weakly textured environments in existing semantic mapping methods for weakly textured environments.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A cognitively inspired semantic mapping method for weakly textured environments includes the following steps: Step 1: Real-time acquisition of panoramic images of the robot's external environment. A visual language model is used to determine if the current panoramic image represents a corridor environment. If not, the next panoramic image is used to determine if it is a corridor environment. If it is, cnOCR is used to perform real-time text recognition on the obtained panoramic image. The confidence level of the recognized address text is checked against a preset threshold. If so, an initial corridor node is created at the current location. Proceed to step 2. If not, continue acquiring and identifying the next panoramic image until the initial corridor node is created. Proceed to step 2; This indicates the robot's pose at the location corresponding to the first corridor node; This indicates the room number corresponding to the first corridor node; Step 2, the robot moves towards the initial corridor node. The system navigates to the room corresponding to the door number and performs judgment one; it also uses a visual language model to identify in real time whether the current environment is a room environment. If so, it uses image feature extraction methods to extract features from the interior environment of the room corresponding to the door number, obtaining image features. Simultaneously, a lidar scanner was used to scan the interior environment of the room, obtaining a two-dimensional point cloud. That is, to obtain the room node. If an edge is formed between the robot and the previous or auxiliary corridor node, the robot leaves the room; otherwise, the identification process continues. This indicates the robot's pose at the location corresponding to the first room node; Judgment 1: The robot uses LiDAR to perceive the external environment, obtains a grid map supporting the construction of the topology, and uses odometry to obtain the robot's current coordinates in real time. The robot's current coordinates are then compared with those of the corridor nodes. Expand the lines connecting the obstacles to obtain a rectangular expansion layer. Determine whether the rectangular expansion layer and the obstacle satisfy formula (1). If so, create an auxiliary corridor node. If a connection edge is formed between this node and the previous corridor node, the robot continues to move and makes further judgments. (1) in: Given the robot's current coordinates and the previous corridor node. The half-width of the expansion layer forming the connecting edge, in meters; Let be the robot's half-width, in meters. The distance from the obstacle to the line connecting them is m; This indicates the robot's pose at the location corresponding to the first auxiliary corridor node; This represents the corresponding room number and is set to 0; The grid map includes free space, unexplored space, and obstacles; The pose includes the robot's two-dimensional coordinates and rotation angle; Step 3: Real-time acquisition of panoramic images of the robot's external environment. A visual language model is used to determine if the current panoramic image represents a corridor environment. If not, the next panoramic image is used to determine if it is a corridor environment. If it is, cnOCR is used to perform real-time text recognition on the obtained panoramic image. The confidence level of the recognized address text is checked against a preset threshold. If so, a corridor node is created at the current location. Simultaneously, an edge is formed between the current node and the previous room node or auxiliary corridor node, proceeding to step 4. Otherwise, continue identifying the next panoramic image until a corridor node is created. Proceed to step 4; (2) in: Represents corridor nodes With corridor nodes The distance between them, in meters; u is a positive integer; This represents the room number corresponding to the i-th corridor node, where i is a positive integer greater than 1; = This represents the robot's pose at the location corresponding to the i-th corridor node. and These are the robot's coordinates in a two-dimensional plane. It is the angle of rotation of the robot, in rad; Step 4, the robot moves towards the corridor node. The system navigates to the room corresponding to the door number and performs judgment two; it uses a visual language model to identify in real time whether the current environment is inside a room and whether the distance between any two room nodes satisfies formula (3). If so, it uses an image feature extraction method to extract features from the room environment corresponding to the door number and obtain image features. LiDAR scans the interior environment of the room to obtain a two-dimensional point cloud. That is, to obtain the room node. If an edge is formed between the robot and the previous corridor node or auxiliary corridor node, the robot leaves the room; otherwise, the identification process continues. (3) in: Represents room node and room nodes The distance between them, in meters; = This represents the robot's pose at the location corresponding to the k-th room node. and These are the robot's coordinates in a two-dimensional plane. It is the angle of rotation of the robot, in rad; k is a positive integer greater than 1; Decision 2: The robot continues to move along the corridor. The robot's current coordinates are obtained in real-time using an odometry system. These coordinates are then compared with the corridor nodes. Room nodes Or auxiliary corridor nodes Expand the lines between them to obtain a rectangular expansion layer. Determine whether the rectangular expansion layer and the obstacle satisfy formula (4). If so, create an auxiliary corridor node. Meanwhile, this auxiliary corridor node and the previous corridor node or room node Or auxiliary corridor nodes If a connecting edge is formed between them, the robot continues to move and makes further judgments. (4) in: The robot's current coordinates and corridor nodes Room nodes Or auxiliary corridor nodes The half-width of the rectangular expansion layer formed by the lines connecting them, in meters; Let be the robot's half-width, in meters. The distance from the obstacle to the line connecting them is m; This represents the robot's pose at the location corresponding to the j-th auxiliary corridor node, where j is a positive integer greater than 1; Indicate the corresponding room number and set it to 0; Step 5: Let i = i+1, k = k+1, return to step 3 to continue identification until k = K, and obtain the node set (V) and edge set (E), that is, obtain the topological connected graph G; The topological graph framework G includes a set of nodes (V) and a set of edges (E); The node set (V) includes corridor nodes, room nodes, and auxiliary corridor nodes; The edge set (E) includes the connecting edges between corridor nodes, the connecting edges between room nodes and corridor nodes, the connecting edges between auxiliary corridor nodes and room nodes, and the connecting edges between auxiliary corridor nodes and corridor nodes.

[0007] In step 1 or step 3, the real-time acquisition of panoramic images of the robot's external environment specifically involves: Four RGB cameras were placed in front of, behind, and on the left and right sides of the robot to collect real-time panoramic images of the robot's external environment.

[0008] In step 2 or step 4, a visual language model is used to identify in real time whether the current environment is the specific content inside the room: A visual language model is used to classify the panoramic image into scenes and obtain node types, which include corridor environment and room environment, thus obtaining the type of the current environment.

[0009] A navigation method, based on the topologically connected graph obtained by the cognitively inspired weakly textured environment semantic mapping method, includes the following steps: Step 1: Perform initial localization on the robot to obtain its pose in the topologically connected graph, i.e., obtain the starting coordinates; Step 2: Use a graph search algorithm to search the topology graph to obtain a sequence of navigation nodes, and then obtain the navigation path from the starting point to the target room node based on the navigation node sequence. Step 3: The robot moves according to the navigation path obtained in Step 2 until it reaches the target room; During the movement, the next path node is set as the current sub-target, and the environment is continuously scanned with LiDAR. If any of the following conditions are met, the sub-target is dynamically updated: 1. When multiple nodes are in line of sight with the robot's current position, update the sub-target to the farthest node in line of sight.

[0010] 2. When the distance between the current sub-target and the obstacle is less than half the width of the robot itself, the node is skipped, the next node is set as the new sub-target, and the robot moves toward the new sub-target until it navigates to the target room node, that is, it arrives at the target room.

[0011] Step one specifically includes the following steps: Step 1.1: Acquire the current panoramic image of the robot and extract the image features of the current panoramic image. Calculate the features of the image The image features obtained in the cognitive-inspired weak-texture environment semantic mapping method. dot product similarity ; (5) Step 1.2: Determine whether the calculated maximum similarity exceeds the threshold M. If yes, complete the preliminary coarse localization and obtain the room where the robot is currently located and the room node; otherwise, end the process. Step 1.3: The LiDAR scanner obtains the robot's current point cloud data, and this point cloud data is compared with the two-dimensional point cloud obtained in the cognitive-inspired weak-texture environment semantic mapping method. Registration is performed to obtain the relative pose transformation matrix T between the two, and the robot's pose p in the topology map is obtained according to the following formula; (6).

[0012] Step two specifically includes the following steps: Step 2.1: Given the door number of the target room, search for the corridor nodes associated with the door number in the topological connected graph obtained by the cognitive-inspired weak texture environment semantic mapping method, and then determine the target room node. Step 2.2: Calculate the shortest path between the robot's starting point and the target room node to obtain the navigation node sequence, which is the navigation path from the starting point to the target room node.

[0013] Compared with the prior art, the beneficial technical effects of this invention are: (I) This invention replaces the traditional geometric mapping that relies on dense feature points or 3D point clouds by constructing a topological graph framework; at the same time, it only retains key semantic nodes such as corridors and rooms. For corridor nodes, only necessary information such as door numbers is stored, and for room nodes, only necessary information such as image features and 2D point clouds is stored. This significantly reduces the storage overhead of environmental representation and improves the long-term operating capability of robots on resource-constrained platforms. It solves the technical problems of high storage cost and weak semantic reasoning ability caused by dense geometric storage in weak texture environment semantic mapping methods in the prior art.

[0014] (II) This invention references abstract semantic text or symbolic information that humans can recognize and utilize, selects house numbers in weak textures as semantic information, and uses this information to construct corridor nodes. Simultaneously, when creating edges, it calculates the shortest distance between the connecting line and obstacles and introduces a rectangular expansion layer constraint to enhance the robot's adaptability and robustness to weak texture scenes. This supports efficient semantic reasoning and map construction, solving the problem of poor adaptability in existing semantic mapping methods for weak texture environments due to their reliance on dense geometric storage.

[0015] (III) This invention constructs a two-stage localization and dynamic sub-target navigation mechanism of "semantic matching + point cloud registration". First, it uses image feature dot product similarity to complete coarse semantic localization, and then obtains precise pose through point cloud registration. Finally, it dynamically updates sub-targets based on line-of-sight relationships. This improvement significantly enhances localization accuracy and navigation success rate, achieving efficient and robust navigation in real and complex environments. Attached Figure Description

[0016] Figure 1 This is a navigation diagram; Figure 2 This is a simulation scene of an indoor office building as shown in Example 1; Figure 3 Here is an example of a house number from Example 1; Figure 4 The robot in Example 1; Figure 5 This refers to the initial mapping in Example 1; Figure 6 This refers to establishing the initial corridor node in Example 1; Figure 7 This refers to the creation of auxiliary corridor nodes and room nodes in Example 1; Figure 8 This is the final topology diagram in Example 1; Figure 9 The navigation start and end points in Example 2; Figure 10 This is the navigation node sequence in Example 2.

[0017] The specific content of the present invention will be further explained in detail below with reference to the embodiments. Detailed Implementation

[0018] It should be noted that, unless otherwise specified, all components in this invention are those known in the art.

[0019] The following are specific embodiments of the present invention. It should be noted that the present invention is not limited to the following specific embodiments. All equivalent modifications made based on the technical solutions of this application fall within the protection scope of the present invention.

[0020] This invention presents a cognitively inspired method for semantic mapping of weakly textured environments, specifically including the following steps: Step 1: Real-time acquisition of panoramic images of the robot's external environment. A visual language model is used to determine if the current panoramic image represents a corridor environment. If not, the next panoramic image is used to determine if it is a corridor environment. If it is, cnOCR is used to perform real-time text recognition on the obtained panoramic image. The confidence level of the recognized address text is checked against a preset threshold. If so, an initial corridor node is created at the current location. Proceed to step 2. If not, continue acquiring and identifying the next panoramic image until the initial corridor node is created. Proceed to step 2; This indicates the robot's pose at the location corresponding to the first corridor node; This indicates the room number corresponding to the first corridor node; Step 2, the robot moves towards the initial corridor node. The system navigates to the room corresponding to the door number and performs judgment one; it also uses a visual language model to identify in real time whether the current environment is a room environment. If so, it uses image feature extraction methods to extract features from the interior environment of the room corresponding to the door number, obtaining image features. Simultaneously, a lidar scanner was used to scan the interior environment of the room, obtaining a two-dimensional point cloud. That is, to obtain the room node. If an edge is formed between the robot and the previous or auxiliary corridor node, the robot leaves the room; otherwise, the identification process continues. This indicates the robot's pose at the location corresponding to the first room node; Judgment 1: The robot uses LiDAR to perceive the external environment, obtains a grid map supporting the construction of the topology, and uses odometry to obtain the robot's current coordinates in real time. The robot's current coordinates are then compared with those of the corridor nodes. Expand the lines connecting the obstacles to obtain a rectangular expansion layer. Determine whether the rectangular expansion layer and the obstacle satisfy formula (1). If so, create an auxiliary corridor node. If a connection edge is formed between this node and the previous corridor node, the robot continues to move and makes further judgments. (1) in: Given the robot's current coordinates and the previous corridor node. The half-width of the expansion layer forming the connecting edge, in meters; Let be the robot's half-width, in meters. The distance from the obstacle to the line connecting them is m; This indicates the robot's pose at the location corresponding to the first auxiliary corridor node; This represents the corresponding room number and is set to 0; The grid map includes free space, unexplored space, and obstacles; The pose includes the robot's two-dimensional coordinates and rotation angle; Step 3: Real-time acquisition of panoramic images of the robot's external environment. A visual language model is used to determine if the current panoramic image represents a corridor environment. If not, the next panoramic image is used to determine if it is a corridor environment. If it is, cnOCR is used to perform real-time text recognition on the obtained panoramic image. The confidence level of the recognized address text is checked against a preset threshold. If so, a corridor node is created at the current location. Simultaneously, an edge is formed between the current node and the previous room node or auxiliary corridor node, proceeding to step 4. Otherwise, continue identifying the next panoramic image until a corridor node is created. Proceed to step 4; (2) in: Represents corridor nodes With corridor nodes The distance between them, in meters; u is a positive integer; This represents the room number corresponding to the i-th corridor node, where i is a positive integer greater than 1; = This represents the robot's pose at the location corresponding to the i-th corridor node. and These are the robot's coordinates in a two-dimensional plane. It is the angle of rotation of the robot, in rad; Step 4, the robot moves towards the corridor node. The system navigates to the room corresponding to the door number and performs judgment two; it uses a visual language model to identify in real time whether the current environment is inside a room and whether the distance between any two room nodes satisfies formula (3). If so, it uses an image feature extraction method to extract features from the room environment corresponding to the door number and obtain image features. LiDAR scans the interior environment of the room to obtain a two-dimensional point cloud. That is, to obtain the room node. If an edge is formed between the robot and the previous corridor node or auxiliary corridor node, the robot leaves the room; otherwise, the identification process continues. (3) in: Represents room node and room nodes The distance between them, in meters; = This represents the robot's pose at the location corresponding to the k-th room node. and These are the robot's coordinates in a two-dimensional plane. It is the angle of rotation of the robot, in rad; k is a positive integer greater than 1; Decision 2: The robot continues to move along the corridor. The robot's current coordinates are obtained in real-time using an odometry system. These coordinates are then compared with the corridor nodes. Room nodes Or auxiliary corridor nodes Expand the lines between them to obtain a rectangular expansion layer. Determine whether the rectangular expansion layer and the obstacle satisfy formula (4). If so, create an auxiliary corridor node. Meanwhile, this auxiliary corridor node and the previous corridor node or room node Or auxiliary corridor nodes If a connecting edge is formed between them, the robot continues to move and makes further judgments. (4) in: The robot's current coordinates and corridor nodes Room nodes Or auxiliary corridor nodes The half-width of the rectangular expansion layer formed by the lines connecting them, in meters; Let be the robot's half-width, in meters. The distance from the obstacle to the line connecting them is m; This represents the robot's pose at the location corresponding to the j-th auxiliary corridor node, where j is a positive integer greater than 1; Indicate the corresponding room number and set it to 0; Step 5: Let i = i+1, k = k+1, return to step 3 to continue identification until k = K, and obtain the node set (V) and edge set (E), that is, obtain the topological connected graph G; The topological graph framework G includes a set of nodes (V) and a set of edges (E); The node set (V) includes corridor nodes, room nodes, and auxiliary corridor nodes; The edge set (E) includes the connecting edges between corridor nodes, the connecting edges between room nodes and corridor nodes, the connecting edges between auxiliary corridor nodes and room nodes, and the connecting edges between auxiliary corridor nodes and corridor nodes.

[0021] In the above technical solution, a topological graph framework is constructed to replace the traditional geometric mapping that relies on dense feature points or 3D point clouds. At the same time, only key semantic nodes such as corridors and rooms are retained. For corridor nodes, only necessary information such as door numbers is stored, and for room nodes, only necessary information such as image features and 2D point clouds is stored. This significantly reduces the storage overhead of environmental representation and improves the robot's long-term operating capability on resource-constrained platforms. It also solves the technical problems of high storage costs and weak semantic reasoning ability caused by dense geometric storage in weak texture environment semantic mapping methods in existing technologies.

[0022] By integrating odometry and LiDAR sensors, the robot's physical environment is simultaneously localized and scanned. A lightweight Cartographer algorithm is then used to process the scanned data, constructing an occupied grid map that divides the environment into three categories: free space, unexplored space, and obstacles. This grid map provides obstacle avoidance constraints for the spatial layout of nodes and edges in the topology graph.

[0023] Referring to the ability of humans to recognize and utilize abstract semantic text or symbolic information, this method selects house numbers in weakly textured environments as semantic information and uses this information to construct corridor nodes. Simultaneously, when creating edges, the shortest distance between the connecting line and obstacles is calculated, and a rectangular expansion layer constraint is introduced to enhance the robot's adaptability and robustness to weakly textured scenes. This supports efficient semantic reasoning and map construction, solving the problem of poor adaptability in existing semantic mapping methods for weakly textured environments due to their reliance on dense geometric storage.

[0024] In step 1 or step 3, the real-time acquisition of panoramic images of the robot's external environment specifically involves: Four RGB cameras were placed in front of, behind, and on the left and right sides of the robot to collect real-time panoramic images of the robot's external environment.

[0025] In the above technical solution, panoramic images are used to eliminate the distortion caused by the robot's rotation angle. The resulting differences in image features.

[0026] In step 2 or step 4, a visual language model is used to identify in real time whether the current environment is the specific content inside the room: A visual language model is used to classify the panoramic image into scenes and obtain node types, which include corridor environment and room environment, thus obtaining the type of the current environment.

[0027] In the above technical solution, when the robot explores its environment, a panoramic camera is used to capture environmental images. Subsequently, the Moonshot-v1 visual language model is used to process the captured panoramic image data, and specific prompts are input to request the model to classify the images as either "room" or "corridor." The output of this step is a preliminary judgment of the node type, providing a decision-making basis for the subsequent dynamic creation of nodes.

[0028] Many indoor environments (such as office buildings and dormitories) typically consist of two parts: corridors and rooms. Therefore, classifying node types into corridor environments and room environments is beneficial for constructing the topology graph. Furthermore, visual language models possess powerful scene understanding and reasoning capabilities, easily distinguishing whether the current scene is in a corridor environment or a room environment.

[0029] This invention also provides a navigation method based on the topologically connected graph obtained by the cognitively inspired weak-texture environment semantic mapping method, comprising the following steps: Step 1: Perform initial localization on the robot to obtain its pose in the topologically connected graph, i.e., obtain the starting coordinates; Step 2: Use a graph search algorithm to search the topology graph to obtain a sequence of navigation nodes, and then obtain the navigation path from the starting point to the target room node based on the navigation node sequence. Step 3: The robot moves according to the navigation path obtained in Step 2 until it reaches the target room; During the movement, the next path node is set as the current sub-target, and the environment is continuously scanned with LiDAR. If any of the following conditions are met, the sub-target is dynamically updated: 1. When multiple nodes are in line of sight with the robot's current position, update the sub-target to the farthest node in line of sight.

[0030] 2. When the distance between the current sub-target and the obstacle is less than half the width of the robot itself, the node is skipped, the next node is set as the new sub-target, and the robot moves toward the new sub-target until it navigates to the target room node, that is, it arrives at the target room.

[0031] The above technical solution constructs a two-stage localization and dynamic sub-target navigation mechanism of "semantic matching + point cloud registration". First, it uses image feature dot product similarity to complete coarse semantic localization, then obtains precise pose through point cloud registration, and dynamically updates sub-targets based on line-of-sight relationships. This improvement significantly enhances localization accuracy and navigation success rate, achieving efficient and robust navigation in real-world complex environments.

[0032] Step one specifically includes the following steps: Step 1.1: Acquire the current panoramic image of the robot and extract the image features of the current panoramic image. Calculate the features of the image The image features obtained in the cognitive-inspired weak-texture environment semantic mapping method. dot product similarity ; (5) Step 1.2: Determine whether the calculated maximum similarity exceeds the threshold M. If yes, complete the preliminary coarse localization and obtain the room where the robot is currently located and the room node; otherwise, end the process. Step 1.3: The LiDAR scanner obtains the robot's current point cloud data, and this point cloud data is compared with the two-dimensional point cloud obtained in the cognitive-inspired weak-texture environment semantic mapping method. Registration is performed to obtain the relative pose transformation matrix T between the two, and the robot's pose p in the topology map is obtained according to the following formula.

[0033] (6).

[0034] In the above technical solution, the threshold M is set to 0.9.

[0035] Step two specifically includes the following steps: Step 2.1: Given the door number of the target room, search for the corridor nodes associated with the door number in the topological connected graph obtained by the cognitive-inspired weak texture environment semantic mapping method, and then determine the target room node. Step 2.2: Calculate the shortest path between the robot's starting point and the target room node to obtain the navigation node sequence, which is the navigation path from the starting point to the target room node.

[0036] Example 1: The cognitive-inspired weak-texture environmental semantic mapping method of this invention was used to conduct mapping tests in an indoor office building scene, such as... Figure 2 As shown, an example of a house number is as follows. Figure 3 As shown, the robot used is as follows Figure 4 As shown.

[0037] Place the robot in its initial position, such as Figure 5 As shown, LiDAR is used to perceive the external environment and obtain a grid map, in which white areas represent free space, gray areas represent unexplored space, and black areas represent obstacles.

[0038] The robot's pose is acquired using odometry and represented on a red, green, and blue coordinate axis. Real-time panoramic images of the robot's external environment are acquired, and a visual language model is used to determine that the current panoramic image represents a corridor environment. Then, cnOCR is used to perform real-time text recognition on the acquired panoramic image. If the confidence level of the recognized door number text is higher than a preset threshold, an initial corridor node is created at the current location. (Indicated in red), such as Figure 6 As shown.

[0039] Robot towards corridor node The robot travels to the room corresponding to its door number, and its current coordinates are compared with... The lines connecting the obstacles are expanded to obtain a real-time rectangular expansion layer. When the expansion layer satisfies formula (1) with the obstacle, an auxiliary corridor node is created. (Indicated in blue), this node and Connecting edges are formed between them.

[0040] A visual language model is used to identify in real time whether the current environment is a room environment. If so, image feature extraction methods are used to extract features of the interior environment of the room corresponding to the door number, resulting in image features. Simultaneously, a lidar scanner was used to scan the interior environment of the room, obtaining a two-dimensional point cloud. That is, to obtain the room node. (Yellow indicates) Room node and auxiliary corridor nodes An edge is formed between them, the robot drives out of the room and back into the corridor, creating an auxiliary corridor node. Room node and auxiliary corridor nodes An edge is formed between them. Then the robot detects the next address and creates a corridor node. Corridor Node and An edge is formed between them, such as Figure 7 As shown.

[0041] The robot continues to acquire panoramic images and create corridor nodes and room nodes until it has explored all the rooms, forming a topologically connected graph, such as... Figure 8 As shown.

[0042] Example 2: Navigation tests were conducted in an indoor office building scenario using the navigation method described in this invention. The robot was placed in... Figure 8 The starting point is located near room node A. The robot acquires its current panoramic image and extracts its image feature β. The image features of β are then compared with those of the remaining room nodes. dot product similarity Image features of β and room node A were discovered. dot product similarity The maximum value exceeds the threshold of 0.9, thus completing the initial coarse localization and obtaining the room and room node where the robot is currently located. The robot's current point cloud data is obtained using LiDAR scanning. This point cloud data is then registered with the two-dimensional point cloud ϕ stored in room node A, and the relative pose transformation matrix T between the two is calculated to obtain the robot's pose p in the topology map.

[0043] Given the address of target room B, the corridor node C associated with that address is searched in the topological connected graph obtained by the cognitive-inspired weak-texture environment semantic mapping method, thereby determining the target room node B, as follows. Figure 9 As shown; Using Dijkstra's algorithm, the shortest path between the robot's starting point A and the target room node B is calculated, resulting in the navigation node sequence A-1-2-3-4-5-6-7-8-9-10-11-12-B. This sequence represents the navigation path from the starting point to the target room node. Figure 10 As shown.

[0044] The robot moves according to the obtained navigation path until it reaches the target room B, such as... Figure 10 As shown.

Claims

1. A cognitive heuristic based weak-textured environment semantic mapping method, characterized in that, Specifically comprising the following steps: Step 1, collect panoramic images of the external environment of the robot in real time, use a visual language model to determine whether the current panoramic image is a corridor environment, if not, continue to determine whether the next panoramic image is a corridor environment; if yes, use cnOCR tool to perform real-time text recognition on the obtained panoramic image, determine whether the confidence of the recognized door number text is higher than a preset threshold, if yes, create an initial corridor node at the current position , enter step 2, if not, continue to collect and recognize the next panoramic image until an initial corridor node is created , enter step 2; represents the pose of the robot at the position corresponding to the first corridor node; represents the room door number corresponding to the first corridor node; Step 2, the robot drives to the initial corridor node The room corresponding to the house number is driven, and the judgment I is executed; and the visual language model is used to identify whether the current environment is a room environment in real time, if yes, the image feature extraction method is used to extract the features of the internal environment of the room corresponding to the house number, to obtain the image features ; At the same time, the laser radar scans the internal environment of the room to obtain a two-dimensional point cloud , that is, the room node is obtained , and an edge is formed between the room node and the previous corridor node or the auxiliary corridor node, and the robot drives out of the room; otherwise, the identification is continued; represents the pose of the robot at the location corresponding to the first room node; Judgment 1: the robot uses laser radar to perceive the external environment, obtains a grid map supporting the construction of a topological graph, adopts a real-time odometer to obtain the current coordinates of the robot, inflates the connecting line between the current coordinates of the robot and the corridor node to obtain a rectangular inflation layer, judges whether the rectangular inflation layer and the obstacle satisfy formula (1), if yes, an auxiliary corridor node is created , a connecting edge is formed between the node and the previous corridor node, if not, the robot continues to travel and continues to judge; （1） Wherein: the current coordinates of the robot and the previous corridor node half-width of the inflated layer forming the connection edge between them, m; half-width of the robot, m; m is the shortest distance from the obstacle to the line; represents the pose of the robot at the position corresponding to the first auxiliary corridor node; representing the corresponding room door number and setting it to 0; The grid map comprises free space, unexplored space and obstacles; The pose comprises two-dimensional coordinates and a rotation angle of the robot; Step 3, collect panoramic images of the external environment of the robot in real time, use a visual language model to determine whether the current panoramic image is a corridor environment, if not, continue to determine whether the next panoramic image is a corridor environment; if yes, use cnOCR tool to perform real-time text recognition on the obtained panoramic image, determine whether the confidence of the recognized door number text is higher than a preset threshold, if yes, create a corridor node at the current position , form an edge between the current node and the previous room node or auxiliary corridor node, enter step 4, if not, continue to recognize the next panoramic image until a corridor node is created , enter step 4; （2） Wherein: representing a distance between the corridor node and the corridor node m; u is a positive integer; represents the room door number corresponding to the i-th corridor node, i is a positive integer with a value greater than 1; = represents the pose of the robot at the position corresponding to the i-th corridor node, and is the coordinate of the robot in a two-dimensional plane, is the angle of rotation of the robot, rad; Step 4, the robot drives to the corridor node The room corresponding to the door number is driven, and the judgment two is executed; whether the current environment is inside the room and the distance between any two room nodes satisfies formula (3) is identified in real time by using a visual language model, if yes, the image features of the room inside the environment corresponding to the door number are extracted by using an image feature extraction method, to obtain image features , the laser radar scans the room inside the environment, to obtain two-dimensional point cloud , to obtain the room node , an edge is formed between the current node and the last corridor node or the auxiliary corridor node, and the robot drives out of the room; if not, continue to identify; （3） Wherein: representing a room node and a distance, m, between room nodes ​ = represents the pose of the robot at the position corresponding to the kth room node, and is the coordinate of the robot in a two-dimensional plane, is the angle of rotation of the robot, rad; k is a positive integer greater than 1; Judgment two: the robot continues to travel in the corridor, adopts the real-time obtained current coordinates of the robot by the odometer, and inflates the connection line between the current coordinates of the robot and the corridor node , the room node or the auxiliary corridor node , to obtain a rectangular inflation layer, judges whether the rectangular inflation layer and the obstacle satisfy formula (4), if yes, an auxiliary corridor node is created, and a connection edge is formed between the auxiliary corridor node and the last corridor node or the room node or the auxiliary corridor node , if not, the robot continues to travel and continues to judge; （4） Wherein: the current coordinates of the robot and the corridor nodes , room nodes or auxiliary corridor nodes between the half-width of the rectangular inflation layer, m; half-width of the robot, m; m is the shortest distance from the obstacle to the line; represents the pose of the robot at the position corresponding to the jth auxiliary corridor node, j being a positive integer greater than 1; representing the corresponding room door number and setting it to 0; Step 5, let i = i + 1, k = k + 1, return to step 3 to continue identification until k = K, obtain the node set (V) and the edge set (E), that is, obtain the topologically connected graph G; The topological graph framework G comprises a node set (V) and an edge set (E); The node set (V) comprises a corridor node, a room node and an auxiliary corridor node; The edge set (E) comprises a connecting edge between corridor nodes, a connecting edge between room nodes and corridor nodes, a connecting edge between auxiliary corridor nodes and room nodes, and a connecting edge between auxiliary corridor nodes and corridor nodes.

2. The cognitive-inspired weak-texture environment semantic mapping method as described in claim 1, characterized in that, In step 1 or step 3, the panoramic image of the external environment of the robot is collected in real time, and specifically: Four RGB cameras are placed in front, back and both sides of the robot to collect panoramic images of the external environment of the robot in real time.

3. The cognitive-inspired weak-texture environment semantic mapping method as described in claim 1, characterized in that, In step 2 or step 4, the visual language model is used to identify whether the current environment is inside a room in real time: The visual language model is used to classify the scene of the panoramic image to obtain the node type, and the node type comprises a corridor environment and a room environment, that is, the type of the current environment is obtained.

4. A navigation method based on the topologically connected graph obtained by the cognitive heuristic-based weak texture environment semantic mapping method according to any one of claims 1-3, comprising the following steps: Step 1, initial positioning of the robot to obtain its pose in the topologically connected graph, that is, to obtain the starting point coordinates; Step 2, searching the topological graph frame using a graph search algorithm to obtain a navigation node sequence, and obtaining a navigation path from the starting point to the target room node according to the navigation node sequence; Step 3, the robot moves according to the navigation path obtained in step 2 until it reaches the target room; During the movement, the next node is set as the current sub-target, and the environment is continuously scanned with a laser radar, and if any of the following conditions is met, the sub-target is dynamically updated: I. When there are multiple nodes behind the robot that are in sight of the robot, the sub-target is updated to the farthest node in sight. Two, when the distance between the current sub-target and the obstacle is less than the half width of the robot itself, the node is skipped, the next node is set as the new sub-target, and the robot moves towards the new sub-target until it navigates to the target room node, that is, reaches the target room.

5. The navigation method of claim 4, wherein, Step 1 specifically comprises the following steps: Step 1.1, capture a current panoramic image of the robot and extract image features of the current panoramic image , calculate the dot product similarity of the image features and the image features obtained in the cognitive heuristic based weak-texture environment semantic mapping method ;​ (5) Step 1.2, determine whether the maximum similarity calculated exceeds the threshold M, if yes, complete the preliminary coarse positioning to obtain the room where the robot is currently located and the room node; if not, end; Step 1.3, laser radar scanning obtains the current point cloud data of the robot, and the two-dimensional point cloud data obtained in the cognitive heuristic-based weak-texture environment semantic mapping method Registration is performed to obtain the relative pose transformation matrix T between the two, and the pose p of the robot in the topological graph is obtained according to the following formula; (6)。 6. The navigation method of claim 4, wherein, Step 2 specifically comprises the following steps: Step 2.1, search for the corridor node associated with the doorplate number of the target room in the topologically connected graph obtained by the cognitive heuristic-based weak texture environment semantic mapping method, and then determine the target room node; Step 2.2, calculate the shortest path from the start node of the robot to the target room node to obtain a navigation node sequence, i.e. to obtain a navigation path from the start node to the target room node.

7. A navigation system characterized by comprising: The navigation method of any one of claims 4-6 is implemented.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the navigation method of any one of claims 4-6.

9. A computer program product, characterised in that, The computer program / instruction is executed by the processor to implement the navigation method of any one of claims 4-6.

Images