Article searching method and device for complex indoor environment

Abstract

The invention relates to an article searching method and device for a complex indoor environment, and the method comprises the steps: dividing objects in the complex indoor environment, obtaining environment articles with relatively fixed positions and dynamic articles with uncertain positions, building a semantic relation graph according to the prior relation between the environment articles and the dynamic articles, and searching the objects in the complex indoor environment according to the semantic relation graph. The search efficiency is improved by utilizing a prior relationship; a semantic layer map containing scene and environment article information is established based on the semantic relation graph, and the semantic layer map provides semantic understanding of the environment, so that the robot can recognize different scenes and articles; constructing a topological layer map containing the optimal observation point of the article and the scene connection point; the topological layer map provides structural information of the environment, so that efficient navigation and search are facilitated; when the robot detects a target object in the complex indoor environment, path planning is carried out according to the semantic layer map, the topology layer map and the semantic relation map, and a global path of the target object is obtained; therefore, path searching and rapid and accurate article positioning are realized.

Description

Technical Field

[0001] This invention relates to the field of robot target search technology, and in particular to a method and apparatus for searching for objects in complex indoor environments. Background Technology

[0002] With the development of intelligent service robots, robot mapping technology, as one of the foundations for service robots to perform intelligent tasks, has become an important part of the field of service robot research.

[0003] In traditional mapping methods, robots often rely on sensors such as LiDAR and depth cameras for environmental perception and use SLAM (Simultaneous Localization and Mapping) algorithms to build environmental maps. However, these methods exhibit many limitations in complex indoor environments. For example, complex spatial layouts, constantly changing obstacles, and dynamic objects cause localization errors during map building using traditional methods, and in some environments, they cannot even build highly accurate maps. Furthermore, as robot tasks have gradually expanded from simple navigation to object recognition and search, traditional mapping techniques have failed to effectively support the rapid localization and recognition of target objects, leading to low efficiency and task execution delays.

[0004] Therefore, there is an urgent need to propose a method and device for searching items in complex indoor environments to solve the technical problems of low efficiency and inability to accurately search for items when robots search for items based on maps constructed using traditional methods. Summary of the Invention

[0005] In view of this, it is necessary to provide a method and apparatus for searching items in complex indoor environments, in order to solve the technical problems of low efficiency and inability to accurately search for items when robots search for items based on maps constructed by traditional methods in the prior art.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a method for searching for items in complex indoor environments, comprising: Based on the image dataset of a complex indoor environment, objects are divided into environmental objects with relatively fixed positions and dynamic objects with variable positions. A semantic relationship graph is then constructed based on the prior relationship between the environmental objects and the dynamic objects. A semantic layer map containing scene and environmental item information is built based on the semantic relationship graph, and a topological layer map containing the best observation point of the item and the scene connection point is constructed. When the robot detects a target object in the complex indoor environment, it performs path planning based on the semantic layer map, the topology layer map, and the semantic relationship graph to obtain the global path of the target object.

[0007] In one possible implementation, the step of establishing a semantic layer map containing scene and environmental item information based on a semantic relationship graph includes: The image dataset is classified into scenes based on the semantic relationship graph to obtain scene labels; A raster map is constructed as a metric map based on a preset laser model and the scene labels; The image dataset is used to perform object recognition based on a preset object detection model to obtain object recognition results. Based on the position of the object in the recognition result obtained by the depth camera, the object position information is obtained; The object recognition results and the object location information are mapped onto the grid map to obtain a semantic layer map with scene semantic information.

[0008] In one possible implementation, constructing a topology layer map containing the best observation point of the item and scene connection points includes: The coordinate system of the robot's viewpoint and the depth camera's viewpoint are transformed to obtain the coordinate transformation matrix; A topology node is established based on the location of the object observed by the robot in the complex indoor environment at the center of the camera's field of view; the topology node includes the best observation point of the object and the scene connection point. Local path planning and weight definition are performed on the topology nodes to obtain the connection weights between the topology nodes, thereby generating a topology layer map.

[0009] In one possible implementation, establishing topological nodes based on the location of objects observed by the robot in the complex indoor environment at the center of the camera's field of view includes: Semantic extraction is performed on the complex indoor environment and objects based on the metric map and the depth camera to obtain semantic information, and the optimal observation point of the objects is determined based on the semantic information; Doors in the complex indoor environment are detected as segmentation nodes between scenes to obtain scene connection points.

[0010] In one possible implementation, detecting doors in the complex indoor environment as segmentation nodes between scenes to obtain scene connection points includes: The preliminary position of the door frame is determined based on the object recognition results; Based on the preliminary positions of all door frames in all rooms within the complex indoor environment, determine the pre-selected door frame area image for each room; The intersection of the line connecting the left and right door frame positions of the pre-selected door frame area image of each room with the perpendicular bisector is recorded as the segmentation node, thus obtaining the scene connection point.

[0011] In one possible implementation, the step of recording the intersection of the line connecting the left and right door frame positions of the pre-selected door frame area image of each room with the perpendicular bisector as the segmentation node to obtain the scene connection point includes: Point cloud extraction is performed on the image of the pre-selected door frame area of ​​each room to obtain a set of door frame points; The point cloud in the door frame point set is fitted with the door frame using the least squares method to obtain the center line of the corresponding door frame for each room. The vertical distance is calculated based on the point cloud of each room to determine the centerline. The point cloud is divided according to the median of the x-coordinate of the point cloud of each room to obtain the left boundary point set and the right boundary point set; The point with the smallest vertical distance in the set of left boundary points and the set of right boundary points are respectively determined as the left key point and the right key point; The scene connection points are obtained by recording the perpendicular intersections of the left and right key points of each room with the center line as segmentation nodes.

[0012] In one possible implementation, the step of performing path planning based on the semantic layer map, the topology layer map, and the semantic relationship graph to obtain the global path of the target item includes: Based on the semantic relationship graph, determine the semantic association strength between the environmental items and the dynamic items; The items are filtered based on their location and semantic association strength in the semantic layer map to obtain a set of nodes; When the node set is not empty, the node with the maximum semantic association strength in the node set is determined as the target node; When the target node includes the target item, the global path of the target item is obtained based on the location of the target node.

[0013] In one possible implementation, the method further includes: When the node set is empty, the path cost is obtained based on the path length from the location of the target item in the topology map to each topology node; Add the best observation point of the item whose path cost is the minimum among all topological nodes to the node set.

[0014] In one possible implementation, the method further includes: When the target node does not include the target item, the best observation point of the item with the minimum path cost is determined as the guiding node to search for the target item, and a new set of nodes is obtained; A global path is obtained by performing global path planning on the target item based on the new set of nodes.

[0015] Secondly, the present invention also provides an object search device for complex indoor environments, comprising: The object segmentation module is used to segment objects based on image datasets in complex indoor environments, obtaining environmental objects with relatively fixed positions and dynamic objects with variable positions, and to build a semantic relationship graph based on the prior relationships between the environmental objects and the dynamic objects. The image acquisition module is used to build a semantic layer map containing scene and environmental object information based on the semantic relationship graph, and to construct a topological layer map containing the best observation point of the object and the scene connection point; The path planning module is used to perform path planning based on the semantic layer map, the topology layer map, and the semantic relationship graph when the robot detects a target object in the complex indoor environment, so as to obtain the global path of the target object.

[0016] The beneficial effects of this invention are as follows: Objects in a complex indoor environment are divided into relatively fixed environmental objects and dynamically changing objects, thus better handling the dynamic changes in object positions within the indoor environment. A semantic relationship graph is constructed based on the prior relationships between environmental and dynamic objects, thereby improving search efficiency by utilizing these prior relationships. A semantic layer map containing scene and environmental object information is established based on the semantic relationship graph. This semantic layer map provides semantic understanding of the environment, enabling the robot to recognize different scenes and objects. A topology layer map containing the best observation points of objects and scene connection points is constructed. This topology layer map provides structural information about the environment, facilitating efficient navigation and search. When the robot detects target objects in a complex indoor environment, path planning is performed based on the semantic layer map, topology layer map, and semantic relationship graph to obtain the global path of the target object. By combining multi-level map information and the semantic relationship graph, the search path is optimized, taking into account the concurrent relationships between objects and the planned path length, achieving faster and more accurate target object localization. Attached Figure Description

[0017] Figure 1 A schematic flowchart of an embodiment of the item search method for complex indoor environments provided by the present invention; Figure 2 For the present invention Figure 1A schematic diagram of an embodiment of step S101; Figure 3 For the present invention Figure 1 Another embodiment of step S101 is illustrated in the flowchart. Figure 4 This is a schematic flowchart of an embodiment of the confirmed scenario connection point approved by this invention. Figure 5 A schematic diagram of an embodiment of the SCP node provided by the present invention; Figure 6 A schematic diagram of an embodiment of the hierarchical map provided by the present invention; Figure 7 For the present invention Figure 1 A schematic flowchart of an embodiment of step S103; Figure 8 A schematic diagram of an embodiment of the item search device for complex indoor environments provided by the present invention. Detailed Implementation

[0018] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0019] A metric map is a commonly used map representation method in SLAM (Simultaneous Localization and Mapping). It emphasizes accurately representing the positional relationships of objects on the map and typically uses specific geometric and metric information to describe the environment.

[0020] like Figure 1 As shown in the figure, a specific embodiment of the present invention discloses a method for searching for items in a complex indoor environment, including: S101. Based on the image dataset of the complex indoor environment, the objects are divided into environmental objects with relatively fixed positions and dynamic objects with uncertain positions. A semantic relationship graph is constructed based on the prior relationship between the environmental objects and the dynamic objects.

[0021] The classification of environmental and dynamic objects refers to a classification method based on the frequency of location changes. Specifically, time-series analysis algorithms can be used to cluster the movement trajectories of objects. Fixed objects serve as the baseline reference for environmental objects, while dynamic objects are associated with their common activity areas through a relationship graph. Indoor objects are divided into relatively fixed environmental objects (EO), such as beds and sofas, and variable-location dynamic objects (DO), such as water glasses and books. The semantic relationship graph is a knowledge network that expresses the strength of associations between objects. Using the EfficientNet-B2 scene classification architecture and semantic information recognized by YOLOv5s, the overall semantic information of the room is constructed, and the semantic labels and specific coordinates of each object are accurately located.

[0022] S102. Based on the semantic relationship graph, establish a semantic layer map containing scene and environmental item information, and construct a topological layer map containing the best observation point of the item and the scene connection point.

[0023] The semantic layer map is a grid map that integrates object recognition results and location information. Specifically, it uses an object detection model to identify objects in an image dataset, calculates their 3D coordinates using a depth camera, and maps these coordinates onto the grid map, providing a visual representation of object distribution for path planning. The topology layer map is a network structure containing optimal observation points (OOPs) and scene connection points. An optimal observation point (OOP) is the location from which the clearest, most complete, and most reliable information is obtained when observing an object on the map, thus aiding the robot's navigation during path planning. Scene connection points (SCPs) are key transition nodes between different regions or sub-scenes on the map, used to describe spatial connectivity and support efficient path planning and environmental understanding. This allows for the identification of doorway connection points, enabling cross-room navigation. Specifically, coordinate system transformation establishes a relationship between the robot's perspective and map coordinates, setting optimal observation points as topology nodes, and using connection weights to reflect the difficulty of traversing between nodes.

[0024] Specifically, when building the semantic layer map, an object detection model is used to identify environmental objects and obtain their location coordinates, forming a raster map with semantic annotations. During the construction of the topology layer map, the observation data from the depth camera is converted into topology nodes, and the gate frame detection algorithm determines the scene connection points to form a topology network.

[0025] S103. When the robot detects target objects in a complex indoor environment, it performs path planning based on the semantic layer map, topology layer map, and semantic relationship graph to obtain the global path of the target object.

[0026] When the robot begins its search, the semantic relationship graph analyzes the correlation strength between the target item and other items, filtering out topological nodes corresponding to high-probability areas. The local path planning module integrates the node connectivity relationships from the topological layer map with the object location information from the semantic layer map to generate a global path that balances observation efficiency and travel cost.

[0027] Compared with existing technologies, this embodiment provides a method for segmenting objects in complex indoor environments, resulting in relatively fixed environmental objects and dynamically changing objects. This allows for better handling of dynamic changes in object positions within the indoor environment. A semantic relationship graph is constructed based on the prior relationships between environmental and dynamic objects, thereby improving search efficiency by leveraging these prior relationships. A semantic layer map containing scene and environmental object information is built based on the semantic relationship graph. This semantic layer map provides semantic understanding of the environment, enabling the robot to recognize different scenes and objects. A topology layer map containing optimal observation points for objects and scene connection points is constructed. This topology layer map provides structural information about the environment, facilitating efficient navigation and search. When the robot detects target objects in a complex indoor environment, path planning is performed based on the semantic layer map, topology layer map, and semantic relationship graph to obtain the global path of the target object. By combining multi-layered map information and the semantic relationship graph, the search path is optimized, taking into account the concurrent relationships between objects and the planned path length, achieving faster and more accurate target object localization.

[0028] Through the above technical solution, this invention effectively solves the problems of low target object positioning accuracy and poor search path efficiency in complex environments. The semantic layer map ensures the accuracy of object location information, the topology layer map optimizes the rationality of the robot's movement path, and the semantic relationship graph enhances the ability to predict the dynamic object location. The collaborative work of these three elements significantly improves the success rate and execution efficiency of the search task.

[0029] In some embodiments of the present invention, such as Figure 2 As shown, step S101 includes: S201. Classify the image dataset into scenes based on the semantic relationship graph to obtain scene labels.

[0030] Scene classification refers to dividing the input image data into different categories such as kitchen and bedroom according to room function. EfficientNet-B2 is used as the network architecture for scene classification. Specifically, it can be implemented using the pre-trained deep learning model ResNet. Based on the semantic information in the semantic relationship graph, scene classification is performed on the image dataset. By extracting image features and matching them with scene labels, the problem of traditional maps lacking scene attribute descriptions can be solved. The scene classification problem can be represented as a probability estimation problem. The scene label is defined as shown in formula (1): (1) In the formula, : The camera captured the first [number]th ... t Image; : No. Each scene category.

[0031] Based on the scene label, its corresponding likelihood function can be expressed as shown in formula (2): (2) In the formula, The scene type corresponding to the image currently being captured by the camera; Camera self arrive A sequence of images at different times; : No. The first image is for the scene. Discrete probability distribution of the class. If we consider the scene semantic classification problem as a Bayesian estimation problem, the Bayesian filtering formula can be expressed as shown in formula (3) after combining the prior probabilities: (3) S202. Construct a raster map as a measurement map based on a preset laser model and scene labels.

[0032] Among them, the preset laser model refers to a mathematical model of the environmental geometry based on LiDAR scanning data, such as a SLAM model. Specifically, a two-dimensional grid map can be generated using the probabilistic grid method. By fusing scene labels, the semantics of the scene and the geometric structure can be associated, thus eliminating the localization drift defect of traditional SLAM algorithms in dynamic environments.

[0033] S203. Perform object recognition on the image dataset according to the preset target detection model to obtain the object recognition results.

[0034] The preset target detection model refers to a convolutional neural network used to identify the categories of objects in an image. Specifically, it can be implemented using the YOLOv5s model. By using bounding box regression and classification prediction, it can distinguish different categories of objects, thus solving the problem that traditional methods can only detect obstacles but cannot identify specific objects.

[0035] S204. Calculate the position of the object in the recognition result based on the depth camera to obtain the object position information.

[0036] Among them, a depth camera is a visual sensor that can acquire the three-dimensional coordinates of an object. Specifically, it can be implemented using structured light or ToF cameras. By registering the depth information with the RGB image, the three-dimensional position of the object in the robot coordinate system is calculated, and the object's semantic annotation is performed in the grid map to avoid spatial position errors caused by two-dimensional projection.

[0037] S205. Map the object recognition results and object location information onto the grid map to obtain a semantic layer map with scene semantic information.

[0038] Specifically, a pre-trained ResNet model extracts and classifies features from the input image, outputting scene labels to annotate functional areas of the raster map. After probabilistic rasterization, the LiDAR scan data embeds scene labels into the raster attributes of the corresponding regions. A YOLOv5s model performs frame-by-frame image detection, outputting item categories and bounding box information. Combined with the depth camera's 3D coordinate calculation module, the position of items in the camera coordinate system is converted to absolute coordinates in the robot coordinate system. Finally, item categories, 3D coordinates, and scene labels are mapped onto the raster map, forming a semantic layer map containing scene semantics and item spatial information. In this process, scene labels provide room functional attribute constraints for path planning, while item coordinate information provides precise target localization for search tasks; both collectively enhance the map's semantic expressive power.

[0039] Compared to existing technologies, traditional methods rely on single geometric information to construct maps, failing to distinguish between scene functions and item categories, resulting in a lack of semantic guidance for search path planning. This solution, through the collaborative processing of scene classification and object detection, adds scene semantic annotation and item category recognition while preserving geometric accuracy. This allows the map to not only reflect the environmental structure but also clearly mark the locations of specific items such as the sideboard in the kitchen area and the wardrobe in the bedroom area, providing the robot with a semantic reference system for item search. Simultaneously, the 3D coordinate calculation by the depth camera effectively overcomes the scale uncertainty problem of traditional monocular vision, ensuring the spatial accuracy of item location information.

[0040] Through the above technical solution, this application achieves deep fusion of scene semantic information and object location information, enabling the robot to accurately identify the functional area and specific coordinates of a target object in a complex indoor environment. For example, when searching for a water cup, it can prioritize searching for the coordinates of already identified water cups near cabinets in the area marked as the kitchen, avoiding ineffective searches in the bedroom area. This solution significantly improves the positioning accuracy of target objects and solves the problem of low search efficiency caused by the lack of semantic information in traditional methods.

[0041] In some embodiments of the present invention, such as Figure 3 As shown, step S101 further includes: S301. Perform coordinate system transformation between the robot's and the depth camera's perspectives to obtain the coordinate transformation matrix.

[0042] Coordinate system transformation refers to spatially aligning the robot's coordinate system with the depth camera's sensor coordinate system. Specifically, a homogeneous transformation matrix can be used to perform rotation and translation operations in 3D space, eliminating data deviations caused by differences in sensor installation positions. When constructing the topology layer map, a camera perspective adjustment method combining the robot body and a single-degree-of-freedom camera gimbal allows for adjustment of the camera perspective in multiple directions, covering a wider spatial area. For the robot coordinate system, For the camera gimbal coordinate system, To establish a camera coordinate system, the specific steps are as follows: (1) Coordinate transformation matrix; By combining the relevant parameters of the robot-gimbal model, the coordinate transformation matrices of formulas (4) and (5) can be derived.

[0043] (4) (5) In the formula, : These are the transformation matrices from the robot coordinate system to the gimbal coordinate system and from the gimbal coordinate system to the camera coordinate system, respectively; : These are the rotation matrix and the translation matrix, respectively.

[0044] (2) Robot around Horizontal rotation of axis ; The specific angle value is derived and determined using geometric relationships, as shown in formula (6): (6) In the formula, The point where the camera's optical axis intersects the imaging plane in the initial state; Center of the object exist Projection onto a plane.

[0045] (3) By using a single-degree-of-freedom gimbal to rotate Axis rotation ; Ensure the camera's optical axis is aligned with the center point of the object. Alignment is determined based on the coordinate transformation matrix mentioned above. And by combining the trigonometric transformation relationships, we can find As shown in formula (7): (7) In the formula, ; ; .

[0046] S302. Establish topology nodes based on the location of objects observed by the robot in a complex indoor environment at the center of the camera's field of view; the topology nodes include the best observation point of the object and the scene connection point;

[0047] Among them, the environment item nodes in the topology nodes are described by a six-tuple, which includes: a type identifier. Robot posture Camera posture Scene tags Item List and environmental item labels Topology nodes can include the best observation point of an object and scene connection points. Topology node establishment refers to using the position of the object's center point on the camera's imaging plane as the observation reference point. Specifically, this can be achieved by identifying the object using an object detection algorithm and then calculating the spatial mapping relationship between its pixel coordinates and depth information, ensuring that each node corresponds to the optimal viewing angle for the robot to observe the object.

[0048] S303. Perform local path planning and weight definition on the topology nodes to obtain the connection weights between the topology nodes, and then generate the topology layer map.

[0049] In a specific embodiment of the present invention, the connection relationships between topological nodes are weighted using the trajectory length cost of path planning, and are represented as connection weights between topological nodes. The connection weight definition refers to constructing a path cost function based on the travel distance between nodes and the observation quality evaluation value. Specifically, the Dijkstra algorithm can be used to calculate the shortest path, and the weight coefficients can be adjusted by combining the item recognition confidence level to achieve a balance between navigation efficiency and target search accuracy. Connection Weights e The calculation is shown in formula (8): (8) In the formula, : Connection weights between topology nodes; : Distance between topological nodes (e.g., Euclidean distance or path length between two nodes); : The speed of travel (or movement) along the path.

[0050] This formula calculates the connection weight between two topological nodes, taking into account distance and speed. Longer paths or slower speeds result in higher connection weights, thus affecting the priority of the robot's path search.

[0051] Specifically, a coordinate transformation matrix is ​​used to unify the robot's pose and sensor data, providing a precise spatial reference for the topology nodes. After an object is identified and located, its spatial coordinates at the center of the camera's field of view are used as the topology node, ensuring that the robot can collect target features from the optimal perspective. Subsequently, a local path planning algorithm is used to calculate reachable paths between nodes, and the connection weights are comprehensively defined by combining path length, turning angle, and object observation quality. This allows the topology map to dynamically reflect scene connectivity and observation priorities, forming a hierarchical structure that balances navigation feasibility and target search efficiency.

[0052] Compared to existing technologies, traditional topology map construction methods typically establish nodes solely based on spatial connectivity, neglecting the impact of observation angle and path weights on item search. This solution addresses the issue of inaccurate node positioning caused by sensor data bias by integrating a coordinate alignment mechanism and an observation perspective optimization strategy. Furthermore, it introduces a dynamic weight adjustment mechanism based on item observation quality, overcoming the limitation of static topology maps in traditional methods that cannot adapt to dynamic environments. This significantly improves the efficiency of path planning and the accuracy of target search in complex scenarios.

[0053] Through the above technical solutions, this application achieves accurate modeling of scene connection relationships by the robot, reducing the probability of path planning failure caused by interference from dynamic objects; by optimizing the observation angle and connection weight of topological nodes, the utilization efficiency of environmental information during the object search process is enhanced, enabling the robot to quickly generate a global path that takes into account both navigation feasibility and target positioning accuracy in complex indoor environments.

[0054] In some embodiments of the present invention, step S302 includes: Semantic extraction is performed on complex indoor environments and objects using metric maps and depth cameras to obtain semantic information, and the optimal observation point for the objects is determined based on the semantic information.

[0055] The BOP topology nodes utilize the classic SLAM algorithm Gmapping to generate a metric map. An RGB-D camera observes the scene and surrounding objects in real time, extracting semantic information and determining the best observation point (BOP) for constructing the topology nodes. The best observation point refers to the optimal position corresponding to the center of the camera's field of view when the robot observes the target object. This can be achieved by combining the depth camera's 3D coordinate transformation technology with semantic information to identify the spatial distribution characteristics of objects, thus determining the localization benchmark for stable robot observation of objects.

[0056] Doors in complex indoor environments are detected as segmentation nodes between scenes to obtain scene connection points.

[0057] The SCP (Scene Connection Point) topology node is designed to improve the robot's maneuverability and path planning efficiency in complex indoor environments. A door, a natural object, is chosen as the segmentation node between scenes, also known as a scene connection point (SCP). A scene connection point refers to the physical separation location between different scene areas. Specifically, it can be achieved by using a door frame geometric feature detection algorithm to fit door frames to point cloud data. The segmentation location is determined by identifying the door frame centerline and boundary key points, thus establishing the topological relationships between scenes.

[0058] Specifically, the spatial coordinates and semantic labels of environmental objects are obtained through the semantic extraction function of the depth camera. Combined with the rasterized spatial information in the metric map, the optimal observation angle and distance of the objects in the robot coordinate system are calculated, forming the optimal observation points of the objects in the topology nodes. At the same time, taking advantage of the characteristic of door frames as natural boundary markers of the scene, the left and right boundary point clouds of the door frame region in the point cloud data are segmented. The least squares method is used to fit the center line of the door frame and calculate the vertical intersection point. The intersection point is embedded as the scene connection point into the topology layer map.

[0059] Compared to existing technologies, traditional methods, which rely on single sensor data to segment scene regions, tend to overlook the structural features of door frames, leading to inaccurate scene segmentation. This solution, by fusing semantic information and geometric feature detection, ensures the positioning accuracy of object observation points while utilizing door frames as physical boundaries to achieve efficient scene region segmentation, avoiding subjective errors from manually setting segmentation nodes.

[0060] Through the above technical solution, this application effectively solves the problem of low path planning efficiency caused by insufficient scene segmentation accuracy in complex indoor environments. By generating topological nodes through dual constraints of semantic and geometric features, it not only improves the stability of observation and positioning during the item search process, but also realizes the automatic identification of the connection relationship between scenes, providing reliable spatial topology support for global path planning.

[0061] In some embodiments of the present invention, doors in complex indoor environments are detected as segmentation nodes between scenes to obtain scene connection points, including: The initial position of the door frame is determined based on the object recognition results.

[0062] The preliminary position of the door frame refers to the two-dimensional or three-dimensional coordinate range obtained by initially locating the edge of the door frame through the object detection model. Specifically, the door frame contour can be extracted using a deep learning-based image segmentation algorithm, combined with point cloud registration technology. This step uses the object recognition results as the basic data source to ensure the reliability of the door frame positioning. The position of the door is detected using a model trained with YOLOv5s, and the preliminary position of the door frame is obtained, as shown in formula (9): (9) In the formula, : Pixel coordinates of the left and right boundaries of the door frame detection box; : Camera depth data; Camera focal length.

[0063] Based on the preliminary positions of all door frames in all rooms within a complex indoor environment, determine the pre-selected door frame area image for each room.

[0064] Among them, the pre-selected door frame area image refers to the candidate area image generated based on the preliminary positions of the door frames of all rooms. Specifically, a multi-room door frame position clustering algorithm can be used to filter out the door frame area corresponding to each room through spatial distribution features. This step eliminates single-point detection errors by integrating global door frame data. Specifically, the image of the rectangular area occupying the door position ±1.25m in the two-dimensional grid map is used as the pre-selected door frame area image of the corresponding room, as shown in formula (10): (10) The intersection of the line connecting the left and right door frame positions of the pre-selected door frame area image of each room with the perpendicular bisector is recorded as the segmentation node, thus obtaining the scene connection point.

[0065] The intersection of the line connecting the left and right door frames and the perpendicular bisector is the center point of the door frame calculated based on geometric relationships. Specifically, the Hough transform can be used to detect the straight edges of the door frame, and the intersection of the perpendicular bisector can be determined by orthogonal projection. This step utilizes the symmetry of the physical structure of the door frame to achieve precise positioning of the segmentation node. Specifically, the intersection of the line connecting the left and right door frames of each room in the two-dimensional occupied grid map (i.e., the position of the top door frame) and the perpendicular bisector is recorded as the door node (i.e., the SCP node), as shown in formula (11): (11) Specifically, the process begins by using an object detection model to initially identify door frames in the indoor environment, generating a point cloud dataset containing door frame edges. Then, spatial clustering is performed on the door frame locations across multiple rooms to select candidate regions for each room, forming pre-selected door frame region images. Finally, a geometric computation model is used to extract the straight-line features of the left and right door frame edges, and the intersection of the perpendicular bisectors is determined through orthogonal projection as the scene connection point. For example, the least squares method is used to fit the straight lines of the door frame edges in the point cloud data, and the geometric center point of the scene segmentation is obtained by calculating the intersection of the perpendicular bisector of the line connecting the left and right door frames and the perpendicular bisector. This establishes a scene segmentation benchmark based on the door frame structure, ensuring the spatial consistency of the segmentation nodes.

[0066] Compared to existing technologies, traditional methods rely on single sensor data or simple geometric rules for scene segmentation, such as scanning door frame height information using LiDAR or segmenting door frame areas based on color thresholds. This is prone to positioning errors due to changes in ambient lighting or occlusion. Our solution, however, combines object recognition with multi-room door frame data integration, calculating segmentation nodes through geometric relationships. This eliminates local detection errors and improves the global consistency of scene segmentation.

[0067] Through the above technical solution, this application solves the problem of scene connection point positioning deviation caused by inaccurate door frame detection in complex indoor environments using traditional methods, and achieves high-precision scene segmentation based on multi-room door frame data integration and geometric feature calculation. Specifically, it eliminates the impact of single-point detection errors on scene segmentation, ensures the positioning accuracy of the door frame center point, and thus provides accurate scene boundary information for robot path planning.

[0068] In some embodiments of the present invention, such as Figure 4 As shown, the intersection of the line connecting the left and right door frame positions in the pre-selected door frame area image of each room with the perpendicular bisector is recorded as the segmentation node, resulting in scene connection points, including: S401. Extract point clouds from the pre-selected door frame area images of each room to obtain a set of door frame points.

[0069] Point cloud extraction refers to obtaining the spatial coordinate information of the door frame region from the 3D point cloud data collected by the depth camera. Specifically, it can be implemented using a region growing algorithm or a boundary detection algorithm to accurately capture the physical structural features of the door frame. Point cloud extraction is performed on the candidate region to obtain the set of door frame points, as shown in formula (12): (12) S402. Using the least squares method, fit the point cloud in the door frame point set to obtain the center line of the corresponding door frame for each room.

[0070] Among them, least squares fitting refers to solving the best fitting line of the door frame point cloud through mathematical optimization methods. Specifically, it can be implemented using a linear regression model to eliminate errors caused by measurement noise and accurately restore the geometric shape of the door frame. The centerline equation of the door frame point cloud is fitted using least squares, as shown in formula (13): (13) S403. Calculate the vertical distance to the centerline based on the point cloud of each room.

[0071] For each point Calculate the vertical distance to the center line, as shown in formula (14): (14) S404. Divide the point cloud according to the median of the x-coordinate of the point cloud of each room to obtain the set of left boundary points and the set of right boundary points.

[0072] The horizontal median division refers to segmenting the point cloud data into left and right regions based on the horizontal distribution characteristics. This is achieved by calculating the median horizontal coordinate of the point cloud dataset, used to eliminate the influence of asymmetric interference points on boundary determination. Based on the median horizontal coordinate... The point cloud is divided into two groups, left and right. Among them: : Set of left boundary points; : Set of right boundary points.

[0073] S405. Determine the point with the smallest vertical distance in the left boundary point set and the right boundary point set as the left key point and the right key point, respectively.

[0074] Among them, the points with the smallest perpendicular distance to the center line from the left and right point sets are selected as the left and right key points, as shown in formulas (15) and (16): (15) (16) S406. Based on the perpendicular intersection of the left and right key points of each room with the center line, the scene connection points are obtained.

[0075] Among them, determining the vertical intersection point refers to calculating the orthogonal projection position of the key point on the center line through the geometric projection method. Specifically, it can be achieved by using the vector projection formula to ensure that the position of the segmentation node conforms to the geometric center characteristics of the actual structure of the door frame.

[0076] Specifically, in the image processing of the pre-selected door frame area, a 3D coordinate dataset of the door frame area is first obtained through point cloud extraction technology, forming a set of spatial points containing the structural features of the door frame. Then, the least squares method is used to fit a straight line to this point set, generating the best-fit straight line representing the central axis of the door frame. This process effectively eliminates point cloud offsets caused by uneven object surfaces or measurement errors. After obtaining the center line, the vertical distance from each point cloud to this line is calculated, and combined with the median of the x-coordinate, the point cloud is divided into left and right subsets. This division method effectively separates interference points on both sides of the door frame. Further, key points closest to the center line are selected from the left and right subsets. These two key points represent the actual boundary positions of the left and right sides of the door frame closest to the center line, respectively. Finally, the precise location of the scene connection points is determined by calculating the intersection of the vertical projections of the left and right key points on the center line, such as... Figure 5 As shown, Figure 5The blank areas between the connecting lines are doors, that is, the areas framed by small squares. The scene connection point of the door (i.e., the SCP node) is the center point of the blank area. This point maintains the geometric symmetry of the door frame and accurately reflects the topological connection relationship between scenes.

[0077] Compared to existing technologies, traditional methods typically rely on 2D image processing for door frame detection, which is easily affected by changes in lighting or occlusions. Our proposed solution, however, utilizes 3D point cloud data processing combined with mathematical modeling to effectively resist environmental noise, especially in scenarios with partially occluded door frames or complex decorative structures, enabling accurate extraction of the door frame's geometric features. Existing segmentation methods based on color or texture features tend to fail when the door frame and wall colors are similar; our solution ensures the reliability of the detection results through geometric feature analysis.

[0078] By employing the aforementioned technical solution, this application addresses the problem of scene segmentation node offset caused by inaccurate door frame position detection in complex indoor environments, achieving accurate calculation of door frame connection points. This solution effectively distinguishes between the interference of real door frame structures and decorative lines, ensuring that scene segmentation nodes are located at the actual geometric center of the door frame, providing accurate scene connection information for subsequent topology map construction. By combining geometric constraints with mathematical optimization, the adaptability of the door frame detection algorithm to complex indoor environments is enhanced, improving the stability and reliability of scene segmentation node localization.

[0079] In embodiments of the present invention, such as Figure 6 As shown, environmental semantic information can be obtained through an RGB-D camera, thereby constructing a semantic layer map. Based on the LiDAR, a metric layer map can be constructed. Through the item nodes and gate nodes in the metric layer map, a topology layer map can be constructed. Through the semantic layer map and the topology layer map, the required hierarchical map can be constructed.

[0080] In some embodiments of the present invention, such as Figure 7 As shown, step S103 includes: S701. Determine the semantic association strength between environmental items and dynamic items based on the semantic relationship graph.

[0081] Semantic association strength refers to an indicator quantified by a semantic relationship graph of the association between environmental and dynamic objects. This can be implemented using association degree algorithms or probabilistic models, such as calculating association weights based on item co-occurrence frequency or spatial proximity. This indicator is used to filter out key nodes with potential associations to the target object, avoiding redundant searches of irrelevant areas in dynamic environments. The semantic association strength between dynamic objects and environmental objects is expressed by the formula... The definition is as shown in formula (17): (17) In the formula, The strength of the relationship between items reflects the degree of association between the items and the items in the environment; The frequency of dynamic objects and environmental items appearing simultaneously in an image; Total frequency of occurrence of dynamic objects. Association strength. The larger the value, the stronger the semantic relationship between the dynamic object and the surrounding objects.

[0082] S702. Filter items based on the location and semantic association strength of the target items in the semantic layer map to obtain a set of nodes.

[0083] The node set refers to the set of candidate observation points extracted from the semantic layer map through a filtering algorithm. This can be implemented using threshold filtering or sorting mechanisms, such as retaining nodes with a correlation strength higher than a preset value. This set is used to narrow the search scope and improve path planning efficiency, for example, by combining the location of the target item with the semantic correlation strength. Filter out the set of nodes related to the target item. .like If the set is empty, the generic bootstrap node (BOP node) with the lowest path cost will be added to the set.

[0084] S703. When the node set is not empty, the node with the maximum semantic association strength in the node set is determined as the target node.

[0085] Here, the target node refers to the observation point with the highest semantic association strength in the node set. Specifically, it can be determined using a maximum value comparison algorithm, for example, by traversing the node set. And record the current maximum association strength value. The corresponding node. This node serves as the priority search area to ensure the effectiveness of path planning. If there are no nodes with a correlation strength exceeding 0.6, the node with the lowest path cost is selected as the guiding node for the next step.

[0086] S704. When the target node includes the target item, obtain the global path of the target item based on the location of the target node.

[0087] The global path refers to the navigation route from the robot's current position to the location of the target item. This route can be generated using path planning algorithms, such as those based on... The algorithm, or Dijkstra's algorithm, calculates the optimal path, selecting the best path from the topology map to optimize search efficiency. This path is generated directly when the target node contains the target item, achieving simultaneous optimization of search and path planning. The heuristic function of algorithm A... The calculation formula is shown in formula (18): (18) In the formula, : Estimated cost from the starting node to the target node; The actual cost from the starting node to the current node; The heuristic cost estimation from the current node to the target node is performed using a heuristic function (e.g., Euclidean distance or Manhattan distance). The formula is... The core part of the algorithm is used to calculate the total cost of a given node. In practical applications, heuristic functions... It will select the appropriate calculation method based on the specific environment and task.

[0088] Specifically, this method analyzes the potential associations between environmental and dynamic items using a semantic relationship graph, quantifying them into a semantic association strength index. Combining the target item's location in the semantic layer map, this index is used to filter candidate observation points, generating a node set. The node with the highest association strength in the set is identified as the target node and observed first. When a target node contains the target item, a global path is generated based on its location information. This process dynamically adjusts the search priority based on semantic association strength, avoiding repeated path adjustments due to environmental changes and reducing ineffective searches of low-association areas.

[0089] Compared to existing technologies, traditional methods typically rely on geometric information or the real-time location of dynamic objects for path planning, lacking analysis of semantic relationships between objects. This results in low search efficiency and susceptibility to dynamic interference. For example, existing technologies may employ dynamic obstacle avoidance strategies, but cannot predict the possible location of target objects. In contrast, this solution establishes logical connections between objects through semantic association strength, enabling rapid identification of high-probability areas in dynamic environments and significantly reducing the computational load of path planning.

[0090] Through the above technical solutions, this application effectively solves the problems of low path planning efficiency and insufficient search accuracy in dynamic environments. By filtering key nodes through semantic association strength, the number of searches in irrelevant areas is reduced, shortening the global path generation time; prioritizing target nodes with high association strength improves the positioning accuracy of target items; and directly generating a global path when a target node contains a target item avoids repeated path adjustments caused by dynamic interference, enhancing the robustness of the system.

[0091] In some embodiments of the present invention, the method further includes: When the node set is empty, the path cost is obtained based on the path length from the location of the target item in the topology layer map to each topology node.

[0092] Path cost refers to the distance or resource consumption required for the robot to travel from its current position to various topological nodes. Specifically, it can be calculated using Euclidean distance or Dijkstra's algorithm to determine the path length between nodes, quantifying reachability metrics to support node selection. Indicates starting from the current position To the node The cost:

[0093] According to the cost The minimum value is used to dynamically adjust the search path.

[0094] Add the best observation point of the item with the minimum path cost among all topological nodes to the node set.

[0095] In this context, a topological node refers to the optimal viewing position when the robot observes an object, i.e., the best observation point of the object. Specifically, the camera's field of view center point can be determined through depth camera coordinate system transformation as the node coordinate, establishing a spatial reference frame for the topological layer map. A node set refers to a temporary storage structure for candidate path nodes, which can be dynamically maintained using a priority queue or hash table to store candidate nodes after semantic association filtering. If the node set... If the node is empty, select the node with the lowest path cost from all nodes as the guiding node. As shown in formula (19): (19) Specifically, when the semantic association strength screening criteria are too strict, resulting in an empty node set, the system automatically traverses all topological nodes and calculates the path length from each node to the target item's location, converting the path length into a path cost. By comparing the path costs of all nodes, the system selects the node with the lowest path cost as a supplementary node to add to the set. For example, when the target item is located in a corner of a room and the surrounding environment has weak associations with dynamic items, the semantic relationship graph may not be able to filter out effective nodes. In this case, the system selects the nearest corridor entrance node to add to the set based on the principle of minimum path cost, allowing the robot to continue path planning.

[0096] Compared to existing technologies, traditional methods fail to generate effective paths when the node set is empty, leading to task interruption. This solution, however, establishes a fault-tolerant mechanism for path planning by dynamically evaluating path costs and forcibly adding nodes. Existing technologies rely on fixed semantic association filtering conditions and lack node addition strategies, while this solution introduces a correlation calculation between path length and the spatial distribution of topological nodes, ensuring that added nodes have optimal reachability.

[0097] Through the above technical solution, this application effectively solves the problem of missing nodes caused by overly strict semantic filtering conditions during the target item search process. By dynamically evaluating path costs and supplementing nodes with minimum cost, the robot is prevented from stopping the search due to the lack of available nodes, thus ensuring the continuity of the item search task in complex indoor environments.

[0098] In some embodiments of the present invention, the method further includes: When the target node does not include the target item, the best observation point of the item with the minimum path cost is determined as the guiding node to search for the target item, and a new set of nodes is obtained. A global path is obtained by performing global path planning on the target item based on the new set of nodes.

[0099] In a specific embodiment of the present invention, when the target item is not covered by the initially planned target node, the node with the lowest path cost from each topological node to the target item is selected as the guiding node by calculating the path cost from each topological node to the target item. After the guiding node is expanded into a new node set, global path planning is performed again based on the connection weights of the topological layer map and the association strength of the semantic relationship graph. This process dynamically binds path costs to topological nodes, responds to environmental changes in real time, and avoids the robot getting stuck in invalid path loops. For example, in a scenario where the dynamic location of an item changes abruptly, the guiding node can quickly locate the potential search area, the new node set covers more possible locations through neighborhood expansion, and the final generated global path will prioritize accessing areas with high semantic association strength, while optimizing the topological connection weights to reduce the travel distance.

[0100] The robot combines topological and grid maps for path planning, ensuring efficient execution of item search tasks and minimizing the shortest path cost. The calculation is shown in formula (20): (20) In the formula, The shortest cost of path planning; The connection weight of each segment in the path is calculated based on the connection relationships between topological nodes; The number of nodes in the path represents the number of nodes traversed from the starting node to the target node. This formula calculates the shortest path cost for the robot from the starting node to the target node, optimizing path selection during item search and reducing unnecessary traversals.

[0101] In some specific implementations, the path cost calculation can be weighted by combining robot movement speed and environmental complexity factors, for example, by incorporating obstacle density as an adjustment coefficient into the distance calculation model. The generation of a new node set can employ a breadth-first search algorithm, expanding outwards from the guiding node to form a three-layer neighborhood of nodes, thus creating a candidate search region.

[0102] Compared to existing technologies, traditional methods rely on static maps for path planning, which cannot handle search failures caused by dynamic changes in item locations. This solution achieves real-time adjustment of the search path through a dynamic path cost evaluation mechanism and a guiding node expansion strategy. For example, existing technologies require rebuilding the global map when the target item moves, while this solution only needs to expand local nodes based on the topology layer map, significantly reducing computational resource consumption.

[0103] Through the above technical solutions, this application can quickly generate alternative paths when the initial search path fails, effectively improving the search efficiency for dynamic items. By employing a dynamic expansion mechanism of guiding nodes and new node sets, the robot avoids getting stuck in local optimum path loops, enhancing search robustness in complex indoor environments. Semantic association and topological weights are integrated during global path planning to ensure that the search process takes into account both the probability distribution of item positions and movement path optimization.

[0104] To better implement the item search method for complex indoor environments in this invention, this invention also provides an item search device for complex indoor environments, based on the existing method. Figure 8 As shown, the item search device 800 for complex indoor environments includes: The object segmentation module 801 is used to segment objects in a complex indoor environment, obtain environmental objects with relatively fixed positions and dynamic objects with variable positions, and build a semantic relationship graph based on the prior relationship between environmental objects and dynamic objects. The image acquisition module 802 is used to build a semantic layer map containing scene and environmental item information based on the semantic relationship graph, and to construct a topological layer map containing the best observation point of the item and the scene connection point; The path planning module 803 is used to perform path planning based on the semantic layer map, topology layer map and semantic relationship graph when the robot detects target objects in a complex indoor environment, so as to obtain the global path of the target object.

[0105] The item search device 800 for complex indoor environments provided in the above embodiments can realize the technical solutions described in the above embodiments of the item search method for complex indoor environments. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the item search method for complex indoor environments, and will not be repeated here.

[0106] The above provides a detailed description of the method and apparatus for searching items in complex indoor environments provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for searching items in a complex indoor environment, characterized in that, include: Based on the image dataset of a complex indoor environment, objects are divided into environmental objects with relatively fixed positions and dynamic objects with variable positions. A semantic relationship graph is then constructed based on the prior relationship between the environmental objects and the dynamic objects. A semantic layer map containing scene and environmental item information is built based on the semantic relationship graph, and a topological layer map containing the best observation point of the item and the scene connection point is constructed. When the robot detects a target object in the complex indoor environment, it performs path planning based on the semantic layer map, the topology layer map, and the semantic relationship graph to obtain the global path of the target object.

2. The method for searching items in complex indoor environments according to claim 1, characterized in that, The establishment of a semantic layer map containing scene and environmental item information based on a semantic relationship graph includes: The image dataset is classified into scenes based on the semantic relationship graph to obtain scene labels; A raster map is constructed as a metric map based on a preset laser model and the scene labels; The image dataset is used to perform object recognition based on a preset object detection model to obtain object recognition results. Based on the position of the object in the recognition result obtained by the depth camera, the object position information is obtained; The object recognition results and the object location information are mapped onto the grid map to obtain a semantic layer map with scene semantic information.

3. The method for searching items in complex indoor environments according to claim 2, characterized in that, The construction of the topology layer map, which includes the best observation point of the item and the scene connection point, includes: The coordinate system of the robot's viewpoint and the depth camera's viewpoint are transformed to obtain the coordinate transformation matrix; A topology node is established based on the location of the object observed by the robot in the complex indoor environment at the center of the camera's field of view; the topology node includes the best observation point of the object and the scene connection point. Local path planning and weight definition are performed on the topology nodes to obtain the connection weights between the topology nodes, thereby generating a topology layer map.

4. The method for searching items in complex indoor environments according to claim 3, characterized in that, The step of establishing topology nodes based on the location of objects observed by the robot in the complex indoor environment at the center of the camera's field of view includes: Semantic extraction is performed on the complex indoor environment and objects based on the metric map and the depth camera to obtain semantic information, and the optimal observation point of the objects is determined based on the semantic information; Doors in the complex indoor environment are detected as segmentation nodes between scenes to obtain scene connection points.

5. The method for searching items in complex indoor environments according to claim 4, characterized in that, The step of detecting doors in the complex indoor environment as segmentation nodes between scenes to obtain scene connection points includes: The preliminary position of the door frame is determined based on the object recognition results; Based on the preliminary positions of all door frames in all rooms within the complex indoor environment, determine the pre-selected door frame area image for each room; The intersection of the line connecting the left and right door frame positions of the pre-selected door frame area image of each room with the perpendicular bisector is recorded as the segmentation node, thus obtaining the scene connection point.

6. The method for searching items in complex indoor environments according to claim 5, characterized in that, The step of recording the intersection of the line connecting the left and right door frame positions of the pre-selected door frame area image of each room with the perpendicular bisector as the segmentation node to obtain the scene connection point includes: Point cloud extraction is performed on the image of the pre-selected door frame area of ​​each room to obtain a set of door frame points; The point cloud in the door frame point set is fitted with the door frame using the least squares method to obtain the center line of the corresponding door frame for each room. The vertical distance is calculated based on the point cloud of each room to determine the centerline. The point cloud is divided according to the median of the x-coordinate of the point cloud of each room to obtain the left boundary point set and the right boundary point set; The point with the smallest vertical distance in the set of left boundary points and the set of right boundary points are respectively determined as the left key point and the right key point; The scene connection points are obtained by recording the perpendicular intersections of the left and right key points of each room with the center line as segmentation nodes.

7. The method for searching items in complex indoor environments according to claim 1, characterized in that, The step of performing path planning based on the semantic layer map, the topology layer map, and the semantic relationship graph to obtain the global path of the target item includes: Based on the semantic relationship graph, determine the semantic association strength between the environmental items and the dynamic items; The items are filtered based on their location and semantic association strength in the semantic layer map to obtain a set of nodes; When the node set is not empty, the node with the maximum semantic association strength in the node set is determined as the target node; When the target node includes the target item, the global path of the target item is obtained based on the location of the target node.

8. The method for searching items in complex indoor environments according to claim 7, characterized in that, The method further includes: When the node set is empty, the path cost is obtained based on the path length from the location of the target item in the topology map to each topology node; Add the best observation point of the item whose path cost is the minimum among all topological nodes to the node set.

9. The method for searching items in complex indoor environments according to claim 8, characterized in that, The method further includes: When the target node does not include the target item, the best observation point of the item with the minimum path cost is determined as the guiding node to search for the target item, and a new set of nodes is obtained; A global path is obtained by performing global path planning on the target item based on the new set of nodes.

10. A device for searching items in a complex indoor environment, characterized in that, include: The object segmentation module is used to segment objects based on image datasets in complex indoor environments, obtaining environmental objects with relatively fixed positions and dynamic objects with variable positions, and to build a semantic relationship graph based on the prior relationships between the environmental objects and the dynamic objects. The image acquisition module is used to build a semantic layer map containing scene and environmental object information based on the semantic relationship graph, and to construct a topological layer map containing the best observation point of the object and the scene connection point; The path planning module is used to perform path planning based on the semantic layer map, the topology layer map, and the semantic relationship graph when the robot detects a target object in the complex indoor environment, so as to obtain the global path of the target object.

Images