Path planning methods for robots, robots, and computer-readable storage media

Abstract

This application discloses a path planning method for a robot, a robot, and a computer-readable storage medium. The path planning method includes: acquiring a global semantic map; wherein the global semantic map is used to represent the semantic information of each object in the explored environment and the positional relationship between each object and the robot; extracting a local semantic map from the global semantic map based on the robot's current position; determining a target frontier point with exploration value in the local semantic map; and planning a path for the robot to reach the target frontier point in the global semantic map. This approach can significantly improve the efficiency and real-time performance of target-driven exploration while reducing computational complexity.

Description

Technical Field

[0001] This application relates to the field of robot navigation technology, and in particular to a path planning method for a robot, a robot, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of science and technology, intelligent mobile robots have become a key force in many fields, including industrial automation, service industries, military, and scientific research. In performing these tasks, autonomous environment exploration technology plays a crucial role, directly impacting a robot's adaptability to unfamiliar environments, deployment costs, and subsequent maintenance expenses. Good environment exploration capabilities not only effectively improve the efficiency and safety of robot task execution, but also enable them to replace humans in performing challenging tasks in complex and dynamic environments, such as indoor delivery, disaster relief, and even military reconnaissance.

[0003] Specifically, how to autonomously explore the environment and efficiently find a designated target object is a crucial technical issue that has garnered significant attention. In recent years, as robots have played an increasingly important role in household services, the need for this capability has become more urgent. In goal-driven exploration tasks in unknown environments, robots not only need to rely on the geometric information of maps for decision-making, but also need to understand the semantic information within the explored environment to comprehend environmental features, spatial layout, and the inherent logical relationships between objects within the environment, in order to find the target object more efficiently and avoid repetitive and inefficient exploration. Summary of the Invention

[0004] This application provides a path planning method for a robot, a robot, and a computer-readable storage medium, which can significantly improve the efficiency and real-time performance of goal-driven exploration while reducing computational complexity.

[0005] The first aspect of this application provides a path planning method for a robot, comprising: acquiring a global semantic map; wherein the global semantic map is used to represent the semantic information of each object in the explored environment and the positional relationship between each object and the robot; extracting a local semantic map from the global semantic map based on the robot's current position; determining a target frontier point with exploration value in the local semantic map; and performing path planning for the robot to reach the target frontier point in the global semantic map.

[0006] In some embodiments, determining target frontier points with exploration value in a local semantic map includes: detecting boundary points in the local semantic map; if boundary points exist in the local semantic map, clustering the boundary points in the local semantic map to obtain local frontier points; and determining target frontier points with exploration value among the local frontier points.

[0007] In some embodiments, after detecting boundary points in the local semantic map, the method further includes: if there are no boundary points in the local semantic map, detecting boundary points in the global semantic map; if there are boundary points in the global semantic map, clustering the boundary points in the global semantic map to obtain global frontier points; and identifying target frontier points with exploration value among the global frontier points.

[0008] In some embodiments, detecting boundary points in a local semantic map or in a global semantic map includes: traversing all grids of the local semantic map or the global semantic map; if the target grid of the local semantic map or the global semantic map is in an explored state and there are unexplored adjacent grids of the target grid, then the target grid is determined to be a boundary point of the local semantic map or the global semantic map.

[0009] In some embodiments, clustering boundary points in a local semantic map to obtain local frontier points, or clustering boundary points in a global semantic map to obtain global frontier points, includes: performing density clustering on the boundary points of the local or global semantic map to obtain at least one cluster; and determining the geometric centroid of the boundary points of each cluster as the local or global frontier point of the corresponding cluster.

[0010] In some embodiments, determining a target frontier with exploration value among local frontier points or among global frontier points includes: performing at least one of geometric evaluation, semantic evaluation, or path cost evaluation on the local or global frontier points to obtain geometric evaluation results, semantic evaluation results, or path cost evaluation results, respectively; wherein the geometric evaluation results are determined based on the size of the boundary where the frontier point is located, the semantic evaluation results are determined based on the semantic association between the objects adjacent to the frontier point and the target object, and the path cost evaluation results are determined based on the path length from the robot's current position to the frontier point; weighting the geometric evaluation results, semantic evaluation results, or path cost evaluation results to determine the final evaluation result of the local or global frontier point; and determining the target frontier with exploration value based on the final evaluation result of the local or global frontier point.

[0011] In some embodiments, semantic evaluation includes: identifying candidate objects among the explored objects that satisfy preset spatial correlation conditions with the leading edge point; wherein the spatial correlation conditions include: there is no occlusion between the leading edge point and the explored objects, and the distance between the leading edge point and the explored objects is less than a set distance threshold; determining the semantic correlation between the candidate objects and the target object to obtain a semantic evaluation result.

[0012] In some embodiments, after obtaining the global semantic map, the method further includes: determining whether a target object exists in the currently explored environment based on the global semantic map; if it exists, performing path planning for the robot to reach the target object in the global semantic map; if it does not exist, performing a step of extracting a local semantic map from the global semantic map based on the robot's current position.

[0013] A second aspect of this application provides a robot including a processor and a memory, the memory for storing program data and the processor for executing the program data to implement the robot path planning method as described above.

[0014] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the path planning method for a robot as described above.

[0015] The above approach, through the decision logic of "if there is no target in the global map, then capture the local map", focuses the exploration on the semantically rich region near the robot, thereby reducing the computational burden and significantly improving the spatiotemporal efficiency and semantic guidance of target-driven exploration. This enables the robot to quickly and intelligently enter the unexplored area that is most likely to contain the target object.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the robot path planning method provided in this application; Figure 2 This is a flowchart illustrating a method for generating / updating a global semantic map in one embodiment; Figure 3 This is a flowchart illustrating step S13 in one embodiment; Figure 4 This is a flowchart illustrating step S133 in one embodiment; Figure 5 This is a flowchart illustrating the second embodiment of the robot path planning method provided in this application; Figure 6 This is a flowchart illustrating the third embodiment of the robot path planning method provided in this application; Figure 7This is a flowchart illustrating the fourth embodiment of the robot path planning method provided in this application; Figure 8 This is a schematic diagram of the framework of an embodiment of the robot provided in this application; Figure 9 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0019] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0021] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] In some embodiments, the strategies for mobile robots to autonomously explore unknown environments mainly include: boundary-based environment exploration methods, sampling-based environment exploration methods, graph search-based environment exploration methods, and learning-based environment exploration methods.

[0023] Among them, the boundary-based environmental exploration method mainly relies on geometric indicators such as information gain and path cost for boundary evaluation. However, the boundary-based environmental exploration method cannot enable the robot to use semantic prior knowledge to prioritize the exploration of boundaries that are highly relevant to the task objective. This leads to the robot being prone to inefficient exploration in areas unrelated to the task objective, thereby increasing path cost.

[0024] Among them, sampling-based environment exploration methods generate candidate target points randomly or heuristically in space and evaluate their information utility and movement cost to decide the next exploration location; graph search-based environment exploration methods abstract the environment into a graph structure and use search algorithms to find exploration paths in the graph that maximize information acquisition (such as covering unknown nodes). However, both sampling-based and graph search-based environment exploration methods have significant limitations in achieving efficient and intelligent target search, thus exhibiting obvious applicability limitations.

[0025] Learning-based environment exploration methods utilize machine learning (especially reinforcement learning) to learn exploration strategies directly from the data end-to-end, determining how to act to efficiently cover unknown environments. However, these methods are only applicable to the closed target object categories used during training and struggle to generalize to object categories not seen during training. Furthermore, learning-based methods heavily rely on simulation data used during policy training, leading to difficulties in transferring simulation data to real-world applications.

[0026] See Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the robot path planning method provided in this application. The path planning method is as follows: Step S11: Obtain the global semantic map.

[0027] The global semantic map is used to represent the semantic information of each object in the explored environment, as well as the positional relationships between each object and the robot. Specifically, the global semantic map is a high-dimensional environmental representation constructed by the robot when exploring and navigating in an unknown environment. Its core tasks are twofold: first, to assign semantic labels (such as category, function, attributes, etc.) to various objects detected in the explored area, forming a semantic understanding; second, to record the spatial positional relationships (including distance, azimuth, relative pose, etc.) between each object and the robot itself, thereby associating semantic information with geometric measurements.

[0028] Optionally, in one embodiment, the robot may update the global semantic map at an appropriate time, such as after step S11, upon startup, before path planning, or during periodic updates.

[0029] In one embodiment, the global semantic map can be updated by: acquiring the robot's current global coordinates and sensor data and preprocessing them, and updating the global semantic map based on the robot's current coordinates and sensor data.

[0030] Specifically, global coordinates are the robot's real-time position and pose information obtained in the global coordinate system through its onboard odometry, providing a pose basis for subsequent localization and map updates. Sensor data includes visual image data (RGB) and depth data output from the RGB-D camera. Global coordinates and sensor data are aligned in time and space. Further, based on the RGB-D sensor information and the robot's current position information, a global semantic map of the environment is updated and constructed. The global semantic map is used to represent the semantic category information of each object in the explored environment, as well as the relative spatial position relationship between each object and the robot. Its specific construction process is as follows: The robot uses the RGB-D sensor to acquire environmental perception data, and uses an open vocabulary detection model for target detection and segmentation. Relying on the model's excellent open vocabulary generalization ability, it can effectively identify unknown objects that are not in a preset category, obtaining the category information of all objects within the robot's current field of view. Then, based on the hardware parameters of the RGB-D sensor (such as field of view angle, lens distortion coefficient, multi-lens extrinsic parameters, etc.), the following are calculated: 1. Point cloud data corresponding to the depth data; 2. The relative coordinates of each identified object and the robot. Finally, based on the robot's current global coordinates, the point cloud data and the relative coordinates of the object and the robot are transformed into the global coordinate system and projected onto a two-dimensional plane, thus completing the construction and updating of the global semantic map.

[0031] In a specific application scenario, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for generating / updating a global semantic map in one embodiment.

[0032] The robot first acquires its current pose information in the global coordinate system in real time through its onboard odometry module, including its position coordinates and heading angle. This pose information will serve as a reference for subsequent environmental perception, map building, and path planning. Simultaneously, the robot acquires visual image data (RGB) and depth image data (Depth) through its onboard RGB-D camera. For the acquired RGB visual image data, this method uses the pre-trained open-vocabulary object detection and semantic segmentation model YOLOE for processing. YOLOE infers from the RGB images, outputting the bounding boxes of all identifiable objects within the field of view and their corresponding category labels. It further performs pixel-by-pixel semantic annotation on the image, generating a semantic segmentation mask with the same resolution as the RGB image, where each pixel is assigned a corresponding object category label.

[0033] Specifically, the robot first uses an odometry system to collect its pose information in real time in the global coordinate system. Simultaneously, an RGB-D camera collects visual image data (RGB) and depth image data (Depth). Data time alignment is achieved based on timestamp matching, and spatial alignment is achieved by combining camera intrinsic parameter calibration and extrinsic parameter transformation, ensuring a one-to-one correspondence between visual, depth, and pose data. The aligned visual image is input into the pre-trained object detection model YOLOE, which outputs the category prediction results of objects within the field of view and pixel-level semantic masks. Detection results with confidence scores below a preset threshold are filtered out to obtain accurate pixel semantic labels. Then, the semantic labels are combined with the depth data of the corresponding pixels, and the pixel coordinates are sequentially converted into three-dimensional coordinates in the camera coordinate system and the robot base coordinate system through camera intrinsic and extrinsic parameters. Each three-dimensional point is assigned a corresponding semantic label, generating a semantically meaningful three-dimensional voxel representation of the current frame. Based on the robot's current global pose, the semantic point cloud in the base coordinate system is transformed to the global coordinate system. Then, interfering point clouds with heights exceeding the robot's interaction range are filtered out. The remaining 3D point cloud is projected onto a 2D plane and mapped onto an occupancy grid map of a preset resolution. Grids within the perception range are marked as explored, while those outside the perception range remain unexplored. A voting method is used to determine the final semantic category label for multiple semantic points within the same grid. Simultaneously, the occupancy probability of the grid is updated based on a Bayesian update rule to distinguish between object-occupied and empty areas. Finally, the updated grid information of the current frame is fused with the historical global semantic map to complete the incremental construction and update of the global semantic map, resulting in a global semantic map that simultaneously includes semantic category, 3D spatial location, and exploration status.

[0034] Step S12: Extract a local semantic map from the global semantic map based on the robot's current position.

[0035] Step S12 is executed if the target object does not exist within the currently explored environment of the global semantic map. Here, the target object refers to an object with a clear semantic identifier that the robot is designated to find or locate. For example, in a home service scenario, the target object could be a "remote control," a "water cup," or a "medicine box"; in an industrial or rescue scenario, it might be a "malfunctioning device" or an "identifier on a trapped person."

[0036] Understandably, the global semantic map records the semantic information and locations of all objects within the explored area. The robot first queries this map to confirm whether the target object (such as a "remote controller") has been discovered. If it does not exist, it means the target object is located in an unexplored area. At this point, the robot should not remain in place or repeat the exploration of the area, but needs to move towards the unexplored area. Therefore, to reach the unexplored area, the robot needs to navigate to the unknown boundary region. However, directly analyzing the entire global map (which may be very large) is computationally intensive, and information about objects far from the current location provides limited help for the next action. Therefore, this embodiment uses a method of extracting a local semantic map.

[0037] Among them, the local semantic map refers to a square area of ​​appropriate size that is cut out from the front of the robot's position in the global semantic map as a local semantic map. The specific size should be as small as possible without being smaller than the effective perception range of the sensor.

[0038] Specifically, a local map refers to a region directly in front of the robot's current field of vision, based on the global semantic map. Its selection and construction process is as follows: First, based on the robot's current position and heading angle obtained through odometry, the spatial range directly in front of the robot's current field of vision is determined. Then, within the global semantic map, using the robot's current position coordinates in the global coordinate system as a reference point, a square region of appropriate size is selected directly in front as the local map. The side length of the square region can be preset according to the actual exploration scenario and accuracy requirements, with 4m being the preferred choice. This ensures that the region size covers the core area of ​​the current field of vision while avoiding reduced decision-making efficiency due to an excessively large local region and the loss of semantic information due to an excessively small local region. This local map maintains coordinate consistency with the global semantic map and can reflect the explored status and environmental semantic information of the local area in real time.

[0039] Step S13: Identify target frontier points with exploratory value in the local semantic map.

[0040] In this context, the leading edge is the boundary between the explored and unexplored areas. The "explored area" includes all the grids that the robot has already detected, while the "unexplored area" refers to the area not yet covered by the sensors. The leading edge with exploration value is the one that is most helpful in reaching the target object.

[0041] Optionally, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating step S13 in one embodiment. Step S13 may include: Step S131: Detect boundary points in the local semantic map.

[0042] Step S131 is used to find the geometric boundary locations between all explored and unexplored regions in the local semantic map. Step S131 will ultimately output a set of boundary points in the local semantic map, representing the boundary contours between explored and unexplored regions.

[0043] Optionally, in one embodiment, step S131 may include: traversing all grids of the local semantic map; if the target grid of the local semantic map is in an explored state and there are unexplored grids adjacent to the target grid, then the target grid is determined to be a boundary point of the local semantic map.

[0044] Specifically, each grid (i.e., the smallest unit of the map) in the local semantic map is traversed. For the current grid g, two conditions are checked: g is in the state of being explored; and at least one of g's neighboring grids (usually four-neighbor or eight-neighbor) is in the state of being unexplored. If both conditions are met, then grid g is marked as a boundary point.

[0045] Step S132: If there are boundary points in the local semantic map, then cluster the boundary points in the local semantic map to obtain local frontier points.

[0046] Step S132 is used to merge spatially adjacent boundary points into regions and use a representative location to concisely describe the region so as to evaluate the exploration value in the future.

[0047] Optionally, in one embodiment, step S132 may include: performing density clustering on the boundary points of the local semantic map to obtain at least one cluster; determining the geometric centroid of the boundary points of each cluster (taking the average of the horizontal and vertical coordinates respectively) as the local front point of the corresponding cluster.

[0048] Specifically, a density-based clustering algorithm (such as DBSCAN) is employed, with a preset cluster radius and minimum number of cluster points. All boundary points are clustered, and those points whose distance is less than the cluster radius and whose number reaches the minimum cluster point count are grouped into a single cluster. Each cluster corresponds to a continuous boundary between an explored and unexplored region on the local map. Finally, the geometric centroid of all boundary points within this cluster on the map plane is used as the unique frontier point corresponding to that cluster. Discrete boundary points that do not reach the minimum cluster point count are deemed invalid and removed to avoid interfering with subsequent exploration value assessments. Through clustering, several effective local frontier points are ultimately obtained on the local map, each corresponding to a potential exploration direction within the local area.

[0049] Step S133: Identify target frontiers with exploratory value among the local frontiers.

[0050] Among them, the frontier points with exploratory value are the frontier points that are most helpful in reaching the target object.

[0051] In an optional embodiment, the frontier with the most exploratory value is selected through a multi-weighted evaluation of "geometric evaluation + semantic evaluation + path cost evaluation".

[0052] like Figure 4 As shown, Figure 4 This is a flowchart illustrating step S133 in one embodiment. Step S133 may include: Step S1331: Perform at least one of geometric evaluation, semantic evaluation, or path cost evaluation on the local frontier point or the global frontier point to obtain the geometric evaluation result, semantic evaluation result, or path cost evaluation result, respectively.

[0053] Among them, the geometric evaluation result is determined based on the size of the boundary where the leading edge point is located, the semantic evaluation result is determined based on the semantic correlation between the objects near the leading edge point and the target object, and the path cost evaluation result is determined based on the path length from the robot's current position to the leading edge point.

[0054] Specifically, geometric evaluation measures the spatial scale of the boundary region corresponding to a local front point, thereby estimating the potential benefits of exploring the unknown environment. Geometric evaluation is determined by calculating the size or area of ​​the boundary where the front point is located. A larger size or area indicates a larger unknown region at the front point's location, meaning the robot is more likely to observe a larger area of ​​the unknown environment after venturing into that region, leading to greater exploration benefits.

[0055] In one embodiment, a circular evaluation neighborhood of the leading edge point is delineated in the global semantic map with a preset neighborhood radius R. The number of grid cells marked as unexplored within the neighborhood is counted. Based on the resolution of the grid map, the actual area of ​​the unexplored region within the neighborhood of the leading edge point is calculated. Max-min normalization is then performed on the unexplored region areas of all leading edges to be evaluated to obtain the final normalized score. .

[0056] in, This represents the maximum area of ​​the unexplored region corresponding to all current frontier points to be evaluated. To find the minimum value, after normalization The range of values ​​is .

[0057] Specifically, semantic evaluation measures the spatial correlation between a front point and known objects in the currently explored environment, and infers the semantic relevance between objects near the front point and the target object based on this correlation, thereby determining the potential value of the front point in finding the target object. Semantic evaluation first needs to accurately determine the spatial correlation between the semantic information of the front point and all known objects in the current environment. The core meaning of this correlation is to evaluate "whether the robot can observe the object when it arrives at the front point".

[0058] In one embodiment, among the explored objects, candidate objects that satisfy preset spatial correlation conditions with the leading edge point are identified. These spatial correlation conditions include: no occlusion between the leading edge point and the explored objects, and the distance between the leading edge point and the explored objects is less than a set distance threshold. The semantic correlation between the candidate objects and the target object is determined to obtain a semantic evaluation result. Based on this, on the current map, two checks are performed on each pair of "leading edge point-object": 1. Whether the two are occluded by objects higher than the robot's field of view; 2. Whether the straight-line distance between the two is less than a given threshold. The straight-line distance threshold can be determined according to specific needs. If the task environment is relatively open, a larger value can be used; otherwise, a smaller value should be used to avoid misjudgment. If both conditions are met simultaneously, it is determined that the leading edge point and the object have a spatial correlation. Next, after evaluating the correlation between semantic elements and each front point, the semantic similarity or correlation between the semantic elements associated with the front point and the target object to be searched is obtained based on the pre-trained language model. The specific method depends on the language model used, but the core idea is to obtain the logical connection between the objects around each front point and the target to be searched. This logical connection is based on the common sense knowledge contained in the language model and conforms to the rules of human daily behavior as a whole.

[0059] In one embodiment, firstly, semantic elements within a radius R centered on the leading edge point are obtained from the global semantic map. During the acquisition of semantic elements around each leading edge point, to avoid mismatches where individual semantic elements meet the radius requirement but are not in the same region as the leading edge point, connection detection is performed on semantic elements that meet the region requirement and their connection to the leading edge point. If a wall is detected between the leading edge point and the semantic element, it is determined that the semantic element is not in the same region as the leading edge point, and the semantic element is discarded.

[0060] Furthermore, the cosine similarity between each leading edge point and the target object is calculated using the pre-trained language model BERT, ultimately obtaining the semantic inference score of the i-th leading edge point at time step t. : .

[0061] in, Representative in the first The number of semantic elements that meet the requirements within a radius R centered at a front point. If there are no semantic elements within this region, the semantic reasoning score of that front point is a constant C. Representing the The reasoning score of each semantic element. The distance coefficient between the frontier point and the semantic element: .

[0062] Where L represents the distance between the semantic element and the frontier point. It is a preset benchmark score. When the inference score is lower than the benchmark, the closer the semantic element is to the frontier, the greater the negative impact it has on the semantic value assessment of the frontier.

[0063] Specifically, path cost assessment quantifies the resource costs required for a robot to move from its current position to a forward point, primarily including time and energy consumption. This guides the robot to balance efficiency and economy during exploration. Path cost assessment calculates the path length from the robot's current position to each forward point using a path planning algorithm on a known environmental map. This path length affects the time and energy costs required for the robot to reach that location, a crucial factor to consider during robot navigation. Finally, the three indicators are normalized and then weighted to obtain a quantitative score for the exploration value of each forward point. The highest score is considered the optimal forward point in the current state, and moving to that point is most beneficial for the robot to achieve its mission objective.

[0064] In one embodiment, the robot is positioned at time steps Starting from the current global pose, with the first... Using a leading edge as the target endpoint, within the traversable area of ​​the global semantic raster map, the A* path planning algorithm is used to solve for the collision-free optimal path. The total length of this optimal path is used as the original path cost for that leading edge. Since path cost is negatively correlated with the value of frontier exploration, a reverse max-min normalization process is used to obtain the final normalized score: .

[0065] in, This represents the maximum value of the original path cost corresponding to all current frontier points to be evaluated. To find the minimum value, after normalization The range of values ​​is A higher score indicates a lower path cost for the robot to reach the frontier point, and a higher feasibility of exploration.

[0066] Step S1332: Weight the geometric evaluation results, semantic evaluation results, or path cost evaluation results to determine the final evaluation results of the local frontier point or the global frontier point.

[0067] Understandably, combining the above formulas (1), (2), and (3), for , and The final exploration value score of this frontier point is obtained by weighting the results: .

[0068] Understandably, for identifying target frontier points with exploratory value in a local semantic map, the weighting coefficients will be... A value of 0 is used to ignore the impact of path cost on the evaluation results, and... A larger value is chosen to strengthen the guiding role of semantic information in the robot's exploratory behavior.

[0069] Step S1333: Based on the final evaluation results of the local or global frontier points, determine the target frontier points with exploratory value.

[0070] Finally, the exploration value scores of all frontier points are ranked, and the frontier point with the highest score is selected as the target frontier point with exploration value. The boundary area corresponding to this frontier point is the exploration direction in which the robot is most likely to find the target object in its current local area.

[0071] Step S14: Perform path planning for the robot to reach the target front point in the global semantic map.

[0072] In step S14, the global coordinates of the robot's current position are first obtained. These coordinates are acquired and calibrated in real time by the robot's odometry. Simultaneously, the global coordinates corresponding to the target leading edge point from step S13 are obtained as the path endpoint. The reachability of the endpoint area is checked using the global semantic map to ensure that there are no insurmountable obstacles in the boundary area corresponding to the target leading edge point. At the same time, the grid information of the explored areas in the map is extracted to clarify the feasible areas for path planning, avoiding the planning of invalid paths that traverse unexplored areas or obstacle areas. Then, the A* path algorithm is started for path planning, using the set map grid resolution as the minimum unit of path planning and the Manhattan distance as the heuristic function to quickly estimate the predicted cost from the robot's current position to the optimal leading edge point. The cost function is set as follows: .

[0073] in, The actual cost of the robot's movement from its current position to the current search grid. To estimate the cost for the heuristic function calculation, the algorithm initializes open and closed lists, continuously searches for the grid with the minimum cost function value, and traverses its effective neighboring grids. It removes obstacles, unexplored areas, and already searched grids, gradually approaching the endpoint until a preliminary grid path is formed through reverse tracing. Finally, the initially planned path is optimized by removing redundant turning grids to shorten the path length, smoothing sharp turns to prevent abrupt changes in robot movement and improve stability, ensuring that all grids on the path are free of obstacles and occlusions, and that the distance between the path and obstacles is not less than a preset safety threshold. After optimization, the final optimal path is output.

[0074] In some embodiments, after path planning is completed, the robot's motion control module receives the path information and parses and decomposes the motion commands based on the planned path. The module generates continuous control commands that conform to the robot's motion characteristics based on the starting point, waypoints, and ending point information of the path. The robot performs continuous and stable autonomous movement along the planned path according to the parsed commands. During the movement, the robot collects its own position information in real time through an odometer and continuously calibrates its current pose. The robot moves smoothly without collision risk and eventually reaches the location of the target object or the optimal leading edge area. During this period, the robot can also continuously update the global semantic map based on the latest acquired data. Furthermore, after the robot completes its movement, the system calculates the spatial straight-line distance between the current position and the target object's coordinates to determine whether the robot has entered a preset effective range. If the robot is determined to be within the preset proximity area of ​​the target object, the unknown environment target exploration task is confirmed to be successfully completed, and the overall exploration process is terminated. If the robot has not entered the preset proximity area of ​​the target object, it indicates that the target object is still in an unexplored space. At this time, the robot will return to step S11 and continue to loop through the exploration process until the task is completed.

[0075] The above approach, through the decision logic of "if there is no target globally, then extract a local map," narrows the exploration focus from the vast but redundant global environment to the semantically rich region near the robot. This significantly improves the spatiotemporal efficiency and semantic guidance of target-driven exploration while reducing computational burden, enabling the robot to quickly and intelligently enter unexplored areas most likely containing the target object. Specifically: 1. This approach addresses the lack of high-level semantic information in environmental representation, achieving accurate and open semantic mapping. By integrating an open vocabulary detection model with 3D point cloud data, this solution constructs a point-by-point associative semantic map. This approach does not rely on a predefined closed category library and can identify and label new object categories not present in the training data. Compared to traditional maps that only contain geometric information such as occupied and idle, the map generated by this method may carry specific object semantic labels for each point, theoretically providing robots with an environmental knowledge base that can be directly used for task reasoning, elevating environmental cognition from the geometric level to the semantic level.

[0076] 2. This approach addresses the issues of disconnect between the exploration process and task objectives, as well as the limited dimensions of frontier evaluation, enabling task-oriented intelligent exploration decision-making. The multi-weighted frontier evaluation method designed in this scheme innovatively introduces a pre-trained language model into the traditional geometric evaluation based on information gain (size of the unknown region to be explored) and path cost (distance to reach). This model can quantitatively evaluate the semantic relevance between the frontier region and the task objective described in natural language. By fusing geometric and semantic weights, the robot can prioritize exploring regions highly relevant to the task. Theoretical analysis shows that this method effectively reduces blind exploration in irrelevant regions, transforming the randomness of exploration into goal-orientedness, and significantly improving the efficiency of target localization in exploration tasks.

[0077] 3. This solution addresses the problems of global exploration easily getting trapped in local optima and the existence of numerous backtracking and redundancy in the path, achieving efficient and coherent layered exploration. The local-global layered exploration strategy designed in this scheme operates by combining the selection of local optimal frontier points with a global relocalization mechanism. The local strategy ensures that the robot moves towards the direction with the highest value (high information gain, low path cost, and high semantic relevance) within its current field of view; when there are no valuable targets in a local area, the global strategy guides the robot to jump to other unexplored high-value boundary areas on the map. This strategy, in principle, avoids the robot lingering in a small area due to local optima and also prevents ineffective long-distance runs caused by a single global point being too far away. It systematically reduces the overlap and backtracking in the overall exploration path, improving path efficiency.

[0078] 4. This solution addresses the issues of data-driven methods relying on extensive training, weak generalization ability, and high deployment costs. The core of this solution leverages the inherent zero-shot recognition and inference capabilities of large language models / visual-language models. The entire system requires no additional model training, fine-tuning, or collection of labeled datasets for specific indoor environments or object sets. From an engineering perspective, this significantly reduces the cost and barrier to entry for technology deployment and maintenance, enhancing the practicality and scalability of the method across various application scenarios.

[0079] See Figure 5 , Figure 5This is a flowchart illustrating a second embodiment of the robot path planning method provided in this application. The path planning method is as follows: Step S51: Obtain the global semantic map.

[0080] The global semantic map is used to represent the semantic information of each object in the explored environment, as well as the positional relationship between each object and the robot.

[0081] Step S52: Extract a local semantic map from the global semantic map based on the robot's current position.

[0082] Step S53: Detect boundary points in the local semantic map.

[0083] Step S54: If there are no boundary points in the local semantic map, then detect the boundary points in the global semantic map.

[0084] Optionally, step S54 may specifically include: traversing all grids of the global semantic map; if the target grid of the global semantic map is in an explored state and there are unexplored adjacent grids of the target grid, then the target grid is determined to be a boundary point of the global semantic map.

[0085] Step S55: If there are boundary points in the global semantic map, then cluster the boundary points in the global semantic map to obtain the global frontier points.

[0086] Optionally, step S55 may specifically include: performing density clustering on the boundary points of the global semantic map to obtain at least one cluster; and determining the geometric centroid of the boundary points of each cluster as the global front point of the corresponding cluster.

[0087] Step S56: Identify target frontiers with exploratory value among the global frontiers.

[0088] Optionally, step S56 may specifically include: performing at least one of geometric evaluation, semantic evaluation, or path cost evaluation on the global front point to obtain geometric evaluation results, semantic evaluation results, or path cost evaluation results, respectively; wherein, the geometric evaluation result is determined based on the size of the boundary where the front point is located, the semantic evaluation result is determined based on the semantic correlation between the objects adjacent to the front point and the target object, and the path cost evaluation result is determined based on the path length from the robot's current position to the front point; weighting the geometric evaluation result, semantic evaluation result, or path cost evaluation result to determine the final evaluation result of the global front point; and determining the target front point with exploration value based on the final evaluation result of the global front point.

[0089] Step S57: Perform path planning for the robot to reach the target front point in the global semantic map.

[0090] Understandably, compared to the first embodiment, this embodiment detects boundary points in the global semantic map when no boundary points exist in the local semantic map. The boundary point detection method, the boundary point clustering method, and the method for selecting target frontier points with exploration value can all refer to the methods used for the local semantic map in the first embodiment, and will not be repeated here. Furthermore, regarding the final exploration value score of frontier points in the global semantic map: .

[0091] Understandably, for identifying target frontier points with exploratory value in a global semantic map, the weighting coefficients... If the value is not 0, an additional global path cost is introduced for the robot to reach the frontier point, thereby enabling a more reasonable decision that better meets the needs of global exploration.

[0092] Furthermore, if no valid boundary points are found in the global semantic map, and no semantic label for the target object is matched, it indicates that the robot has traversed the entire explorable area. At this point, the entire unknown environment has been fully explored, and there are no more unknown areas available for further exploration. In the absence of valid boundaries and target objects, the exploration task is deemed unsuccessful in finding the target, and the task ends.

[0093] The above method, through the mechanism of "backtracking to global detection when there are no local boundary points," completely solves the problem of the robot getting stuck in exploration due to limited local field of view, ensuring that the algorithm can always output feasible target frontier points in any complex indoor environment; at the same time, it maintains the efficient strategy of "prioritizing local and guaranteeing global detection," taking into account both real-time performance and completeness, and significantly improving the robustness and task completion rate of the autonomous exploration system. Specifically: 1. Avoid exploration deadlock and ensure task continuity. In real indoor environments, robots may encounter situations where no boundary points exist in the local semantic map. For example, the robot may be in a completely enclosed room; the robot may be completely surrounded by an explored area, with all adjacent grid cells already explored; or the robot may be at the end of a dead end. If the algorithm relies only on local leading edges, it will be unable to find any candidate exploration targets, causing the robot to stagnate or enter an infinite waiting state. This embodiment adds a judgment: when there are no boundary points in the local semantic map, it automatically switches to the global semantic map to detect boundary points. The global map covers the entire explored environment, which inevitably contains pathways to unexplored areas (such as the entrance to a room that has not yet been entered). The robot can thus escape local predicaments, regain feasible forward targets, and ensure the continuity of the exploration task.

[0094] 2. Expand the exploration horizon and avoid local optima traps. Local semantic maps are centered on the robot and have a limited scope. When the robot is located in a fully explored sub-region (such as the end of a thoroughly searched corridor), there may indeed be no boundary points within the local map, but there are unexplored branches slightly further away (beyond the local capture radius). The global boundary point detection in this embodiment can discover these unexplored area entrances that are far from the robot but do exist. The robot plans a long path to reach this global frontier point, thereby escaping local optima (such as idling in a dead end) and achieving a higher level of spatial coverage.

[0095] 3. Layered exploration strategy, balancing efficiency and completeness. Prioritizing local frontier points: When there are boundary points within the local map, the robot prioritizes exploring nearby areas, maintaining short movement paths and fast response times, adhering to the principle of prioritizing exploration efficiency. Backtracking to global frontier points: Global boundary points are only used when there are no local boundary points, avoiding unnecessary long-distance path planning; simultaneously, this backtracking mechanism ensures that the algorithm can provide a feasible solution under any circumstances, meeting the completeness requirements of the exploration. This hierarchical design of "local first, global later" utilizes the speed of local computation while providing a fallback through global detection, making the algorithm robust even in complex and large-scale environments.

[0096] See Figure 6 , Figure 6 This is a flowchart illustrating a third embodiment of the robot path planning method provided in this application. The path planning method is as follows: Step S61: Obtain the global semantic map.

[0097] The global semantic map is used to represent the semantic information of each object in the explored environment, as well as the positional relationship between each object and the robot.

[0098] Step S62: If it is determined that there is a target object in the currently explored environment based on the global semantic map, then the path planning for the robot to reach the target object is performed in the global semantic map.

[0099] The path planning for the robot to reach the target object in the global semantic map is similar to the method of "path planning for the robot to reach the target front point in the global semantic map" in the above embodiment, and will not be repeated here.

[0100] The above approach prioritizes determining whether a target object exists in the global semantic map. If the target has appeared, the exploration is immediately terminated and the system switches to direct path planning. This avoids meaningless calculations and movements, significantly shortens task completion time, saves system resources, and improves the intuitiveness and interpretability of the decision-making logic.

[0101] See Figure 7 , Figure 7This is a flowchart illustrating the fourth embodiment of the robot path planning method provided in this application. The path planning method is as follows: Step S71: Obtain the global semantic map.

[0102] Step S72: Determine whether there is a target object in the currently explored environment based on the global semantic map.

[0103] If the result of step S72 is negative, step S73 is executed; if the result of step S72 is positive, step S78 is executed.

[0104] Step S73: Extract a local semantic map from the global semantic map based on the robot's current position.

[0105] Step S74: Determine whether boundary points are detected in the local semantic map.

[0106] If the judgment result of step S74 is yes, then step S75 is executed; if the judgment result of step S74 is no, then step S76 is executed.

[0107] Step S75: Cluster the boundary points in the local semantic map to obtain local frontier points, and identify target frontier points with exploration value among the local frontier points.

[0108] Step S76: Determine whether boundary points are detected in the global semantic map.

[0109] If the result of step S76 is yes, then step S77 is executed; if the result of step S76 is no, then the task ends.

[0110] Step S77: Cluster the boundary points in the global semantic map to obtain global frontier points, and identify target frontier points with exploration value among the global frontier points.

[0111] Step S78: Perform path planning for the robot to reach the target object or the target front point in the global semantic map.

[0112] Understandably, this embodiment combines the first, second, and third embodiments described above. In this way, a logically complete and hierarchically distinct decision state machine is constructed. Through a four-level judgment process of "target priority → local frontier point → global frontier point → task termination," the robot can make the optimal response in any environmental state (target has appeared / has not appeared, local entry / exit is present, global entry / exit is present). This not only ensures exploration efficiency and navigation real-time performance but also provides a robust fault tolerance and termination mechanism, comprehensively improving the intelligence level and task completion rate of autonomous mobile robots in target-driven tasks.

[0113] See Figure 8 , Figure 8 This is a schematic diagram of a robot embodiment provided in this application. The robot 800 includes a processor 81 and a memory 82 connected to the processor 81.

[0114] The memory 82 is used to store computer programs, and the processor 81 is used to execute the computer programs to implement the following path planning method: acquiring a global semantic map; wherein the global semantic map is used to represent the semantic information of each object in the explored environment, as well as the positional relationship between each object and the robot; if it is determined based on the global semantic map that there is no target object in the currently explored environment, then a local semantic map is extracted from the global semantic map according to the robot's current position; a target frontier point with exploration value is determined in the local semantic map; and path planning for the robot to reach the target frontier point is performed in the global semantic map.

[0115] Specifically, processor 81 controls itself and memory 82 to implement the steps of any of the above-described reversing image path planning method embodiments. Processor 81 can also be referred to as a CPU (Central Processing Unit). Processor 81 may be an integrated circuit chip with signal processing capabilities. Processor 81 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 81 can be implemented using integrated circuit chips.

[0116] In other embodiments, the memory 82 of the processor 81 described above can also be located away from the robot and function as a separate device, such as a computer device. This computer device communicates with the robot via, for example, wireless communication. The robot sends data acquired by sensors to the computer device, which performs path planning according to the method described above and then sends motion commands to the robot so that the robot moves according to the commands.

[0117] Please see Figure 9 , Figure 9This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 900 stores a computer program 91 that can be executed by a processor. The computer program 91 is used to implement the steps of any of the above-described path planning method embodiments.

[0118] In some embodiments, this application also provides a computer program product, which, when executed by a processor, is used to implement the steps of the above-described path planning method embodiments.

[0119] Understandably, a computer program product can be a computer program product contained on a tangible computer-readable medium, which includes program code for performing the above-described method for displaying a reversing image. In some embodiments, the computer program product can be downloaded and installed from a network, and can also be copied, transferred, and installed between different computer hardware. Its wireless transmission method can include the Internet, Bluetooth, WIFI, etc., and its wired transmission method can include USB, Lightning, Type-C, etc.

[0120] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0121] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A path planning method for a robot, characterized in that, include: Obtain a global semantic map; wherein the global semantic map is used to represent the semantic information of each object in the explored environment, as well as the positional relationship between each object and the robot; A local semantic map is extracted from the global semantic map based on the robot's current position; Identify target frontier points with exploratory value in the local semantic map; The path planning for the robot to reach the target front point is performed in the global semantic map.

2. The path planning method for a robot according to claim 1, characterized in that, The step of identifying target frontier points with exploratory value in the local semantic map includes: Detect boundary points in the local semantic map; If boundary points exist in the local semantic map, then the boundary points in the local semantic map are clustered to obtain local frontier points; Among the local frontier points, target frontier points with exploratory value are identified.

3. The path planning method for a robot according to claim 2, characterized in that, After detecting the boundary points in the local semantic map, the method further includes: If no boundary points exist in the local semantic map, then boundary points in the global semantic map are detected; If boundary points exist in the global semantic map, then the boundary points in the global semantic map are clustered to obtain global frontier points; Among the global frontier points, target frontier points with exploratory value are identified.

4. The path planning method for a robot according to claim 3, characterized in that, The detection of boundary points in the local semantic map, or the detection of boundary points in the global semantic map, includes: Traverse all grids of the local semantic map or the global semantic map; If the target grid of the local semantic map or the global semantic map is in an explored state and there are unexplored adjacent grids of the target grid, then the target grid is determined to be a boundary point of the local semantic map or the global semantic map.

5. The path planning method for a robot according to claim 3, characterized in that, The step of clustering boundary points in the local semantic map to obtain local frontier points, or the step of clustering boundary points in the global semantic map to obtain global frontier points, includes: Density clustering is performed on the boundary points of the local semantic map or the global semantic map to obtain at least one cluster. Determine the geometric centroid of the boundary point of each cluster as the local or global front point of the corresponding cluster.

6. The path planning method for a robot according to claim 3, characterized in that, The step of determining a target frontier point with exploratory value among the local frontier points, or the step of determining a target frontier point with exploratory value among the global frontier points, includes: At least one of geometric evaluation, semantic evaluation, or path cost evaluation is performed on the local front point or the global front point to obtain geometric evaluation results, semantic evaluation results, or path cost evaluation results, respectively; wherein, the geometric evaluation result is determined based on the size of the boundary where the front point is located, the semantic evaluation result is determined based on the semantic correlation between the objects adjacent to the front point and the target object, and the path cost evaluation result is determined based on the path length from the robot's current position to the front point; The geometric evaluation results, the semantic evaluation results, or the path cost evaluation results are weighted to determine the final evaluation result of the local front point or the global front point. Based on the final evaluation results of the local frontier points or the global frontier points, target frontier points with exploratory value are determined.

7. The path planning method for a robot according to claim 6, characterized in that, The semantic evaluation includes: Among the explored objects, candidate objects that satisfy preset spatial correlation conditions with the leading edge point are identified; wherein, the spatial correlation conditions include: there is no occlusion between the leading edge point and the explored objects, and the distance between the leading edge point and the explored objects is less than a set distance threshold. The semantic relationship between the candidate object and the target object is determined to obtain the semantic evaluation result.

8. The path planning method for a robot according to claim 1, characterized in that, After obtaining the global semantic map, the process also includes: Based on the global semantic map, determine whether there is a target object in the currently explored environment; If it exists, then the path planning for the robot to reach the target object is performed in the global semantic map; If it does not exist, then perform the step of extracting a local semantic map from the global semantic map based on the robot's current position.

9. A robot, characterized in that, The robot includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to implement the path planning method for the robot as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the path planning method for the robot as described in any one of claims 1-8.

Images