Robot article navigation method and system in unknown environment
By combining a search strategy based on large language models and multimodal large models with an object detection model, the problems of insufficient reasoning ability and detection accuracy in robot object navigation in unknown environments are solved, achieving efficient and accurate object localization and path planning.
Patent Information
- Application Number
- CN202511647114.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-11
- Publication Date
- 2026-02-10
AI Technical Summary
Existing technologies for robot object navigation in unknown environments suffer from problems such as insufficient reasoning ability of large language models, limited robot exploration capabilities, and high false positive and false negative rates of zero-shot target detection models.
By combining large language models and multimodal large models, and constructing occupancy maps and search strategies, the robot performs fine, coarse, and comprehensive searches using object detection models. It also utilizes a Grounding-Dino-Base zero-shot object detection model for cross-validation and plans robot paths to find target items.
It enables efficient and accurate object navigation in unknown environments, reduces the false detection and false negative rates of target detection, and improves the efficiency and accuracy of robot exploration in unknown environments.
Smart Images
Figure CN121498693A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot autonomous navigation technology, specifically to a robot object navigation method and system in unknown environments. Background Technology
[0002] Intelligent robots are increasingly used in daily life. Especially in home environments or commercial buildings, robots capable of autonomous navigation and object recognition can significantly improve quality of life and work efficiency. However, most current navigation systems rely on pre-trained datasets, meaning that to enable robots to recognize new objects or adapt to new environments, large amounts of data need to be collected and models retrained. This is not only time-consuming and labor-intensive but also limits the robot's flexibility and adaptability. Zero-shot learning, as an emerging learning method, aims to solve this problem, enabling robots to accurately perform tasks, such as finding specified objects, in unseen environments.
[0003] Currently, most zero-shot navigation methods utilize the commonsense reasoning capabilities of large language models (LLMs) to generate high-level navigation strategies, while relying on pre-trained object detection models to identify surrounding objects. However, the effectiveness of LLM-based reasoning weakens when the target object lacks strong contextual or spatial associations with a specific environment or object. For example, when the target is a television, the model can effectively infer that it is likely to be in the living room, possibly adjacent to related furniture such as a sofa or table, indicating strong semantic and spatial relationships. In contrast, objects such as chairs or potted plants are more prevalent in various types of rooms and exhibit weaker environmental dependencies, thus limiting the utility of high-level semantic reasoning. When exploring unknown environments, robots need a certain level of search capability; they cannot simply repeat explorations within their surroundings but must also possess the ability to find pathways to other scenes.
[0004] Furthermore, during the detection process, relying solely on a pre-trained target detection model is insufficient to achieve satisfactory performance. Experiments have shown that detection failure is the main cause of navigation errors. These failures manifest as missed detections or misidentifications, severely hindering the overall success rate. In this case, zero-shot target detection models have two key limitations: (1) Due to the difficulty in adaptively determining the optimal confidence threshold, the system is prone to numerous false detections (misclassifying non-targets as targets) and missed detections (failing to identify real targets) during target recognition. (2) When a target is missing, the model often generates false detections, attempting to produce output in negative scenarios. For example, when searching for a "bed" in a living room with a sofa, the detector may incorrectly classify the sofa as a bed.
[0005] In summary, existing technologies have the following problems: large language models struggle to exert their reasoning ability when faced with a given item and a specific environment or when the item lacks semantic and spatial association; the robot's limited exploration capabilities restrict its ability to navigate objects; and zero-shot object detection models still suffer from false detections and false negatives. Summary of the Invention
[0006] In view of this, the present invention provides a robot object navigation method and system in an unknown environment, which can still perform high-efficiency exploration when the reasoning ability of large language models is difficult to exert, and can effectively reduce the false detection and false detection rates of target detection models.
[0007] Therefore, the present invention provides the following technical solution: On one hand, the present invention provides a method for robot object navigation in unknown environments, comprising: S1. Acquire the RGB image and depth image of the current scene; S2. Construct an occupancy map based on the depth image; S3. Utilize large language models and multimodal large models to determine search strategies based on the type of searched items; S4. Based on the search strategy and the map, use the target detection model to detect and locate target items in the RGB image; S5. Based on the target detection and localization results and the robot's position, plan the robot's path.
[0008] Furthermore, the search strategy includes: fine-grained search, coarse-grained search, and comprehensive search; the determination of the search strategy based on the type of search item using large language models and multimodal large models includes: Large language models are used to infer whether a target item has a strong semantic association with a specific room type. Using a multimodal large model, the room type of the robot's current environment can be inferred; If the target item has a strong semantic relationship, determine whether the current room is the room type where the target item is most likely to appear; if so, perform a fine search on the current room; if not, perform a coarse search and move to other potential functional areas to continue the search. If the target item does not have a strong semantic relationship, a systematic and comprehensive search of the surrounding environment will be conducted, ensuring that the same area is not searched repeatedly.
[0009] Further, the step of detecting and locating target objects in the RGB image using a target detection model based on the search strategy and the map includes: A multimodal large model is used to perform preliminary detection of surrounding objects; If the target item is not detected, continue exploring and update the map; If the target item is detected, the target detection model is invoked for fine-grained detection; If the object detection model does not detect the target item, continue exploring and updating the map; If the target detection model detects a target item, the detection result is fed back to the multimodal large model for cross-validation; If verification fails, continue exploring and updating the map; if verification succeeds, calculate the actual 3D coordinates based on the image area where the target item is located.
[0010] Furthermore, the search strategies are all based on cutting-edge exploration methods and the explored frontiers are filtered; the map includes: obstacle areas, navigable areas, and unknown areas; The cutting-edge exploration methods include: The explorable frontier is obtained based on the map, including: extracting the boundary of the navigable area and expanding the obstacle area; the explorable frontier is the part of the boundary of the navigable area that does not overlap with the obstacle area; the explorable frontier includes several frontier point data. Each front point in the explorable frontier is filtered.
[0011] Furthermore, each front point in the explorable front is filtered, including: definition This is the set of leading edge points obtained from the most recent detection; This refers to a set of previously detected but not yet explored frontier points; This is a collection of previously explored frontier points; This represents the robot's position in a 2D base coordinate system. Defined as a point With point Horizontal distance; This represents the unknown region in the 2D occupancy map, where the value of the unknown region is 0; gather any point in For one of the points The horizontal distance is less than The set is defined as ,in It is a constant; If the following formula is satisfied, point and set Replaced by point d:
[0012] If the frontier point If there are other frontier points within the range, then the coordinates of this point are the average of the coordinates of all points within that range; if this point... If there are no other frontier points within the range, the position remains unchanged.
[0013] Furthermore, filtering each front point in the explorable front also includes: gather The following conditions must be met: the location of the leading edge point is not on the map of the already explored area, or it belongs to an unknown area on the map.
[0014] Furthermore, filtering each front point in the explorable front also includes: The leading edge point within the preset range of the filter robot's foot.
[0015] Furthermore, filtering each front point in the explorable front also includes: The cutting edge that filtering robots have already explored.
[0016] Furthermore, the target detection model is a Grounding-Dino-Based zero-shot target detection model, and the output is the coordinate position of the target object in the image.
[0017] In another aspect, the present invention also provides a robot object navigation system in an unknown environment, comprising: The image acquisition module uses an RGB-D camera to capture RGB and depth images of the current scene; A module for constructing an occupancy map is used to construct an occupancy map based on the depth image. The navigation decision and search module utilizes large language models and multimodal large models to determine search strategies based on the type of searched items; The target detection and localization module, based on the search strategy and the map, uses a target detection model to detect and locate target objects in the RGB image; The path planning module plans the robot's path based on the target detection and localization results and the robot's position.
[0018] The advantages and positive effects of this invention are as follows: This invention integrates the semantic prior of a large model with the precise perception and planning capabilities of traditional robot systems. It enables highly efficient exploration even when the reasoning capabilities of large language models are limited, achieving zero-shot indoor object navigation without the need for target category labeling training. This solves the problems of insufficient generalization ability in open and dynamic environments, reliance on large amounts of labeled data, difficulty in handling unknown object categories, low exploration efficiency due to the lack of correlation between objects and areas in current zero-shot object navigation methods, and false positives and false negatives in zero-shot target detection models. Furthermore, the exploration method of this invention improves upon frontier-based exploration methods by filtering the obtained frontier, thereby preventing the robot from repeatedly exploring the same area and improving the robot's ability to search other rooms. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of robot object navigation in an unknown environment according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the robot's object navigation decision-making process in an unknown environment in an embodiment of the present invention; Figure 3 This is a schematic diagram of the target detection process for robot object navigation in an unknown environment in an embodiment of the present invention; Figure 4 This is a reference diagram of the robot's object navigation process in an unknown environment in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the reasoning process of a large-scale robot object navigation model in an unknown environment, as described in an embodiment of the present invention. Detailed Implementation
[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0023] The task requirement for robot object navigation in unknown environments is that the robot autonomously finds and navigates to a designated object in an unknown indoor environment without human intervention. The robot randomly initializes its pose within the indoor environment and receives the target object category (e.g., "television") specified in natural language. After system startup, the robot collects RGB-D data using its onboard depth camera, constructs a local occupancy map in real time, and continuously updates the environmental representation. To improve exploration efficiency, the system introduces a high-level navigation strategy based on common sense reasoning. Specifically, it first uses a Large Language Model (LLM) combined with the common functional attributes of the target object to infer the functional area where it has the highest probability of appearing in the architectural space. Simultaneously, a Multimodal Large Model (MMLM) is used to identify the room type of the current observation scene, enabling the discrimination of the semantic category of the environment. Based on the results, a search strategy is formulated: if the current room type matches a high-probability area, a fine-grained search mode is entered; otherwise, a coarse search strategy is adopted, quickly traversing the current area and migrating to other potential functional areas to continue the search. If no high-probability areas exist (e.g., "potted plants" or "chairs" have a probability of existing in multiple room types), a systematic and comprehensive search of the surrounding environment is performed, ensuring that the same area is not explored repeatedly. In the object detection phase, the system first uses a multimodal large model to perform preliminary detection of objects within the field of view to determine the presence of the specified object. Upon receiving a positive response, a dedicated object detection model is invoked for refined identification. The detection results are fed back to the multimodal large model for cross-validation to improve the accuracy and robustness of the judgment. Once the target object is successfully located, the system uses the A* algorithm to plan the optimal path on the constructed map and controls the robot to navigate to the vicinity of the target, completing the object localization task. This method integrates the semantic prior of the large model with the precise perception and planning capabilities of traditional robot systems, achieving zero-shot indoor object navigation without the need for target category labeling training.
[0024] like Figure 1As shown, a robot object navigation method in an unknown environment includes: S1. Acquire the RGB image and depth image of the current scene; S2. Construct an occupancy map based on depth images; S3. Utilize large language models and multimodal large models to determine search strategies based on the type of searched items; The search strategies include: fine search, coarse search, and comprehensive search; preferably, the large language model can adopt the QWen-Plus large language model, and the multimodal large model can adopt the QWen-vl-max-latest multimodal large model.
[0025] Before the search, a large language model and a multimodal large model are used to determine the search pattern. The fine-grained search, coarse search, and comprehensive search are all improvements on the frontier-based exploration method. The frontier is obtained through an occupancy map. Specifically, the boundaries of the navigable area in the occupancy map are extracted, and the obstacle area is expanded. The portion of the resulting navigable area boundary that does not overlap with the obstacle area is the explorable frontier. However, since it is calculated on an occupancy map, the frontier consists of many points. Therefore, filtering these points and selecting which frontier points to explore are crucial. To address this issue, a definition is provided. This is the set of leading edge points obtained from the most recent detection; This refers to a set of previously detected but not yet explored frontier points; This is a collection of previously explored frontier points; This represents the robot's position in a 2D base coordinate system. Defined as a point With point Horizontal distance; This represents the unknown region in a 2D occupancy map, where the value of the unknown region is 0. (Set) any point in For one of the points The horizontal distance is less than The set is defined as Where S is a constant. If the following formula is satisfied, point... and set Replaced by point d.
[0026]
[0027] After obtaining a large number of frontier points using a frontier-based exploration method, each frontier point is filtered using the above method. If the frontier points... If there are other frontier points within the range, then the coordinates of this point are the average of the coordinates of all points within that range. If this point... If there are no other frontier points within the range, the position remains unchanged.
[0028] To ensure that the robot does not repeatedly explore the same area, the following formula is defined:
[0029] If the location of the leading edge point is not on the map of the already explored area, or it belongs to an unknown area on the map, then the robot has not yet explored the area where that point is located. To avoid duplicate explorations, the following set should be used. All of these conditions must be met. Furthermore, due to perspective issues, robots often generate leading-edge points near their feet; this area needs to be filtered out, meaning it must meet certain conditions. .
[0030] In addition, the robot's previously explored frontier points are used to further ensure that it does not explore redundant areas, satisfying the following formula: .
[0031] In specific implementation, such as Figure 2 As shown, S3 is executed according to the following steps: S31. Use large language models to infer whether the target item has a strong semantic association with a specific room type; S32. Use a multimodal large model to infer the room type of the robot's current environment; S33. If the target item has a strong semantic relationship, determine whether the current room is the room type where the target item is most likely to appear; if so, perform a fine search on the current room; if not, perform a coarse search and move to other potential functional areas to continue the search. S34. If the target item does not have a strong semantic association, a systematic and comprehensive search of the surrounding environment will be conducted, ensuring that the same area is not searched repeatedly.
[0032] The search strategies are all based on cutting-edge exploration methods; the map includes: obstacle areas, navigable areas, and unknown areas; S4. Based on the search strategy and map, use the object detection model to detect and locate target objects in RGB images; In specific implementation, such as Figure 3 As shown, S4 is executed according to the following steps: S41. Use a multimodal large model to perform preliminary detection of surrounding objects; S42. If the target item is not detected, continue exploring and update the map; S43. If the target item is detected, the target detection model is invoked for fine-grained detection; S44. If the target detection model does not detect the target item, continue exploring and update the map; S45. If the target detection model detects the target item, the detection result is fed back to the multimodal large model for cross-validation. S46. If verification fails, continue exploring and update the map; if verification succeeds, calculate the actual 3D coordinates based on the image area where the target item is located.
[0033] In this embodiment, the object detection model is the Grounding-DINO-Base zero-shot object detection model, which outputs the coordinates of the target object in the image. Grounding-DINO-Base is a zero-shot object detection model based on the Transformer architecture. By combining the cross-modal alignment capabilities of language and vision, it can detect any open-world object without specific category training data. Its core innovation lies in utilizing DINO (Self-Supervised Visual Representation) and Grounding (Language-Guided Localization) techniques, directly guiding the model to locate and identify corresponding objects in images through text prompts (such as category names or descriptions), supporting flexible natural language interaction. This model performs excellently in open-vocabulary detection tasks, especially adept at handling categories not seen during training, and is suitable for general detection scenarios requiring rapid adaptation to new scenes.
[0034] S5. Based on the target detection and localization results and the robot's position, plan the robot's path.
[0035] The following example illustrates the process of navigating objects in an unknown indoor environment. Figure 4 As shown, the target object is a "television set". Initially, the robot is randomly initialized in the bedroom. Cueed by the multimodal large model, it infers that the television is likely located in the living room and therefore performs only a coarse search in the current room. After a series of exploratory actions, the robot navigates to the living room, where it initially mistakenly identifies a chair as the target television. Guided by the corrective guidance of the multimodal large model, the robot subsequently identifies the correct object, accurately locates the television, and successfully completes the navigation task.
[0036] The zero-shot indoor object navigation method described in the above embodiments was validated on the Habitat-Matterport 3D and Matterport3D datasets, where object navigation methods are widely used, and obtained excellent experimental results. In this embodiment, the navigation decision and search methods utilize two language models, QWen-Plus and QWen-vl-max-latest, and a multimodal large model. The language model infers the location of the target object, while the multimodal large model infers the current location. The search method employs a frontier-based exploration approach, which is further improved to prevent the robot from repeatedly exploring the same area. The detection part of the navigation method utilizes the QWen-vl-max-latest multimodal large model and the Grounding-DINO zero-shot object detection model. QWen-vl-max-latest is used for initial detection and to validate the Grounding-DINO detection results. Grounding-DINO performs further detection after the multimodal large model has initially detected the presence of the target object.
[0037] The process of performing indoor object navigation is as follows: Input: RGB-D image of the environment, text information of the target item; Step 1: The robot uses an RGB-D camera to acquire RGB and depth images of the current scene for subsequent processing; Step 2: Convert the depth map into a set of points in 3D space, and transform it from the camera coordinate system to the robot base coordinate system, then perform downsampling and filtering; Step 3: Build an occupancy map using the processing results of Step 2, classify the point cloud data into obstacle areas, navigable areas and unknown areas, and project it onto a 2D plane, continuously updating it as the robot moves.
[0038] Step 4: Process the text information of the target item using the QWen-Plus large language model to infer whether the target item has a strong or weak semantic relationship with a certain type of room and the most likely room type, such as... Figure 5 As shown. QWen-Plus large language model, system text input: Text_1 = "You are a wheeled mobile robot working in an indoorenvironment. \ Your task is to find the location of the specified object in the room. \ I will provide you with a specified object, and you need to answer with the category of room that has the strongest association with that object. If the object does not have a single room category with the strongest association, or if it is strongly associated with multiple rooms, then you only need to respond: 'Weak association' — there is no need to specify a room type. For example: (1) Specified object: bed, you can respond: Strong association, bedroom. (2) Specified object: toilet, you can respond: Strong association, bathroom. (3)If the specified object is a potted plant, it could appear in any room, so you only need to respond: Weak association. Your answer should be formatted as a dict, for example: {'association':'Strong association','room':bathroom}. " For example, if the user's text input target item is Text_2 = "the television.", the result is obtained (such as Text_2 = "the toilet.", answer={'association':'Strong association','room':bathroom}).
[0039] Step 5: Using the RGB image of the current environment obtained in Step 1, infer the current room type using QWen-vl-max-latest. Multimodal large model system text input: text_vl="You are a helpful assistant.\ I will provide you with an indoor panoramic image composed of scenes from six directions. I need you to determine the type of the room based on the image. \ Your answer should be formatted, and you can only answer yes or no inlowercase.\ The user inputs text “question_vl = “Is this a " + answer_ll["room"] + "?"”, and answer_ll["room"] is the output of the large language model, thus obtaining the result (e.g., input “Is this a bathroom?”, answer = “yes”).
[0040] Step 6: Determine if there is a strong correlation between the target item obtained in Step 4 and a certain room. If not, proceed with a systematic and comprehensive search. If there is a strong correlation, then determine if the room types obtained in Step 4 and Step 5 are the same (generally they are not the same and require exploration). If they are the same, proceed with a detailed search; if they are different, perform a rough search of the current room, then quickly traverse the current area and move to other potential functional areas to continue the search.
[0041] Step 7: The final step in the search is to perform object detection: use a multimodal large model to perform preliminary detection of surrounding objects; if no target object is detected, continue exploring and update the map; if a target object is detected, call a dedicated object detection model for fine detection; if the dedicated object detection model does not detect the target object, continue exploring and update the map; if the dedicated object detection model detects the target object, the detection result is fed back to the multimodal large model for cross-validation. The QWen-vl-max-latest multimodal large model was used. During the initial detection process, the system text input was: text3="You are a helpful assistant.\ Your answer should be formatted, and you can only answer yes or no inlowercase.\ " The user inputs text such as "Is there a television in this picture?", and gets the result (e.g., input "Is there a television in this picture?", answer="yes").
[0042] The dedicated object detection model adopts the Grounding-Dino-Base zero-shot object detection model, with the text confidence score set to 0.3 and the bounding box confidence score set to 0.3. The output is the coordinate position of the target object in the image.
[0043] The QWen-vl-max-latest multimodal large model was adopted, and the system text input was the same as the preliminary detection process during the verification process.
[0044] Step 8: After detecting the target, the depth image obtained in Step 1 is processed through Step 2 to calculate the target item's location. Then, the occupancy map obtained in Step 3 is used to plan a path and navigate to the vicinity of the target item.
[0045] In another embodiment, a robot object navigation system for unknown environments includes: The image acquisition module uses an RGB-D camera to capture RGB and depth images of the current scene; A module for constructing an occupancy map is used to construct an occupancy map based on the depth image. The navigation decision and search module utilizes large language models and multimodal large models to determine search strategies based on the type of searched items; The target detection and localization module, based on the search strategy and the map, uses a target detection model to detect and locate target objects in the RGB image; The path planning module plans the robot's path based on the target detection and localization results and the robot's position.
[0046] In the above embodiments, the powerful reasoning capabilities of the large model can be fully utilized to achieve the object navigation task of the indoor robot without additional training. The detection method combining the large model and the detection model significantly improves the accuracy of object detection.
[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for robot object navigation in an unknown environment, characterized in that, include: S1. Acquire the RGB image and depth image of the current scene; S2. Construct an occupancy map based on the depth image; S3. Utilize large language models and multimodal large models to determine search strategies based on the type of searched items; S4. Based on the search strategy and the map, use the target detection model to detect and locate target items in the RGB image; S5. Based on the target detection and localization results and the robot's position, plan the robot's path.
2. The robot object navigation method in an unknown environment according to claim 1, characterized in that, The search strategies include: fine-grained search, coarse-grained search, and comprehensive search; the method of determining the search strategy based on the type of search item using large language models and multimodal large models includes: Large language models are used to infer whether a target item has a strong semantic association with a specific room type. Using a multimodal large model, the room type of the robot's current environment can be inferred; If the target item has a strong semantic relationship, determine whether the current room is the room type where the target item is most likely to appear; if so, perform a fine search on the current room; if not, perform a coarse search and move to other potential functional areas to continue the search. If the target item does not have a strong semantic relationship, a systematic and comprehensive search of the surrounding environment will be conducted, ensuring that the same area is not searched repeatedly.
3. The robot object navigation method in an unknown environment according to claim 2, characterized in that, The step of detecting and locating target objects in the RGB image using a target detection model based on the search strategy and the map includes: A multimodal large model is used to perform preliminary detection of surrounding objects; If the target item is not detected, continue exploring and update the map; If the target item is detected, the target detection model is invoked for fine-grained detection; If the object detection model does not detect the target item, continue exploring and updating the map; If the target detection model detects a target item, the detection result is fed back to the multimodal large model for cross-validation; If verification fails, continue exploring and updating the map; if verification succeeds, calculate the actual 3D coordinates based on the image area where the target item is located.
4. The robot object navigation method in an unknown environment according to claim 3, characterized in that, The search strategies described are all based on cutting-edge exploration methods and the discovered cutting-edge methods are then filtered. The map includes: obstacle areas, navigable areas, and unknown areas; The cutting-edge exploration methods include: The explorable frontier is obtained based on the map, including: extracting the boundary of the navigable area and expanding the obstacle area; the explorable frontier is the part of the boundary of the navigable area that does not overlap with the obstacle area; the explorable frontier includes several frontier point data. Each front point in the explorable frontier is filtered.
5. The robot object navigation method in an unknown environment according to claim 4, characterized in that, Filtering each front point in the explorable front includes: definition This is the set of leading edge points obtained from the most recent detection; This refers to a set of previously detected but not yet explored frontier points; This is a collection of previously explored frontier points; This represents the robot's position in a 2D base coordinate system. Defined as a point With point Horizontal distance; This represents the unknown region in the 2D occupancy map, where the value of the unknown region is 0; gather any point in For one of the points The horizontal distance is less than The set is defined as ,in It is a constant; If the following formula is satisfied, point and set Replaced by point d: If the frontier point If there are other frontier points within the range, then the coordinates of this point are the average of the coordinates of all points within that range; if this point... If there are no other frontier points within the range, the position remains unchanged.
6. The robot object navigation method in an unknown environment according to claim 5, characterized in that, Filtering each front point in the explorable front also includes: gather The following conditions must be met: the location of the leading edge point is not on the map of the already explored area, or it belongs to an unknown area on the map.
7. The robot object navigation method in an unknown environment according to claim 6, characterized in that, Filtering each front point in the explorable front also includes: The leading edge point within the preset range of the filter robot's foot.
8. The robot object navigation method in an unknown environment according to claim 7, characterized in that, Filtering each front point in the explorable front also includes: The cutting edge that filtering robots have already explored.
9. A robot object navigation method in an unknown environment according to claim 3, characterized in that, The target detection model is a Grounding-Dino-Base zero-shot target detection model, and the output is the coordinate position of the target object in the image.
10. A robot object navigation system for unknown environments, characterized in that, include: The image acquisition module uses an RGB-D camera to capture RGB and depth images of the current scene; A module for constructing an occupancy map is used to construct an occupancy map based on the depth image. The navigation decision and search module utilizes large language models and multimodal large models to determine search strategies based on the type of searched items; The target detection and localization module, based on the search strategy and the map, uses a target detection model to detect and locate target objects in the RGB image; The path planning module plans the robot's path based on the target detection and localization results and the robot's position.