Scene map navigation method and device based on knowledge base and medium

Abstract

The embodiment of the invention discloses a scene map navigation method and device based on a knowledge base and a medium, and the method comprises the steps: receiving a task instruction of a current user based on a preset language large model, so as to judge whether the task instruction is associated with an existing instruction of a preset knowledge base; if not, acquiring the current location of the robot corresponding to the task instruction, and performing target search on the scene map according to the current location to determine the room to which the target object belongs; if yes, based on the current positioning of the robot corresponding to the existing instruction and the task instruction, performing target search to determine the room to which the target object belongs; and receiving the current location of the robot, the scene map and the task instruction to perform task planning to generate an action sequence corresponding to the task instruction, so as to execute navigation of each target position point in the action sequence based on a preset function packet in the scene map.

Description

Technical Field

[0001] This specification relates to the field of large language models, and in particular to a knowledge base-based scene graph navigation method, device, and medium. Background Art

[0002] Robotic SLAM technology is the foundation for autonomous robot navigation. It enables robots to use their sensors to collect data in unknown environments, then analyze this data using specialized algorithms to determine their position and posture, thereby building a map of their surroundings. In recent years, with the rapid development of information technologies such as artificial intelligence and knowledge graphs, scene graphs have emerged as a new technological path. Traditional map-based navigation methods, limited by map accuracy and update efficiency, struggle to cope with unknown or changing environments. Scene graphs, by transforming the robot's environment into a multidimensional relationship graph composed of nodes and edges, provide a new solution for indoor robot positioning and navigation.

[0003] However, in the current traditional navigation process, all possible task instructions need to be defined first. When the user issues an unforeseen instruction, the robot may need to rely on manual intervention or become unresponsive, resulting in a fragmented interactive experience. In addition, current robots generally obtain positioning through SLAM and then match the target position based on a static database. However, they are unable to dynamically optimize the search path by combining scene maps, resulting in low efficiency. In addition, in cross-room navigation tasks, the robot needs to simultaneously process multiple sources of information, such as user instructions, real-time positioning, and scene object relationships. Traditional methods rely on hard-coded rules and are difficult to adapt to scene changes, resulting in rigid robot action sequences and poor fault tolerance. Summary of the Invention

[0004] In order to solve the above technical problems, one or more embodiments of this specification provide a knowledge base-based scene graph navigation method, device and medium.

[0005] One or more embodiments of this specification adopt the following technical solutions: One or more embodiments of this specification provide a scene graph navigation method based on a knowledge base, the method comprising: Receive the task instruction of the current user based on the preset language model to determine whether the task instruction is related to the existing instructions in the preset knowledge base; If not, obtaining the current location of the robot corresponding to the task instruction, and performing a target search on the scene map according to the current location to determine the room to which the target object belongs; If there is a correlation, a target search is performed to determine the room to which the target object belongs based on the current position of the robot corresponding to the existing instruction and the task instruction; The current positioning of the robot, the scene map and the task instructions are received to perform task planning and generate an action sequence corresponding to the task instructions, so as to execute navigation to each target position point in the action sequence based on a preset function package in the scene map.

[0006] Optionally, in one or more embodiments of the present specification, receiving a task instruction of the current user based on a preset language macro model to determine whether the task instruction is associated with an existing instruction in a preset knowledge base specifically includes: receiving a task instruction of the current user based on the preset language model, and determining a sentence structure of the task instruction according to the part of speech corresponding to each word in the task instruction; Encoding each short sentence in the task instruction to obtain a sentence vector, so as to extract semantic information of the task instruction based on the sentence vector; Based on the semantic structure and the semantic information, similarity matching is performed with existing instructions in a preset knowledge base to determine whether the task instruction is associated with the existing instructions in the preset knowledge base.

[0007] Optionally, in one or more embodiments of this specification, obtaining the current location of the robot corresponding to the task instruction, and performing a target search on the scene graph according to the current location to determine the room to which the target object belongs, specifically includes: Determining the current position of the robot corresponding to the task instruction based on the robot number corresponding to the task instruction and the preset position sensor of each robot; Aligning the current location with the hierarchical structure corresponding to the scene graph to determine the location node of the current location within the scene graph; Obtaining a room list of a hierarchical structure where the location node is located, traversing each room in the room list based on the location node, and determining a plurality of candidate rooms corresponding to the task instruction according to a semantic relationship between the scene graph and the task instruction; The object lists in each candidate room are queried in turn. If it is determined that the target object exists in the object list of the current room, the room is returned as the room to which it belongs.

[0008] Optionally, in one or more embodiments of this specification, after obtaining the current location of the robot corresponding to the task instruction, and performing a target search on the scene graph based on the current location to determine the room to which the target object belongs, the method further includes: A data structure corresponding to the task instruction is created in a preset knowledge base, so as to store the target object, the room and the current location related to the task instruction in the preset knowledge base based on the data result.

[0009] Optionally, in one or more embodiments of the present specification, performing target search to determine the room to which the target object belongs based on the existing instruction and the current location of the robot corresponding to the task instruction specifically includes: Obtaining target object and target room information associated with each of the existing instructions, and determining a distance between the current location and the target room based on the target room information associated with each of the existing instructions and the current location of the robot corresponding to the task instruction; sorting the target rooms based on the distances to select a preset number of target rooms closest to the target rooms as candidate rooms; Expanding the object list in the candidate room, and if it is determined that the target object does not exist in the object list of the candidate room, continuing to expand other target rooms; If it is determined that the target object exists in the object list of the current room, the room is returned as the room to which it belongs.

[0010] Optionally, in one or more embodiments of this specification, receiving the current position of the robot, the scene map, and the task instruction to perform task planning and generate an action sequence corresponding to the task instruction specifically includes: Integrating the current position of the robot, the scene map, and the task instruction to obtain integrated task-related information, and determining a preset prompt text corresponding to the integrated task-related information; The integrated task-related information and the preset prompt text are input into the preset large language model to perform task planning, and an action sequence corresponding to the task instruction is generated.

[0011] Optionally, in one or more embodiments of the present specification, performing navigation of each target location point in the action sequence based on a preset function package in the scene graph specifically includes: Obtaining a target position point and a corresponding execution target for each action in the action sequence, and determining the position information of each action in the scene graph based on the target position point and the execution target; Inputting the posture information of each action into the preset function package based on the order of the action sequence, so as to determine a feasible path from the current position corresponding to each execution target of the robot to the target position point based on the preset function package; A navigation instruction is generated based on the feasible path and sent to the robot to control the robot to perform navigation movement and iteratively execute navigation corresponding to each target position point.

[0012] Optionally, in one or more embodiments of the present specification, the intelligent body device of the robot is equipped with a computing unit, a camera, an IMU and a sensor; a ROS system is deployed in the computing unit; and each of the sensors has a corresponding joint calibration.

[0013] One or more embodiments of this specification provide a knowledge base-based scene graph navigation device, the device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: perform any of the above methods.

[0014] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute any of the above-described methods.

[0015] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: Using a large, pre-built language model, the robot determines the relevance of task instructions to existing instructions in the pre-built knowledge base. For unrelated task instructions, the robot's current location is determined, and the target room is determined through a scene graph search. This eliminates the need for manual intervention, ensuring smooth interaction and significantly improving the user-robot experience. For related task instructions, the robot fully considers the distance between the navigation room and the robot's current location, avoiding the tendency of the large language model to prioritize distant locations when planning tasks. When determining the target room and subsequently planning tasks, the robot leverages the rich information provided by the scene graph to dynamically optimize the search path, enabling it to reach the target more efficiently and saving time and energy. Task planning is performed by integrating the task instruction, the robot's current location, and the scene graph to generate an action sequence, which is then executed using a pre-built function package. This approach better processes information from multiple sources, flexibly responds to scene changes, and makes the robot's action sequences more flexible and fault-tolerant. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some of the embodiments described in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without inventive work. In the drawings: Figure 1 A schematic flow chart of a scene graph navigation method based on a knowledge base provided in an embodiment of this specification; Figure 2 A schematic diagram of a knowledge-based scene graph navigation technology in a scenario provided by an embodiment of this specification; Figure 3 This is an example diagram of a prompt text provided in the embodiments of this specification; Figure 4 A schematic diagram of the structure of a scene graph navigation device based on a knowledge base provided in an embodiment of this specification; Figure 5 A schematic diagram of the structure of a non-volatile storage medium provided in an embodiment of this specification. DETAILED DESCRIPTION

[0017] The embodiments of this specification provide a knowledge base-based scene graph navigation method, device, and medium.

[0018] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative work should fall within the scope of protection of this specification.

[0019] like Figure 1 As shown in FIG, the embodiment of this specification provides a flow chart of a scene graph navigation method based on a knowledge base. Figure 1 It can be seen that in one or more embodiments of this specification, a scene graph navigation method based on a knowledge base includes: S101: receiving a task instruction of the current user based on a preset language macro model to determine whether the task instruction is associated with an existing instruction in a preset knowledge base.

[0020] In the embodiments of this specification, the current user's task instructions are received based on a pre-set language model to determine whether the task instructions are related to existing instructions in the pre-set knowledge base. For related instructions, the relevant information and processing methods in the knowledge base can be reused, saving time and computing resources, improving the robot's task execution efficiency, and enabling it to respond to user instructions in a shorter time. For unrelated task instructions, the scene graph can be used to re-target the room. This avoids the problem of the robot needing manual intervention or becoming unresponsive when the user issues an unexpected instruction, resulting in a fragmented interactive experience. It should be noted that the pre-set knowledge base contains historical information on navigation tasks, including the name of each task instruction, the target object, the room to which the target object belongs, and the starting position of the chassis. For example, if the robot is given the instruction "Take me to see TV" in the bedroom, the information stored in the knowledge base will be: {"Task instruction": "Take me to see TV"; "Target object": "TV"; "Target room": "Living room"; "Starting position": "Bedroom"}.

[0021] Specifically, in one or more embodiments of the present specification, receiving the current user's task instruction based on the preset language model to determine whether the task instruction is associated with existing instructions in the preset knowledge base specifically includes: like Figure 2 The system receives the current user's task instructions based on a preset language model and then analyzes the part of speech of each word in the instruction, such as noun, verb, adjective, etc. Based on the part of speech corresponding to each word in the task instruction, the sentence structure of the task instruction can be determined, such as the subject-verb-object structure or the verb-object structure, which helps to grasp the basic framework and logical order of the instruction. The task instruction is then divided into several short sentences, each of which is encoded and converted into a sentence vector. The semantic information of the task instruction is then extracted based on the sentence vector. The sentence structure and semantic information obtained above are then matched with the existing instructions in the preset knowledge base for similarity. If the similarity reaches a certain threshold, the task instruction is determined to be related to the existing instructions; otherwise, it is considered unrelated.

[0022] This process comprehensively considers both sentence structure and semantic information, enabling a more comprehensive and accurate understanding of the meaning of task instructions. Sentence structure reflects the grammatical rules and logical relationships of the instruction, while semantic information reflects the actual meaning of the instruction. Combining the two can avoid misunderstandings caused by different expressions. Furthermore, by extracting semantic information for similarity matching, existing instructions related to the task instruction can be more accurately identified, improving matching accuracy.

[0023] S102: If not, the current location of the robot corresponding to the task instruction is obtained, so as to perform a target search on the scene map according to the current location to determine the room to which the target object belongs.

[0024] If the current user's task instruction is determined to be unrelated to existing instructions in the pre-set knowledge base, this means that the task cannot be handled directly based on the existing experience and logic in the knowledge base. At this point, the robot's onboard positioning system will be used to obtain the robot's current location corresponding to the task instruction. Based on this current location, a target search is then performed on the scene graph to determine the room to which the target object belongs. It should be noted that the scene graph is a data structure that converts the robot's environment into a multidimensional relationship graph composed of nodes and edges, containing rich environmental semantic information, such as room layout and object position relationships. It should also be noted that the robot's intelligent body device is equipped with a computing unit, camera, IMU, and sensors. Specifically, a single robot intelligent body device is equipped with a computing unit, camera, IMU, sensors, etc. The ROS system is already deployed in the computing unit, and the sensors have been jointly calibrated accordingly.

[0025] In certain application scenarios, a large language model is used to determine the room within the scene graph where the target object resides, using inputs such as a scene graph, the robot's current location, and task instructions. To avoid excessively large tokens being passed to the large language model in large scenes, which would reduce the model's response speed, the scene graph often adopts a hierarchical structure: "building - floor - room - object." The scene graph passed to the large language model is displayed only up to the "building - floor - room" level, preserving the main structure of the scene graph while reducing the amount of scene data passed to the large language model. The large language model is then iteratively expanded to the room most likely to contain the target object. Once the target object is found, the target search is complete.

[0026] In this process, the problem of traditional navigation methods based on fixed rules or maps being difficult to quickly and accurately locate target objects in complex and changing environments is solved. The scene graph provides a more flexible and semantic environmental representation, and combined with the robot's current positioning for search, it can better adapt to different indoor environment layout changes and improve the robot's adaptability and navigation efficiency in complex environments.

[0027] Specifically, in one or more embodiments of this specification, obtaining the current location of the robot corresponding to the task instruction, and performing a target search on the scene graph to determine the room to which the target object belongs based on the current location, specifically includes: Different robots are distinguished by their unique numbers, and each is equipped with pre-installed position sensors such as lidar, visual sensors, and GPS. Based on the robot number corresponding to the task instruction, the system can accurately identify the target robot from among a large number of robots. Using the robot's onboard position sensors, the system can determine the current location of the robot corresponding to the task instruction. Because the scene graph is a hierarchical data representation that abstracts and organizes spatial information and object information in the real environment, in order to determine the robot's position within the scene graph's hierarchical structure, the embodiments of this specification align the current location with the hierarchical structure corresponding to the scene graph to determine the location node of the current location within the scene graph. Each location node in the hierarchical structure is associated with a room list containing information about possible rooms surrounding the location node. Starting from this location node, the system traverses each room in the room list. Based on the semantic information contained in the scene graph, such as the room's function and the relationship between objects and rooms, as well as the semantic content of the task instruction, the system analyzes the relevance of the task instruction to each room and selects multiple candidate rooms that match the semantics of the task instruction. For example, if the task instruction is "find a book," the room marked "Study" in the scene graph is more likely to be the target room and will be included in the candidate room list. For each candidate room obtained in the previous step, the object list within it is queried in turn. The object list records detailed information about the various objects in the room. During the query process, it is determined whether the target object exists in the object list of the current room. If so, the room is determined to be the room to which the target object belongs and is returned as the result. If the target object is not found after traversing all candidate rooms, it may be necessary to further expand the search range or prompt that the task cannot be completed.

[0028] In this process, the current position is determined using the robot number and pre-positioned position sensors, achieving high accuracy and quickly locating the target robot from a large number of robots. Aligning the current position with the scene graph hierarchy to determine the location node makes searching for relevant information in the scene graph more efficient, enabling rapid acquisition of surrounding room information, reducing the search scope, and improving positioning and matching speed. Candidate rooms are determined based on the semantic relationship between the scene graph and the task instructions, selecting the rooms most likely to contain the target object. This improves search accuracy and specificity, and avoids ineffective searches in unrelated rooms.

[0029] Furthermore, in one or more embodiments of the present specification, after obtaining the current position of the robot corresponding to the task instruction and performing a target search on the scene map based on the current position to determine the room to which the target object belongs, in order to be able to subsequently reuse the task instruction-related data that does not exist in the existing task instructions, the method also includes the following process: creating a data structure corresponding to the task instruction in a preset knowledge base, so as to store information such as the target object, room and current position related to the task instruction in the preset knowledge base based on the data results, so as to update the existing task instructions in the preset database.

[0030] S103: If there is an association, a target search is performed to determine the room to which the target object belongs based on the existing instruction and the current position of the robot corresponding to the task instruction.

[0031] When there is an association, in order to fully consider the distance information between the target room and the current position of the robot in the scene graph, the embodiments of this specification prioritize searching for rooms that are close and associated with the current task instructions. A target search is performed based on the existing instructions and the current position of the robot corresponding to the task instructions, thereby determining the room to which the target object belongs. Specifically, in one or more embodiments of this specification, a target search is performed based on the existing instructions and the current position of the robot corresponding to the task instructions to determine the room to which the target object belongs, specifically including the following process: First, the target object and target room information associated with each existing instruction is obtained, and then the distance between the current location and the target room is determined based on the target room information associated with each existing instruction and the current location of the robot corresponding to the task instruction. For example: In a certain application scenario, the target room information associated with each existing instruction will be obtained. The distance from the current position of the robot chassis : Where, and Represents the target room and the position coordinates of the robot chassis.

[0032] Then, sort the target rooms based on the distance. The robot then sorts the objects from closest to farthest, selecting a preset number of the closest target rooms as candidate rooms. For example, the three closest target rooms are selected as candidate rooms. The object list within each candidate room is expanded sequentially. If the target object is determined not to be in the candidate room's object list, the robot continues expanding the remaining target rooms, and this process continues until all candidate rooms have been traversed. Once the target object is found in a candidate room's object list, that room is identified as the target object's room and returned as the result. By calculating distances and prioritizing searches of closer rooms, this approach reduces unnecessary robot movement and reduces the search range. Compared to blindly searching all rooms, this method can more quickly locate the area where the target object may be located, saving significant search time. This advantage is particularly evident in larger environments or complex scenes. Furthermore, the candidate room object list is searched sequentially in a specific order, ensuring a comprehensive and accurate search.

[0033] S104: Receive the current position of the robot, the scene map and the task instructions to perform task planning and generate an action sequence corresponding to the task instructions, so as to execute navigation of each target position point in the action sequence based on a preset function package in the scene map.

[0034] The robot's current position, scene map, and task instructions are received during the above process to perform task planning and generate an action sequence corresponding to the task instructions, so as to execute navigation to each target position point in the action sequence in the scene map based on the preset function package. Specifically, in one or more embodiments of this specification, the robot's current position, scene map, and task instructions are received to perform task planning and generate an action sequence corresponding to the task instructions, specifically including: The robot's current location, scene map and task instructions are integrated to obtain integrated task-related information. Based on the integrated task-related information, matching preset prompt texts are selected from the pre-set prompt text library. These prompt texts are pre-written based on different task scenarios, environmental conditions and robot status factors, and can provide guidance and supplementary information for subsequent task planning, such as Figure 3The following figure shows an example of a prompt text in a certain scenario, including: agent role, knowledge base, instructions, output format, and examples. The "agent role" defines the main functions of the large language model; the "knowledge base" explains the format of historical instructions; the "instructions" are the task instructions issued by the current user; the "output format" specifies the data structure of the large language model output, including relevant instructions, relevant objects, and relevant rooms; the "examples" illustrate the input and output mechanism of the large language model through specific cases. After the user issues a task instruction to the robot, the large language model determines whether the current task instruction is associated with the historical instructions in the knowledge base. For example, the new task instruction "I want to watch cartoons" can be associated with the historical instruction "Take me to see TV" in the knowledge base. The target object of both instructions is "TV" and the target room is "living room."

[0035] The integrated task-related information and preset prompt text are then input into the preset large language model for task planning, generating the action sequence corresponding to the task instruction. For example, for the task instruction "I want to watch cartoons" issued in the "bedroom", the target object "TV" has been determined to be located in the "living room" in the scene graph. The task planning can generate the action sequence "Go to the living room - approach the TV" through the large language model. After completing the task planning, the generated action sequence is passed to the large language model again to verify the executable nature of the action sequence. For example, for the action of taking out a cold drink from the refrigerator, make sure to open the refrigerator door before grabbing the cold drink, and close the refrigerator door after taking it out.

[0036] Specifically, in one or more embodiments of this specification, performing navigation to each target location point in an action sequence based on a preset function package in a scene graph specifically includes: The target location and execution goal corresponding to each action are extracted from the action sequence. The target location specifies the specific location the robot needs to reach in the scene, while the execution goal describes the task to be completed at that location, such as grasping an object or performing environmental monitoring. Then, the pose information of each action within the scene graph is determined based on the target location and execution goal. Based on the order of the action sequence, the pose information of each action is input into the preset function package. Based on the preset function package, a feasible path from the robot's current position to the target location corresponding to each execution goal is determined. This preset function package can be the movebase function package. Navigation instructions are generated based on the feasible path and sent to the robot to control navigation movement, iteratively executing navigation corresponding to each target location. Specifically, navigation to the target location is executed by the movebase function package, while actions involving grasping are performed by the robot arm and object recognition. Path planning and robot movement control according to the action sequence ensure consistent and efficient task execution. Iterative navigation to each target location enables the robot to complete complex tasks in an orderly manner, reducing unnecessary waiting and movement time and improving overall task execution efficiency.

[0037] like Figure 4 As shown in FIG, an embodiment of this specification provides a structural diagram of a scene graph navigation device based on a knowledge base. Figure 4 It can be seen that in one or more embodiments of this specification, a scene graph navigation device based on a knowledge base includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: perform any of the above methods.

[0038] like Figure 5 As shown in FIG, the embodiment of this specification provides a structural diagram of a non-volatile storage medium. Figure 5 It can be seen that in one or more embodiments of this specification, a non-volatile storage medium stores computer-executable instructions 501, and the computer-executable instructions 501 are configured to be capable of executing any of the methods described above.

[0039] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0040] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0041] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A scene graph navigation method based on a knowledge base, characterized in that: The method comprises: Receive the task instruction of the current user based on the preset language model to determine whether the task instruction is related to the existing instructions in the preset knowledge base; If not, obtaining the current location of the robot corresponding to the task instruction, and performing a target search on the scene map according to the current location to determine the room to which the target object belongs; If there is a correlation, a target search is performed to determine the room to which the target object belongs based on the current position of the robot corresponding to the existing instruction and the task instruction; The current positioning of the robot, the scene map and the task instructions are received to perform task planning and generate an action sequence corresponding to the task instructions, so as to execute navigation to each target position point in the action sequence based on a preset function package in the scene map.

2. A scene graph navigation method based on a knowledge base according to claim 1, characterized in that: Receive the current user's task instruction based on the preset language model to determine whether the task instruction is associated with the existing instructions in the preset knowledge base, specifically including: receiving a task instruction of the current user based on the preset language model, and determining a sentence structure of the task instruction according to the part of speech corresponding to each word in the task instruction; Encoding each short sentence in the task instruction to obtain a sentence vector, so as to extract semantic information of the task instruction based on the sentence vector; Based on the semantic structure and the semantic information, similarity matching is performed with existing instructions in a preset knowledge base to determine whether the task instruction is associated with the existing instructions in the preset knowledge base.

3. The scene graph navigation method based on a knowledge base according to claim 1, characterized in that: Obtaining the current location of the robot corresponding to the task instruction, and performing a target search on the scene map according to the current location to determine the room to which the target object belongs, specifically includes: Determining the current position of the robot corresponding to the task instruction based on the robot number corresponding to the task instruction and the preset position sensor of each robot; Aligning the current location with the hierarchical structure corresponding to the scene graph to determine the location node of the current location within the scene graph; Obtaining a room list of a hierarchical structure where the location node is located, traversing each room in the room list based on the location node, and determining a plurality of candidate rooms corresponding to the task instruction according to a semantic relationship between the scene graph and the task instruction; The object lists in each candidate room are queried in turn. If it is determined that the target object exists in the object list of the current room, the room is returned as the room to which it belongs.

4. The scene graph navigation method based on a knowledge base according to claim 1, characterized in that: After obtaining the current location of the robot corresponding to the task instruction and performing a target search on the scene map based on the current location to determine the room to which the target object belongs, the method further includes: A data structure corresponding to the task instruction is created in a preset knowledge base, so as to store the target object, the room and the current location related to the task instruction in the preset knowledge base based on the data result.

5. The scene graph navigation method based on a knowledge base according to claim 1, characterized in that: Based on the existing instructions and the current location of the robot corresponding to the task instructions, a target search is performed to determine the room to which the target object belongs, specifically including: Obtaining target object and target room information associated with each of the existing instructions, and determining a distance between the current location and the target room based on the target room information associated with each of the existing instructions and the current location of the robot corresponding to the task instruction; sorting the target rooms based on the distances to select a preset number of target rooms closest to the target rooms as candidate rooms; Expanding the object list in the candidate room, and if it is determined that the target object does not exist in the object list of the candidate room, continuing to expand other target rooms; If it is determined that the target object exists in the object list of the current room, the room is returned as the room to which it belongs.

6. The scene graph navigation method based on a knowledge base according to claim 1, characterized in that: Receiving the current position of the robot, the scene map, and the task instruction to perform task planning and generate an action sequence corresponding to the task instruction, specifically including: Integrating the current position of the robot, the scene map, and the task instruction to obtain integrated task-related information, and determining a preset prompt text corresponding to the integrated task-related information; The integrated task-related information and the preset prompt text are input into the preset large language model to perform task planning, and an action sequence corresponding to the task instruction is generated.

7. The scene graph navigation method based on a knowledge base according to claim 1, characterized in that: Executing navigation of each target location point in the action sequence based on the preset function package in the scene graph specifically includes: Obtaining a target position point and a corresponding execution target for each action in the action sequence, and determining the position information of each action in the scene graph based on the target position point and the execution target; Inputting the posture information of each action into the preset function package based on the order of the action sequence, so as to determine a feasible path from the current position corresponding to each execution target of the robot to the target position point based on the preset function package; A navigation instruction is generated based on the feasible path and sent to the robot to control the robot to perform navigation movement and iteratively execute navigation corresponding to each target position point.

8. The scene graph navigation method based on a knowledge base according to claim 1, characterized in that: The robot's intelligent body device is equipped with a computing unit, a camera, an IMU and a sensor; the computing unit is equipped with a ROS system; and each of the sensors has a corresponding joint calibration.

9. A scene graph navigation device based on a knowledge base, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: execute the method according to any one of claims 1 to 8.

10. A non-volatile storage medium storing computer-executable instructions, characterized in that: The computer executable instructions can execute the method according to any one of claims 1 to 8.

Images