Robot positioning and navigation method and device based on large language model thinking chain

Abstract

The invention discloses a robot positioning and navigation method and device based on a large language model thinking chain, and belongs to the field of intelligent navigation, and the method comprises the steps: obtaining a target instruction, and collecting the current position of a robot and the image information of the current position; performing semantic analysis on the target instruction and the image information of the current position by using a visual language model to generate a navigation thinking chain; and determining the next position of the robot according to the navigation thinking chain, and generating a navigation route of the robot according to the next position and the current position. Therefore, the problems that an existing navigation method generally lacks a systematic reasoning process, and abundant semantic information contained in a model cannot be fully utilized to optimize a navigation task are solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent navigation technology, and in particular to a robot positioning and navigation method and device based on a large language model thinking chain. Background Art

[0002] Traditional robot navigation systems rely on geometric and physical sensor data for environment recognition and path planning. These methods face limitations when dealing with complex instructions and unknown environments. In recent years, large language models (LLMs) have been developed to process and understand complex language instructions, but most existing works use end-to-end methods, which usually lack a systematic reasoning process and cannot fully utilize the rich semantic information contained in the model to optimize navigation tasks. Summary of the invention

[0003] The present invention provides a robot positioning and navigation method and device based on a large language model thinking chain, so as to solve the problem that the existing navigation methods usually lack a systematic reasoning process and cannot fully utilize the rich semantic information contained in the model to optimize the navigation task.

[0004] The first aspect of the present invention provides a robot positioning and navigation method based on a large language model thinking chain, comprising the following steps:

[0005] Obtaining target instructions, collecting the current position of the robot, and image information of the current position;

[0006] Using a visual language model to perform semantic analysis on the target instruction and the image information of the current position to generate a navigation thought chain;

[0007] The navigation thought chain is analyzed by using a language scoring model to determine the next position of the robot, and a navigation route of the robot is generated according to the next position and the current position.

[0008] Optionally, in one embodiment of the present invention, a GPS system installed on the robot is used to collect the current position of the robot; and a camera installed on the robot is used to collect image information of the current position of the robot.

[0009] Optionally, in one embodiment of the present invention, a visual language model is used to perform semantic analysis on the target instruction and the image information of the current position to generate a navigation thought chain, including:

[0010] The image information of the current position is analyzed according to the semantics of the target instruction to obtain multiple position point information in the image information, and the navigation thinking chain composed of multiple paths to realize the target instruction is generated according to the multiple position point information.

[0011] Optionally, in one embodiment of the present invention, determining the next position of the robot according to the navigation thought chain and the current position of the robot to generate a navigation route of the robot includes:

[0012] Scoring the multiple paths in the navigation thinking chain, selecting the path with the highest score among the multiple scores as the next position of the robot, and generating the navigation route of the robot according to the next position and the current position.

[0013] Optionally, in one embodiment of the present invention, the formula for scoring the multiple paths in the navigation thinking chain is:

[0014] S(i)=\sum_{k=1}^{K}w_k\cdot\text{LM}(c_k|\text{Context})

[0015] Among them, (S(i)) represents the score of each position i, w_k is the feature weight, (text{LM}) represents the call of the large language model, (c_k) is the associated semantic feature, and (text{Context}) is the current task context.

[0016] Optionally, in one embodiment of the present invention, the formula for selecting the path with the highest score among multiple scores as the next position of the robot is:

[0017] text{NextStep}=\arg\max_{s\in\text{Steps}}\left(\alpha\cdot S(s)-\beta\cdot\text{Dist}(s)\right)

[0018] Among them, (text{NextStep}) is the next action selected, (alpha\) and (beta\) are weight coefficients used to adjust the influence of the thought chain score S(s) and the distance cost (text{Dist}(s)), and (s) represents a possible navigation step.

[0019] A second aspect of the present invention provides a robot positioning and navigation device based on a large language model thinking chain, comprising:

[0020] A data acquisition module is used to obtain target instructions, collect the current position of the robot, and image information of the current position;

[0021] A thought chain generation module, used for performing semantic analysis on the target instruction and the image information of the current position by using a visual language model to generate a navigation thought chain;

[0022] The navigation decision module is used to analyze the navigation thought chain by using a language scoring model, determine the next position of the robot, and generate a navigation route of the robot according to the next position and the current position.

[0023] Optionally, in one embodiment of the present invention, the data acquisition module specifically includes:

[0024] The GPS system installed on the robot is used to collect the current position of the robot;

[0025] The camera installed on the robot is used to collect image information of the robot's current position.

[0026] Optionally, in one embodiment of the present invention, the thinking chain generation module is specifically used to analyze the image information of the current position according to the semantics of the target instruction, obtain multiple location point information in the image information, and generate the navigation thinking chain composed of multiple paths to implement the target instruction according to the multiple location point information.

[0027] Optionally, in one embodiment of the present invention, the navigation decision module is specifically used to score multiple paths in the navigation thinking chain, select the path with the highest score among the multiple scores as the next position of the robot, and generate a navigation route of the robot based on the next position and the current position.

[0028] The robot positioning and navigation method and device based on the large language model thinking chain of the embodiment of the present invention optimizes the robot's navigation capability through in-depth semantic reasoning, and can achieve accurate positioning and navigation in complex or unknown environments by combining the thinking chain generated by the large language model with the real-time perception data of the environment.

[0029] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0031] Figure 1 A flowchart of a robot positioning and navigation method based on a large language model thinking chain provided according to an embodiment of the present invention;

[0032] Figure 2 This is an example diagram of a robot positioning and navigation device based on a large language model thinking chain according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0034] Figure 1 The present invention provides a flowchart of a robot positioning and navigation method and device based on a large language model thinking chain according to an embodiment of the present invention.

[0035] like Figure 1 As shown, the robot positioning and navigation method based on the large language model thinking chain includes the following steps:

[0036] Step S1, obtain target instructions, collect the current position of the robot, and image information of the current position.

[0037] Optionally, in one embodiment of the present invention, a GPS system installed on the robot is used to collect the current position of the robot; and a camera installed on the robot is used to collect image information of the current position of the robot.

[0038] The embodiment of the present invention collects RGBD images and position data of the robot's environment in real time, providing real-time input for thought chain generation and navigation decision-making. In the embodiment of the present invention, the robot can be any intelligent agent used for navigation, the collected image information can be RGBD images or images in other formats, and the position data can be coordinate data.

[0039] Step S2, using the visual language model to perform semantic analysis on the target instruction and the image information of the current position to generate a navigation thought chain.

[0040] Optionally, in one embodiment of the present invention, a visual language model is used to perform semantic analysis on the target instruction and the image information of the current position to generate a navigation thought chain, including:

[0041] The image information of the current position is analyzed according to the semantics of the target instruction to obtain multiple position point information in the image information, and a navigation thinking chain consisting of multiple paths to realize the target instruction is generated according to the multiple position point information.

[0042] The visual language model is a model that combines image processing and natural language understanding. It can analyze image and text information at the same time. Its purpose is to extract useful information from the image and combine it with the given instructions or goals to generate reasonable action suggestions. In the embodiments of the present application, the visual language model can be fine-tuned using an existing model architecture, such as Segment Anything (SAM), and no specific limitation is made to this.

[0043] The target instruction is a specific navigation requirement given by the user, such as "go to the conference room" or "find the nearest exit". The image information of the current location is a real-time image of the current location captured by the camera. These images provide visual information of the environment, such as the layout of the building, signs, obstacles, etc. The visual language model will first analyze the content in the image. For example, identify the logo of the building, the sign on the wall, or the layout of the passage. The visual language model interprets the semantics of the target instruction, such as "conference room" may point to a specific room or area. Based on the image information, the visual language model understands the specific details of the current location, such as "you are currently in the lobby. The visual language model combines the target instruction to generate a specific navigation thought chain. For example, if the goal is "go to the conference room", the visual language model may determine multiple paths from the current location to the conference room and provide steps, such as "go left from the lobby, turn right after passing two doors, and the conference room is on the left side of the first corridor" or "go right from the lobby, turn left after passing the corridor, and the conference room is at the second door".

[0044] It should be noted that the visual language model used in the present invention is a universal model architecture, which is obtained by fine-tuning the architecture and adjusting the parameters according to specific actual conditions and by training.

[0045] Step S3, using the language scoring model to analyze the navigation thought chain, determine the next position of the robot, and generate the navigation route of the robot according to the next position and the current position.

[0046] Optionally, in one embodiment of the present invention, a language scoring model is used to analyze the navigation thinking chain to determine the next position of the robot to generate a navigation route of the robot, including: scoring multiple paths in the navigation thinking chain, selecting the path with the highest score from multiple scores as the next position of the robot, and generating the navigation route of the robot based on the next position and the current position.

[0047] Specifically, there are multiple paths in the navigation thinking chain, that is, different navigation options. The language scoring model is used to score the multiple paths to evaluate their effectiveness and priority.

[0048] For example, suppose the robot is in a hall and needs to go to a conference room. There are several passages in the hall, and the robot needs to choose the best path. The navigation thinking chain obtained through the above embodiment includes three paths, which are scored. Path A: Go straight for 100 meters, turn left to enter the conference room, and score 8. Path B: Go straight for 50 meters, turn right, and go straight for another 50 meters to reach the conference room, and score 7. Path C: Go straight for 80 meters, turn left and go straight for 20 meters to reach the conference room, and score 9. Path C has the highest score, so this path is selected as the optimal path. The optimal path is used as the next position of the robot, and the navigation route of the robot is obtained in combination with the current position of the robot. In the specific navigation process, the robot can also realize obstacle avoidance function and correct the route in real time.

[0049] It should be noted that the language scoring model used in the present invention is a general model architecture, which is obtained by fine-tuning the architecture and adjusting the parameters according to specific actual conditions and performing training.

[0050] Optionally, in a specific embodiment of the present invention, the formula for scoring multiple paths in the navigation thinking chain is:

[0051] S(i)=\sum_{k=1}^{K}w_k\cdot\text{LM}(c_k|\text{Context})

[0052] Among them, \(S(i)\) represents the score of sub-goal \(i\), \(w_k\) is the feature weight, \(\text{LM}\) represents the call of the large language model, \(c_k\) is the associated semantic feature, and \(\text{Context}\) is the current task context.

[0053] Optionally, in a specific embodiment of the present invention, the formula for selecting the path with the highest score among multiple scores as the next position of the robot is:

[0054] \text{NextStep}=\arg\max_{s\in\text{Steps}}\left(\alpha\cdot S(s)-\beta\cdot\text{Dist}(s)\right)

[0055] Among them, \(\text{NextStep}\) is the next action selected, \(\alpha\) and \(\beta\) are weight coefficients that adjust the influence of the thought chain score \(S(s)\) and the distance cost \(\text{Dist}(s)\), and \(s\) represents a possible navigation step.

[0056] The robot positioning and navigation method based on the large language model thinking chain of the present invention is described in detail below through specific embodiments.

[0057] Take the inspection robot as an example. The high-resolution camera and GPS positioning system on the inspection robot. The robot is deployed in a wide area of ​​power transmission lines for routine inspections. The inspection mission aims to check the integrity of the transmission lines and identify and report potential damage or abnormal conditions. The specific steps include:

[0058] Task initiation: When the inspection task begins, the operator uses natural language to input the designated inspection area and specific inspection focus, such as "check the integrity of the near-ground transmission lines in the northern area."

[0059] Data acquisition: The observation module collects images of the transmission line and the position data of the robot in real time.

[0060] Thought chain construction: The thought chain generation module analyzes the operator's instructions and the collected images to generate a thought chain about how to perform the inspection. For example, the model may infer that special attention needs to be paid to a line that may be covered by branches or blown by the wind.

[0061] Navigation and execution: The navigation decision system plans the best inspection route based on the thought chain output and real-time geographic location. The system dynamically adjusts the inspection route to cover all key areas while avoiding obstacles such as large trees.

[0062] Anomaly detection and reporting: If the observation module detects potential problems, such as line damage or abnormal wear, the robot will stop for a detailed inspection and send the data back to the center.

[0063] In the power inspection application scenario, assume that the robot faces three potential inspection path options:

[0064] Path A: Through densely wooded areas, you may encounter power lines obscured by tree branches.

[0065] Path B: Along open ground, with wires exposed but further away.

[0066] Path C: A near-ground path that passes directly through multiple known damage points.

[0067] Apply the Thought Chain scoring model to these three path options:

[0068] Path A: Low score because tree branches in the wooded area may obstruct vision and make inspection more difficult.

[0069] Path B: The score is medium, with a wide line of sight but requires a longer distance of movement, and the efficiency is not the highest.

[0070] Path C: High score, although there may be obstacles close to the ground, because the path passes directly through multiple known damage points, problems can be quickly inspected and located.

[0071] Ultimately, the navigation decision system may choose path C as the optimal inspection route because it is directly related to the focus of the current task, although it may need to be adjusted in time to avoid ground obstacles. This decision reflects the strategy of optimizing actual operations by using the thinking chain output by the large language model.

[0072] The robot positioning and navigation method based on the large language model thinking chain proposed in an embodiment of the present invention introduces a "thinking chain" mechanism, so that the navigation system can not only understand the direct intention in the instructions, but also perform in-depth logical reasoning and environmental adaptation, thereby providing more accurate navigation support. By combining semantic understanding with the traditional navigation system, it can provide more accurate and efficient navigation solutions in complex or unknown environments.

[0073] Next, a robot positioning and navigation device based on a large language model thinking chain proposed in accordance with an embodiment of the present invention will be described with reference to the accompanying drawings.

[0074] Figure 2 This is an example diagram of a robot positioning and navigation device based on a large language model thinking chain according to an embodiment of the present invention.

[0075] like Figure 2 As shown, the robot positioning and navigation device 10 based on the large language model thinking chain includes:

[0076] The data acquisition module 100 is used to obtain target instructions, collect the current position of the robot, and image information of the current position.

[0077] The thought chain generation module 200 is used to use the visual language model to perform semantic analysis on the target instruction and the image information of the current position to generate a navigation thought chain.

[0078] The navigation decision module 300 is used to analyze the navigation thought chain using the language scoring model, determine the next position of the robot, and generate the navigation route of the robot according to the next position and the current position.

[0079] Optionally, in one embodiment of the present invention, the data acquisition module specifically includes:

[0080] The GPS system installed on the robot is used to collect the current position of the robot;

[0081] The camera installed on the robot is used to collect image information of the robot's current position.

[0082] Optionally, in one embodiment of the present invention, the thinking chain generation module is specifically used to analyze the image information of the current position according to the semantics of the target instruction, obtain multiple location point information in the image information, and generate a navigation thinking chain consisting of multiple paths to implement the target instruction based on the multiple location point information.

[0083] Optionally, in one embodiment of the present invention, the navigation decision module is specifically used to score multiple paths in the navigation thinking chain, select the path with the highest score as the next position of the robot, and generate the navigation route of the robot according to the next position and the current position.

[0084] Optionally, in one embodiment of the present invention, the formula for scoring multiple paths in the navigation thought chain is:

[0085] S(i)=\sum_{k=1}^{K}w_k\cdot\text{LM}(c_k|\text{Context})

[0086] Among them, (S(i)) represents the score of each position i, w_k is the feature weight, (text{LM}) represents the call of the large language model, (c_k) is the associated semantic feature, and (text{Context}) is the current task context.

[0087] Optionally, in one embodiment of the present invention, the formula for selecting the path with the highest score among multiple scores as the next position of the robot is:

[0088] text{NextStep}=\arg\max_{s\in\text{Steps}}\left(\alpha\cdot S(s)-\beta\cdot\text{Dist}(s)\right)

[0089] Among them, (text{NextStep}) is the next action selected, (alpha\) and (beta\) are weight coefficients used to adjust the influence of the thought chain score S(s) and the distance cost (text{Dist}(s)), and (s) represents a possible navigation step.

[0090] It should be noted that the aforementioned explanation of the embodiment of the robot positioning and navigation method based on the large language model thinking chain is also applicable to the robot positioning and navigation device based on the large language model thinking chain of this embodiment, and will not be repeated here.

[0091] The robot positioning and navigation device based on the large language model thinking chain proposed in the embodiment of the present invention optimizes the robot's navigation ability through in-depth semantic reasoning, and can achieve accurate positioning and navigation in complex or unknown environments by combining the thinking chain generated by the large language model with the real-time perception data of the environment.

[0092] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0093] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0094] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

Claims

1. A robot positioning and navigation method based on a large language model thinking chain, characterized in that: The following steps are involved: Obtaining target instructions, collecting the current position of the robot, and image information of the current position; Using a visual language model to perform semantic analysis on the target instruction and the image information of the current position to generate a navigation thought chain; The navigation thought chain is analyzed by using a language scoring model to determine the next position of the robot, and a navigation route of the robot is generated according to the next position and the current position.

2. The method according to claim 1, characterized in that The current position of the robot is collected using the GPS system installed on the robot; and the image information of the current position of the robot is collected using the camera installed on the robot.

3. The method according to claim 1, characterized in that The target instruction and the image information of the current position are semantically analyzed by using a visual language model to generate a navigation thought chain, including: The image information of the current position is analyzed according to the semantics of the target instruction to obtain multiple position point information in the image information, and the navigation thinking chain composed of multiple paths to realize the target instruction is generated according to the multiple position point information.

4. The method according to claim 3, characterized in that The navigation thought chain is analyzed by using a language scoring model to determine the next position of the robot to generate a navigation route of the robot, including: Scoring the multiple paths in the navigation thinking chain, selecting the path with the highest score among the multiple scores as the next position of the robot, and generating the navigation route of the robot according to the next position and the current position.

5. The method according to claim 4, characterized in that The formula for scoring multiple paths in the navigation thinking chain is: S(i)=\sum_{k=1}^{K}w_k\cdot\text{LM}(c_k|\text{Context}) Among them, (S(i)) represents the score of each position i, w_k is the feature weight, (text{LM}) represents the call of the large language model, (c_k) is the associated semantic feature, and (text{Context}) is the current task context.

6. The method according to claim 5, characterized in that The formula for selecting the path with the highest score among multiple scores as the next position of the robot is: text{NextStep}=\arg\max_{s\in\text{Steps}}\left(\alpha\cdot S(s)-\beta\cdot\text{Dist}(s)\right) Among them, (text{NextStep}) is the next action selected, (alpha\) and (beta\) are weight coefficients used to adjust the influence of the thought chain score S(s) and the distance cost (text{Dist}(s)), and (s) represents a possible navigation step.

7. A robot positioning and navigation device based on a large language model thinking chain, characterized in that: include: A data acquisition module is used to obtain target instructions, collect the current position of the robot, and image information of the current position; A thought chain generation module, used for performing semantic analysis on the target instruction and the image information of the current position by using a visual language model to generate a navigation thought chain; The navigation decision module is used to analyze the navigation thought chain by using a language scoring model, determine the next position of the robot, and generate a navigation route of the robot according to the next position and the current position.

8. The device according to claim 7, characterized in that The data acquisition module specifically includes: The GPS system installed on the robot is used to collect the current position of the robot; The camera installed on the robot is used to collect image information of the robot's current position.

9. The device according to claim 7, characterized in that The thought chain generation module is specifically used to analyze the image information of the current position according to the semantics of the target instruction, obtain multiple location point information in the image information, and generate the navigation thought chain composed of multiple paths to realize the target instruction according to the multiple location point information.

10. The device according to claim 9, characterized in that The navigation decision module is specifically used to score multiple paths in the navigation thinking chain, select the path with the highest score as the next position of the robot, and generate a navigation route of the robot according to the next position and the current position.

Images