An air-ground collaborative adaptive tracking method

Abstract

The present invention discloses an air-ground collaborative adaptive tracking method. The method comprises: a large language model receives a natural language prompt word sent by a target user, and sends the natural language prompt word to an aerial robot and a ground robot, wherein the natural language prompt word is generated by the target user discovering the target robot; an observation encoder obtains target environment information of a target frequency determined by the aerial robot and the ground robot based on the natural language prompt word, and transmits the target environment information of the target frequency to a large language model module and a reinforcement learning module respectively. The present invention solves the technical problem that the existing air-ground collaborative tracking method does not fully utilize the complementary advantages of the aerial and ground robots, does not improve the intelligence of the air-ground robots, and makes the air-ground robots unable to carry out complex work in a dynamic environment, resulting in inaccurate tracking of the air-ground collaborative tracking method.

Description

Technical Field

[0001] The present invention relates to the field of heterogeneous multi-robot collaborative technology enabled by a large language model, and in particular to an air-ground collaborative adaptive tracking method. Background Art

[0002] Robots, hailed as the "crown jewel of manufacturing," are gaining increasing attention from technologically advanced nations worldwide, characterized by intelligence and autonomy. However, the overall functionality, task execution efficiency, and fault tolerance of individual robots are relatively limited, making them incapable of adapting to complex tasks and dynamic environments. Consequently, many researchers are turning their attention to multi-robot swarm systems, which offer greater task execution efficiency and environmental adaptability. A heterogeneous multi-robot collaborative system, comprised of aerial and ground robots, is a typical example of a multi-robot swarm system. It combines the complementary strengths of aerial and ground swarms, extending the two-dimensional motion plane of ground robots to three-dimensional space. By leveraging the agility and wide field of view of aerial robots with the high-load and robust computational reasoning capabilities of ground robots, it demonstrates significant advantages in task execution efficiency, environmental adaptability, and fault tolerance. It has successfully replaced humans in tasks such as search and rescue, autonomous exploration, and patrol and maintenance in hazardous, dirty, and extreme temperature environments.

[0003] However, facing complex tasks and dynamic environments, current air-ground collaborative systems still face challenges such as low intelligence and weak autonomy. Therefore, applying artificial intelligence (AI) and natural crowd intelligence theory to air-ground collaborative systems is an effective solution for enhancing both individual and collective intelligence. Although artificial intelligence (AI), represented by deep learning, has been widely used in robotic perception, decision-making, and control in recent years, air-ground robots perform complex tasks in dynamic environments, requiring the development of more intelligent decision-making methods to cope with these dynamic environments. Furthermore, the long design and development cycle for air-ground collaborative algorithms requires significant time investment from researchers. In particular, directly deploying a designed air-ground collaborative algorithm on the physical body of an air-ground robot consumes significant time for debugging and testing, and if the collaborative algorithm is not robust, it may damage the physical body. Therefore, many researchers currently choose to test algorithm performance in simulated environments before deploying the algorithm in real-world environments. However, there is currently no fully designed simulation environment specifically for air-ground collaboration, requiring researchers to design and consider essential steps in the air-ground collaboration process, such as the robot's structural design, communication algorithms, and perception algorithms.

[0004] Numerous studies have been conducted on air-ground collaboration. The earliest work, based on classical control theory, focused on tracking targets in static environments from a planning perspective. These studies often overlooked the information sources of the perception layer, relying instead on pre-processed data. Furthermore, they employed the same model for both air and ground robots, employing a homogeneous strategy to address the heterogeneous air-ground robot collaboration problem. However, this approach fails to fully exploit the complementary strengths of air and ground robots. Multi-Agent Deep Reinforcement Learning (MADRL) has long been widely used in robot collaboration, particularly in decision-making and planning tasks. To address environmental instability, most work has followed a centralized training distributed execution (CTDE) framework. This approach requires global information to be acquired through real-time communication during training. However, this is uncommon in the real world due to high bandwidth requirements. Furthermore, each agent's decision-making, based solely on local observations, limits the ability of multiple agents to collaborate. Recent research aims to improve collaboration efficiency through effective communication between agents. These efforts focus on reducing communication resource consumption during collaboration, primarily by selecting communication partners, learning communication content, and managing communication frequency. Although existing work has made many efforts to reduce communication bandwidth usage by managing communication frequencies, it has not studied how to enhance the intelligence of air-ground robots to carry out complex tasks in dynamic environments. Summary of the Invention

[0005] An embodiment of the present invention provides an air-ground collaborative adaptive tracking method, which at least solves the technical problem that the existing air-ground collaborative tracking method does not fully utilize the complementary advantages of air and ground robots, does not improve the intelligence of air-ground robots, makes air-ground robots unable to carry out complex work in a dynamic environment, and causes inaccurate tracking by the air-ground collaborative tracking method.

[0006] According to one aspect of an embodiment of the present invention, an air-ground collaborative adaptive tracking method is provided. The method may include: a large language model receiving a natural language prompt word sent by a target user, and sending the natural language prompt word to an aerial robot and a ground robot, wherein the natural language prompt word is generated by the target user to discover a target robot; an observation encoder obtaining target environment information of a target frequency determined by the aerial robot and the ground robot based on the natural language prompt word, and transmitting the target environment information of the target frequency to a large language model module and a reinforcement learning module, respectively; the large language model obtaining a planning instruction based on the natural language prompt word and the environmental feature information of the target frequency, and transmitting the planning instruction to the reinforcement learning module; the reinforcement learning module obtaining tracking information based on the environmental feature information of the target frequency and the planning instruction, and sending the tracking information to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information.

[0007] Optionally, before the aerial robot and the ground robot obtain the fixed-frequency target information corresponding to the prompt words based on natural language, the method also includes: the observation encoder obtains first environmental information of the aerial robot and second environmental information of the ground robot, wherein the first environmental information is the sum of environmental information collected in real time by the aerial robot and environmental information sent by the ground robot that is different from the environmental information collected in real time by the aerial robot, and the second environmental information is the sum of environmental information collected in real time by the ground robot and environmental information sent by the aerial robot that is different from the environmental information collected in real time by the ground robot; the observation encoder fuses the first environmental information and the second environmental information to obtain fused environmental information; the observation encoder learns the fused environmental information to obtain target environmental information of the target frequency.

[0008] Optionally, the observation encoder fuses the first environment information and the second environment information to obtain the fused environment information in the following process: the observation encoder determines the sum of the first environment information and the second environment information and environment information different from the first environment information as the fused environment information.

[0009] Optionally, the environmental feature information of the target frequency is represented by a one-hot vector.

[0010] Optionally, the aerial robot and the ground robot track the target robot based on the tracking information, including: the aerial robot uses a continuous motion space to track according to the tracking information, wherein the continuous motion space includes direction and speed, and the speed is a three-dimensional vector, and the speed includes: angular velocity around the yaw axis and linear velocity along the x and y axes; the ground robot uses a McNamee wheel to track based on the tracking information.

[0011] Optionally, the tracking information is to control the aerial robot and the ground robot to avoid collision with target environment information of a target frequency, and the expression of the collision reward is:

[0012]

[0013] in, in When it is A, For aerial robots, in When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

[0014] Beneficial effects of the present invention:

[0015] The present invention proposes an air-ground collaborative adaptive tracking method. By drawing on the human thought and decision-making process during the air-ground collaborative process, the method utilizes a large language model to manage the high-level decision-making of the air-ground robot, and reinforcement learning to manage the low-level planning of the air-ground robot, enabling the air-ground robot to make autonomous decisions. Simultaneously, an observation encoder selects a fixed time to transmit the environmental state to the large language model module, and transmits observation information to the reinforcement learning module. This solves the technical problem of existing air-ground collaborative tracking methods, which fail to fully utilize the complementary advantages of air and ground robots, fail to improve the intelligence of air-ground robots, and thus prevent air-ground robots from performing complex tasks in dynamic environments, leading to inaccurate tracking by air-ground collaborative tracking methods. The method achieves the technical effect of designing a complex dynamic environment similar to the real environment, with a large language model responsible for the high-level decision-making of the air-ground robot and reinforcement learning responsible for the low-level planning of the air-ground robot, enabling the air-ground robot to make autonomous decisions and ensuring accurate tracking by the air-ground collaborative tracking method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0017] Figure 1 is a flow chart of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention;

[0018] Figure 2 is a framework diagram of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention;

[0019] Figure 3 This is a diagram of a simulation environment construction framework according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0022] Example 1

[0023] According to an embodiment of the present invention, an air-ground collaborative adaptive tracking method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system containing at least one set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 1 is a flow chart of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0025] In step S101 , the large language model receives a natural language prompt word sent by a target user, and sends the natural language prompt word to the aerial robot and the ground robot, wherein the natural language prompt word is generated when the target user discovers the target robot.

[0026] In the technical solution provided in the above step S101 of the present invention, Figure 2 is a framework diagram of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention, such as Figure 2As shown, when the target user finds the target robot, the target user sends a natural language prompt word to the large language model module, the ground robot and the aerial robot, where the target robot is a ground robot.

[0027] In step S102, the observation encoder obtains target environment information of target frequencies determined by the aerial robot and the ground robot based on the prompt words of the natural language, and transmits the target environment information of the target frequencies to the large language model module and the reinforcement learning module respectively.

[0028] In the technical solution provided in step S102 of the present invention, the aerial robot and the ground robot are provided with target environment information of the target frequency corresponding to the prompt word of the natural language. The target frequency can be 5 seconds, and the target environment information can be obstacles, space, etc. Figure 2 As shown, the observation encoder transmits the environmental feature information that needs to be transmitted at a specific time to the large language model module and the reinforcement learning module respectively. The observation encoder here transmits the environmental feature information that needs to be transmitted at a specific time, and does not transmit the environmental feature information that does not need to be transmitted, which plays a filtering role.

[0029] Since the target environment information is changeable, the present invention simulates the environment information. Figure 3 This is a simulation environment construction framework diagram according to an embodiment of the present invention. Figure 3 As shown, the design of the air-ground collaborative simulation environment must reflect the heterogeneity of air-ground robots and the communication process between them. The heterogeneity of air-ground robots includes heterogeneity in perception, motion, and energy. Aerial robots perceive the environment from a vertical bird's-eye view, while ground robots observe the environment from a first-person perspective. In terms of motion, aerial robots are often faster than ground robots and can fly vertically. In terms of energy, aerial robots have limited flight time due to their limited payload and energy, while ground robots can operate for extended periods of time. The communication process between air-ground robots is a crucial component of air-ground collaboration; collaboration cannot be achieved without considering this process. However, communication is not real-time. Due to the surrounding environment, communication between air-ground robots often has a certain delay, and the impact of this delay needs to be considered. After considering these key issues, the simulation environment was constructed using the Pybullet physics engine, allowing users to select air-ground robots and adhering to the OpenAI Gym specification. A large language model was integrated for high-level planning of the air-ground robots, while reinforcement learning was used for low-level control. At the same time, the delay process of the air-ground robot communication process is modeled to provide an interface for exploring the impact of delay.

[0030] In step S103 , the large language model obtains a planning instruction based on the natural language prompt word and the environmental feature information of the target frequency, and transmits the planning instruction to the reinforcement learning module.

[0031] In the technical solution provided in step S103 of the present invention, if Figure 2 As shown in the figure, the large language model module obtains planning instructions based on the natural language prompt words and the environmental feature information of the target frequency. The planning instructions are as follows: Figure 2 In the reward model, the large language model module passes the planning instructions to the reinforcement learning module.

[0032] In step S104, the reinforcement learning module obtains tracking information based on the environmental feature information of the target frequency and the planning instructions, and sends the tracking information to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information.

[0033] In the technical solution provided in step S104 of the present invention, if Figure 2 As shown, the reinforcement learning module obtains tracking information based on the environmental feature information and planning instructions of the target frequency, and sends the tracking information to the ground robot and the aerial robot, so that the aerial robot and the ground robot track the target robot according to the tracking information.

[0034] The above method of this embodiment is further introduced below.

[0035] As an optional embodiment, in step S102, before the aerial robot and the ground robot obtain the fixed-frequency target information corresponding to the prompt words based on natural language, the method also includes: the observation encoder obtains the first environmental information of the aerial robot and the second environmental information of the ground robot, wherein the first environmental information is the sum of the environmental information collected in real time by the aerial robot and the environmental information sent by the ground robot that is different from the environmental information collected in real time by the aerial robot, and the second environmental information is the sum of the environmental information collected in real time by the ground robot and the environmental information sent by the aerial robot that is different from the environmental information collected in real time by the ground robot; the observation encoder fuses the first environmental information and the second environmental information to obtain fused environmental information; the observation encoder learns the fused environmental information to obtain the target environmental information of the target frequency.

[0036] In this embodiment, the fusion environment information is studied to obtain the target environment information of the target frequency, that is, to extract the target environment information of a fixed time. Since the environment information is changeable, the present invention simulates the environment information. Figure 3 This is a simulation environment construction framework diagram according to an embodiment of the present invention. Figure 3As shown, the design of the air-ground collaborative simulation environment must reflect the heterogeneity of air-ground robots and the communication process between them. The heterogeneity of air-ground robots includes heterogeneity in perception, motion, and energy. Aerial robots perceive the environment from a vertical bird's-eye view, while ground robots observe the environment from a first-person perspective. In terms of motion, aerial robots are often faster than ground robots and can fly vertically. In terms of energy, aerial robots have limited flight time due to their limited payload and energy, while ground robots can operate for extended periods of time. The communication process between air-ground robots is a crucial component of air-ground collaboration; collaboration cannot be achieved without considering this process. However, communication is not real-time. Due to the surrounding environment, communication between air-ground robots often has a certain delay, and the impact of this delay needs to be considered. After considering these key issues, the simulation environment was constructed using the Pybullet physics engine, allowing users to select air-ground robots and adhering to the OpenAI Gym specification. A large language model was integrated for high-level planning of the air-ground robots, while reinforcement learning was used for low-level control. At the same time, the delay process of the air-ground robot communication process is modeled to provide an interface for exploring the impact of delay.

[0037] As an optional implementation method, the observation encoder fuses the first environmental information and the second environmental information to obtain the fused environmental information. The process is: the observation encoder determines the sum of the first environmental information and the second environmental information and the environmental information that is different from the first environmental information as the fused environmental information.

[0038] In this embodiment, the process of fusing the first environment information and the second environment information is as follows: aggregating all the first environment information and the second environment information and deleting the same information. The fused environment information is learned to obtain the target environment information of the target frequency, that is, to extract the target environment information of the fixed time.

[0039] In step S102 , the environmental feature information of the target frequency is represented by a one-hot vector.

[0040] For example, obstacles of the same type are represented by one character, and environmental information of the same type is represented by one character table.

[0041] As an optional embodiment, in step S104, the aerial robot and the ground robot track the target robot based on the tracking information, including: the aerial robot uses a continuous motion space to track according to the tracking information, wherein the continuous motion space includes direction and speed, and the speed is a three-dimensional vector, and the speed includes: angular velocity around the yaw axis and linear velocity along the x and y axes; the ground robot uses a McNamee wheel to track based on the tracking information.

[0042] In this embodiment, when the aerial robot tracks the target robot, the aerial robot Continuous action space , the speed of the continuous part It is a three-dimensional vector, including the angular velocity around the yaw axis and the linear velocity along the x and y axes. For example, the aerial robot runs eastward at 10 meters per second. When the ground robot tracks the target robot, the ground robot The McNamee wheels allow for free change of direction. For example, a ground robot can run horizontally or vertically by changing the steering wheel.

[0043] As an optional embodiment, in step S104, the tracking information is to control the aerial robot and the ground robot to avoid collision with the target environment information of the target frequency. The expression of the collision reward is:

[0044]

[0045] in, in When it is A, For aerial robots, in When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

[0046]

[0047] in, in When it is A, For aerial robots, in When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

[0048] In this embodiment, the reinforcement learning module is responsible for the underlying control of the aerial robot and the ground robot. Specifically, each robot uses the Proximal Policy Optimization algorithm to ensure that the aerial robot and the ground robot do not collide with obstacles. Specifically defined as:

[0049]

[0050] When an aerial robot or ground robot collides with an obstacle Less than the safe distance Punishment will be given.

[0051] In an embodiment of the present invention, a natural language prompt word sent by a target user is received through a large language model, and the natural language prompt word is sent to the aerial robot and the ground robot, wherein the natural language prompt word is generated by the target user discovering the target robot; the observation encoder obtains the target environment information of the target frequency determined by the aerial robot and the ground robot based on the natural language prompt word, and transmits the target environment information of the target frequency to the large language model module and the reinforcement learning module respectively; the large language model obtains a planning instruction based on the natural language prompt word and the environmental feature information of the target frequency, and transmits the planning instruction to the reinforcement learning module; the reinforcement learning module obtains the target environment information of the target frequency and the planning instruction based on the environmental feature information of the target frequency. The tracking information is obtained and sent to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information, which solves the technical problem that the existing air-ground collaborative tracking method does not fully utilize the complementary advantages of the aerial and ground robots and does not improve the intelligence of the air-ground robots, so that the air-ground robots cannot carry out complex work in a dynamic environment, resulting in inaccurate tracking of the air-ground collaborative tracking method. The design achieves a complex dynamic environment similar to the real environment, the large language model is responsible for the high-level decision-making of the air-ground robot, and the reinforcement learning is responsible for the low-level planning of the air-ground robot, which enables the air-ground robot to make autonomous decisions and enables the air-ground collaborative tracking method to track accurately.

[0052] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0053] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0054] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0055] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0056] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0057] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An air-ground collaborative adaptive tracking method, characterized in that: include: The large language model receives the natural language prompt words sent by the target user and sends the natural language prompt words to the aerial robot and the ground robot, wherein the natural language prompt words are generated by the target user when discovering the target robot; The observation encoder obtains target environment information of the target frequency determined by the aerial robot and the ground robot based on the natural language prompt words, and transmits the target environment information of the target frequency to the large language model module and the reinforcement learning module respectively; The large language model obtains planning instructions based on natural language prompt words and environmental feature information of target frequency, and transmits the planning instructions to the reinforcement learning module; The reinforcement learning module obtains tracking information based on the environmental feature information of the target frequency and the planning instructions, and sends the tracking information to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information.

2. The method according to claim 1, characterized in that Before the aerial robot and the ground robot receive fixed-frequency target information corresponding to the prompt word in natural language, the method further includes: The observation encoder obtains first environmental information of the aerial robot and second environmental information of the ground robot, wherein the first environmental information is the sum of environmental information collected in real time by the aerial robot and environmental information sent by the ground robot that is different from the environmental information collected in real time by the aerial robot, and the second environmental information is the sum of environmental information collected in real time by the ground robot and environmental information sent by the aerial robot that is different from the environmental information collected in real time by the ground robot; The observation encoder fuses the first environment information and the second environment information to obtain fused environment information; The observation encoder integrates the environmental information for learning and obtains the target environmental information of the target frequency.

3. The method according to claim 2, characterized in that The observation encoder fuses the first environment information and the second environment information to obtain the fused environment information in the following process: The observation encoder determines the sum of the first environmental information and the second environmental information different from the first environmental information as the fused environmental information.

4. The method according to claim 1, wherein The environmental feature information of the target frequency is represented by a one-hot vector.

5. The method according to claim 1, wherein The aerial robot and the ground robot track the target robot based on the tracking information, including: The aerial robot uses a continuous motion space for tracking based on the tracking information. The continuous motion space includes direction and velocity. Velocity is a three-dimensional vector that includes the angular velocity around the yaw axis and the linear velocity along the x and y axes. The ground robot uses McNamee wheels for tracking based on the tracking information.

6. The method according to claim 1, characterized in that The tracking information is used to control the aerial robot and the ground robot to avoid collision with the target environment information of the target frequency. The expression of the collision reward is: in, in When it is A, For aerial robots, in When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

7. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 6 when running.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

Images