Air-ground cooperative adaptive tracking method

Abstract

The invention discloses an air-ground cooperative adaptive tracking method. The method comprises the steps that a big language model receives prompt words of a natural language sent by a target user and sends the prompt words of the natural language to an aerial robot and a ground robot, and the prompt words of the natural language are generated when the target user discovers the target robot; the observation encoder obtains target environment information of a target frequency determined by the aerial robot and the ground robot based on prompt words of a natural language, and transmits the target environment information of the target frequency to the large language model module and the reinforcement learning module. The technical problem that an existing air-ground cooperative tracking method does not make full use of the complementary advantages of an air robot and a ground robot, the intelligence of the air-ground robot is not improved, the air-ground robot cannot carry out complex work in a dynamic environment, and the air-ground cooperative tracking method is inaccurate in tracking is solved.

Description

Technical Field

[0001] The present invention relates to the field of heterogeneous multi-robot collaborative technology enabled by a large language model, and in particular to an air-ground collaborative adaptive tracking method. Background Art

[0002] Robots are known as the "crown jewel of the manufacturing industry". Robots with intelligence and autonomy as their main features are receiving more and more attention from the world's technological powers. However, the overall functions, task execution efficiency and fault tolerance of a single robot are relatively limited, making it difficult to adapt to complex tasks and dynamic environments. Therefore, many researchers have begun to turn their attention to multi-robot cluster systems with higher task execution efficiency and environmental adaptability. The heterogeneous multi-robot collaborative system composed of aerial robots and ground robots is a typical multi-robot cluster system. It combines the complementary advantages of aerial groups and ground groups, expands the two-dimensional plane of ground robot movement to three-dimensional space, and utilizes the advantages of aerial robots' flexibility and wide viewing angles, combined with the high load and strong computing and reasoning capabilities of ground robots. It has shown strong advantages in task execution efficiency, environmental adaptability and fault tolerance, and has successfully replaced humans in dangerous, dirty and extreme temperature environments to complete search and rescue, autonomous exploration, patrol and maintenance tasks.

[0003] However, facing complex tasks and dynamic environments, the current air-ground collaborative system still faces challenges such as low intelligence and weak autonomy. Based on this, applying artificial intelligence technology and natural group intelligence theory to air-ground collaborative systems is an effective solution to improve individual intelligence and group intelligence. Although artificial intelligence (AI) technology represented by deep learning has been widely used in the fields of robot perception, decision-making and control in recent years, since air-ground robots perform responsible tasks in dynamic environments, it is necessary to design more intelligent decision-making methods for air-ground robots to cope with dynamic environments. At the same time, due to the long design and development cycle of air-ground collaborative algorithms, researchers need to invest a lot of time costs, especially directly deploying the designed air-ground collaborative algorithm on the physical entity of the air-ground robot, which will take a lot of time to debug and test, and if the collaborative algorithm is not robust, it may damage the physical entity. Therefore, a large number of researchers currently choose to test the algorithm performance in a simulated environment before deploying the algorithm in a real environment. However, there is currently no simulation environment that is fully oriented to air-ground collaboration, and researchers need to design and consider the structural design, communication algorithm and perception algorithm of air-ground robots and other necessary steps in the air-ground collaboration process.

[0004] There are many research works on air-ground collaboration. The earliest works started from the perspective of planning and tracked targets in a static environment based on classical control theory. These studies often ignored the information source of the perception layer and relied on pre-processed data. And they set the same model for air-ground robots, that is, they adopted isomorphic strategies to solve the problem of air-ground heterogeneous robot collaboration. However, this approach does not fully utilize the complementary advantages of air and ground robots. Multi-Agent Deep Reinforcement Learning (MADRL) has been widely used in robot collaboration for a long time, especially in decision-making and planning tasks. To solve the problem of environmental instability, most works follow the centralized training distributed execution (CTDE) framework. This approach requires global information to be obtained through real-time communication during training. However, this situation is not common in the real world due to high bandwidth transmission requirements. In addition, the decision-making of each agent based only on local observations during execution also limits the ability of multiple agents to cooperate. Recent studies aim to improve the efficiency of cooperation through effective communication between agents. They focus on reducing the consumption of communication resources during the collaboration process, mainly by selecting communication objects, learning communication content, and managing the frequency of communication. Although existing work has made many efforts in reducing communication bandwidth usage by managing communication frequencies, it has not studied how to enhance the intelligence of air-ground robots to carry out complex tasks in dynamic environments. Summary of the invention

[0005] An embodiment of the present invention provides an air-ground collaborative adaptive tracking method, which at least solves the technical problem that the existing air-ground collaborative tracking method does not fully utilize the complementary advantages of air and ground robots, does not improve the intelligence of air-ground robots, makes the air-ground robots unable to carry out complex work in a dynamic environment, and causes inaccurate tracking of the air-ground collaborative tracking method.

[0006] According to one aspect of an embodiment of the present invention, an air-ground collaborative adaptive tracking method is provided. The method may include: a large language model receives a natural language prompt word sent by a target user, and sends the natural language prompt word to an aerial robot and a ground robot, wherein the natural language prompt word is generated by the target user discovering the target robot; an observation encoder obtains target environment information of a target frequency determined by the aerial robot and the ground robot based on the natural language prompt word, and transmits the target environment information of the target frequency to a large language model module and a reinforcement learning module respectively; the large language model obtains a planning instruction based on the natural language prompt word and the environmental feature information of the target frequency, and transmits the planning instruction to the reinforcement learning module; the reinforcement learning module obtains tracking information based on the environmental feature information of the target frequency and the planning instruction, and sends the tracking information to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information.

[0007] Optionally, before the aerial robot and the ground robot provide target information of a fixed frequency corresponding to prompt words based on natural language, the method further includes: an observation encoder obtains first environmental information of the aerial robot and second environmental information of the ground robot, wherein the first environmental information is the sum of environmental information collected in real time by the aerial robot and environmental information sent by the ground robot that is different from the environmental information collected in real time by the aerial robot, and the second environmental information is the sum of environmental information collected in real time by the ground robot and environmental information sent by the aerial robot that is different from the environmental information collected in real time by the ground robot; the observation encoder fuses the first environmental information and the second environmental information to obtain fused environmental information; the observation encoder learns the fused environmental information to obtain target environmental information of the target frequency.

[0008] Optionally, the observation encoder fuses the first environment information and the second environment information to obtain the fused environment information in the following process: the observation encoder determines the sum of the first environment information and the environment information different from the first environment information as the fused environment information.

[0009] Optionally, the environmental feature information of the target frequency is represented by a one-hot vector.

[0010] Optionally, the aerial robot and the ground robot track the target robot based on the tracking information, including: the aerial robot uses a continuous motion space to track according to the tracking information, wherein the continuous motion space includes direction and speed, and the speed is a three-dimensional vector, and the speed includes: angular velocity around the yaw axis, and linear velocity along the x and y axes; the ground robot uses a McNamee wheel to track based on the tracking information.

[0011] Optionally, the tracking information is to control the aerial robot and the ground robot not to collide with the target environment information of the target frequency, and the expression of the collision reward is:

[0012] in, In When A, For aerial robots, In When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

[0013] Beneficial effects of the present invention: The present invention proposes an air-ground collaborative adaptive tracking method, which can make the air-ground robot make autonomous decisions by using a large language model to make high-level decisions of the air-ground robot and reinforcing learning to make low-level planning of the air-ground robot in the process of air-ground collaboration. At the same time, the observation encoder will select a fixed time to transmit the environmental state to the large language model module, and transmit the observation information to the reinforcement learning module, which solves the technical problem that the existing air-ground collaborative tracking method does not fully utilize the complementary advantages of air and ground robots and does not improve the intelligence of air-ground robots, so that the air-ground robot cannot carry out complex work in a dynamic environment, resulting in inaccurate tracking of the air-ground collaborative tracking method, and achieves the technical effect that the large language model is responsible for the high-level decision-making of the air-ground robot and the reinforcement learning is responsible for the low-level planning of the air-ground robot by designing a complex dynamic environment like the real environment, so that the air-ground robot can make autonomous decisions and the air-ground collaborative tracking method can track accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention; Figure 2 is a framework diagram of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention; Figure 3 It is a simulation environment construction framework diagram according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0017] Example 1 According to an embodiment of the present invention, an air-ground collaborative adaptive tracking method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0018] Figure 1 is a flow chart of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps: Step S101, the large language model receives the natural language prompt words sent by the target user, and sends the natural language prompt words to the aerial robot and the ground robot, wherein the natural language prompt words are generated by the target user discovering the target robot.

[0019] In the technical solution provided in the above step S101 of the present invention, Figure 2 is a framework diagram of an air-ground collaborative adaptive tracking method according to an embodiment of the present invention, such as Figure 2 As shown, when the target user finds the target robot, the target user sends a natural language prompt word to the large language model module and the ground robot and the aerial robot, wherein the target robot is a ground robot.

[0020] Step S102, the observation encoder obtains the target environment information of the target frequency determined by the aerial robot and the ground robot based on the prompt words of the natural language, and transmits the target environment information of the target frequency to the large language model module and the reinforcement learning module respectively.

[0021] In the technical solution provided in step S102 of the present invention, the aerial robot and the ground robot receive target environment information of the target frequency corresponding to the prompt word of the natural language. The target frequency may be 5 seconds, and the target environment information may be obstacles, space, etc., such as Figure 2 As shown, the observation encoder transmits the environmental feature information that needs to be transmitted at a specific time to the large language model module and the reinforcement learning module respectively. The observation encoder here transmits the environmental feature information that needs to be transmitted at a specific time, and does not transmit the environmental feature information that does not need to be transmitted, thereby playing a filtering role.

[0022] Since the target environment information is changeable, the present invention simulates the environment information. Figure 3 1 is a diagram of a simulation environment construction framework according to an embodiment of the present invention. Figure 3 As shown in the figure, the design of the air-ground collaborative simulation environment needs to reflect the heterogeneity of air-ground robots and the communication process between air-ground robots. The heterogeneity of air-ground robots includes heterogeneity in perception, motion, and energy. The perception mode of aerial robots is vertical bird's-eye view, while ground robots observe the environment from a first-person perspective. In terms of motion, aerial robots are often faster than ground robots and can fly vertically. In terms of energy, aerial robots have limited flight time due to limited load and energy, while ground robots can work for a long time. The communication process of air-ground robots is an important part of air-ground collaboration. If the communication process is not considered, it cannot be called collaboration. However, communication is not real-time. Considering the state of the surrounding environment, there is often a certain delay in the communication between air-ground robots, and the impact of this delay needs to be considered. After considering the above key issues, the Pybullet physics engine is used to build a simulation environment, which supports users to import selectable air-ground robots independently and follows the OpenAI Gym specification. The large language model is connected to be responsible for the high-level planning of the air-ground robot, and reinforcement learning is used to be responsible for the underlying control of the air-ground robot. At the same time, the delay process of the air-to-ground robot communication process is modeled to provide an interface for exploring the impact of delay.

[0023] Step S103: The large language model obtains a planning instruction based on the prompt words of the natural language and the environmental feature information of the target frequency, and transmits the planning instruction to the reinforcement learning module.

[0024] In the technical solution provided in step S103 of the present invention, if Figure 2As shown in the figure, the large language model module obtains planning instructions based on the natural language prompt words and the environmental feature information of the target frequency. The planning instructions are as follows: Figure 2 Based on the reward model in , the large language model module passes the planning instructions to the reinforcement learning module.

[0025] Step S104: The reinforcement learning module obtains tracking information based on the environmental feature information of the target frequency and the planning instructions, and sends the tracking information to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information.

[0026] In the technical solution provided in step S104 of the present invention, if Figure 2 As shown, the reinforcement learning module obtains tracking information according to the environmental feature information and planning instructions of the target frequency, and the reinforcement learning module sends the tracking information to the ground robot and the aerial robot, so that the aerial robot and the ground robot track the target robot according to the tracking information.

[0027] The above method of this embodiment is further introduced below.

[0028] As an optional implementation method, step S102, before the aerial robot and the ground robot receive the fixed-frequency target information corresponding to the prompt words based on natural language, the method also includes: an observation encoder obtains first environmental information of the aerial robot and second environmental information of the ground robot, wherein the first environmental information is the sum of environmental information collected in real time by the aerial robot and environmental information sent by the ground robot that is different from the environmental information collected in real time by the aerial robot, and the second environmental information is the sum of environmental information collected in real time by the ground robot and environmental information sent by the aerial robot that is different from the environmental information collected in real time by the ground robot; the observation encoder fuses the first environmental information and the second environmental information to obtain fused environmental information; the observation encoder learns the fused environmental information to obtain target environmental information of the target frequency.

[0029] In this embodiment, the fused environmental information is learned to obtain the target environmental information of the target frequency, that is, to extract the target environmental information of a fixed time. Since the environmental information is changeable, the present invention simulates the environmental information. Figure 3 1 is a diagram of a simulation environment construction framework according to an embodiment of the present invention. Figure 3As shown in the figure, the design of the air-ground collaborative simulation environment needs to reflect the heterogeneity of air-ground robots and the communication process between air-ground robots. The heterogeneity of air-ground robots includes heterogeneity in perception, motion, and energy. The perception mode of aerial robots is vertical bird's-eye view, while ground robots observe the environment from a first-person perspective. In terms of motion, aerial robots are often faster than ground robots and can fly vertically. In terms of energy, aerial robots have limited flight time due to limited load and energy, while ground robots can work for a long time. The communication process of air-ground robots is an important part of air-ground collaboration. If the communication process is not considered, it cannot be called collaboration. However, communication is not real-time. Considering the state of the surrounding environment, there is often a certain delay in the communication between air-ground robots, and the impact of this delay needs to be considered. After considering the above key issues, the Pybullet physics engine is used to build a simulation environment, which supports users to import selectable air-ground robots independently and follows the OpenAI Gym specification. The large language model is connected to be responsible for the high-level planning of the air-ground robot, and reinforcement learning is used to be responsible for the underlying control of the air-ground robot. At the same time, the delay process of the air-to-ground robot communication process is modeled to provide an interface for exploring the impact of delay.

[0030] As an optional implementation method, the observation encoder fuses the first environmental information and the second environmental information to obtain the fused environmental information. The process is: the observation encoder determines the sum of the first environmental information and the environmental information that is different from the first environmental information as the fused environmental information.

[0031] In this embodiment, the process of fusing the first environment information and the second environment information is: aggregating all the first environment information and the second environment information, and deleting the same information. The fused environment information is learned to obtain the target environment information of the target frequency, that is, to extract the target environment information of the fixed time. Step S102: The environmental feature information of the target frequency is represented by a one-hot vector.

[0032] For example, obstacles of the same type are represented by one character, and environmental information of the same type is represented by one character table.

[0033] As an optional implementation method, step S104, the aerial robot and the ground robot track the target robot based on the tracking information, including: the aerial robot uses a continuous motion space to track according to the tracking information, wherein the continuous motion space includes direction and speed, and the speed is a three-dimensional vector, and the speed includes: angular velocity around the yaw axis, and linear velocity along the x and y axes; the ground robot uses a McNamee wheel to track based on the tracking information.

[0034] In this embodiment, when the aerial robot tracks the target robot, the aerial robot Continuous action space , the speed of the continuous part is a three-dimensional vector, including the angular velocity around the yaw axis and the linear velocity along the x and y axes. For example, the aerial robot runs eastward at 10 meters per second. When the ground robot tracks the target robot, the ground robot The McNamee wheels allow for free change of direction, for example, a ground robot can run horizontally or vertically, allowing the steering wheel to be changed.

[0035] As an optional embodiment, in step S104, the tracking information is to control the aerial robot and the ground robot not to collide with the target environment information of the target frequency, and the expression of the collision reward is:

[0036] in, In When A, For aerial robots, In When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

[0037]

[0038] in, In When A, For aerial robots, In When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

[0039] In this embodiment, the reinforcement learning module is responsible for the underlying control of the aerial robot and the ground robot. Specifically, each robot uses the proximal policy optimization algorithm to ensure that the aerial robot does not collide with obstacles. The specific definition is:

[0040] When an aerial robot or ground robot collides with an obstacle Less than the safe distance Punishment will be imposed.

[0041] In an embodiment of the present invention, a natural language prompt word sent by a target user is received through a large language model, and the natural language prompt word is sent to an aerial robot and a ground robot, wherein the natural language prompt word is generated by the target user discovering the target robot; an observation encoder obtains target environment information of a target frequency determined by the aerial robot and the ground robot based on the natural language prompt word, and transmits the target environment information of the target frequency to a large language model module and a reinforcement learning module respectively; the large language model obtains a planning instruction based on the natural language prompt word and the environmental feature information of the target frequency, and transmits the planning instruction to the reinforcement learning module; the reinforcement learning module obtains the planning instruction based on the environmental feature information of the target frequency and the planning instruction. The tracking information is obtained and sent to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information, which solves the technical problem that the existing air-ground collaborative tracking method does not fully utilize the complementary advantages of aerial and ground robots and does not improve the intelligence of air-ground robots, making the air-ground robots unable to carry out complex work in a dynamic environment, resulting in inaccurate tracking of the air-ground collaborative tracking method. The complex dynamic environment designed is the same as the real environment. The large language model is responsible for the high-level decision-making of the air-ground robot, and reinforcement learning is responsible for the low-level planning of the air-ground robot, which can enable the air-ground robot to make autonomous decisions, so that the air-ground collaborative tracking method can track accurately.

[0042] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0043] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0044] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0045] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0046] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0047] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An air-ground collaborative adaptive tracking method, characterized in that: include: The large language model receives the natural language prompt words sent by the target user, and sends the natural language prompt words to the aerial robot and the ground robot, wherein the natural language prompt words are generated by the target user discovering the target robot; The observation encoder obtains target environment information of target frequencies determined by the aerial robot and the ground robot based on the prompt words of the natural language, and transmits the target environment information of the target frequencies to the large language model module and the reinforcement learning module respectively; The large language model obtains planning instructions based on the natural language prompt words and the environmental feature information of the target frequency, and transmits the planning instructions to the reinforcement learning module; The reinforcement learning module obtains tracking information based on the environmental feature information and planning instructions of the target frequency, and sends the tracking information to the aerial robot and the ground robot, wherein the aerial robot and the ground robot track the target robot based on the tracking information.

2. The method according to claim 1, characterized in that Before the aerial robot and the ground robot provide fixed-frequency target information corresponding to the prompt words in natural language, the method further includes: The observation encoder obtains first environmental information of the aerial robot and second environmental information of the ground robot, wherein the first environmental information is the sum of environmental information collected in real time by the aerial robot and environmental information sent by the ground robot that is different from the environmental information collected in real time by the aerial robot, and the second environmental information is the sum of environmental information collected in real time by the ground robot and environmental information sent by the aerial robot that is different from the environmental information collected in real time by the ground robot; The observation encoder fuses the first environment information and the second environment information to obtain fused environment information; The observation encoder will integrate the environmental information for learning and obtain the target environmental information of the target frequency.

3. The method according to claim 2, characterized in that The observation encoder fuses the first environment information and the second environment information to obtain the fused environment information in the following process: The observation encoder determines the sum of the first environmental information and the second environmental information and environmental information different from the first environmental information as the fused environmental information.

4. The method according to claim 1, characterized in that: The environmental feature information of the target frequency is represented by a one-hot vector.

5. The method according to claim 1, characterized in that The aerial robot and the ground robot track the target robot based on the tracking information, including: The aerial robot uses continuous action space for tracking based on the tracking information, where the continuous action space includes direction and speed. The speed is a three-dimensional vector, which includes: angular velocity around the yaw axis, and linear velocity along the x and y axes; The ground robot uses McNamee wheels for tracking based on the tracking information.

6. The method according to claim 1, characterized in that The tracking information is used to control the aerial robot and the ground robot not to collide with the target environment information of the target frequency. The expression of the collision reward is: in, In When A, For aerial robots, In When G, For ground robots, is one of the target environment information of the target frequency, For collision rewards, For a safe distance.

7. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 6 when running.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

Images