A demand-driven robust tracking method for air-ground collaboration

Abstract

The present invention relates to a demand-driven robust tracking method for air-ground collaboration, belonging to the field of heterogeneous robot collaborative adaptive tracking. During the air-ground collaborative adaptive tracking process, the decision-making strategy of the air-ground robot is influenced by its own needs and the needs of the target. By drawing on the ERG demand theory, the air-ground robot can autonomously predict the target strategy and formulate a corresponding strategy accordingly. Based on this ERG demand mechanism, when the target changes its escape strategy, the air-ground robot can still accurately predict the target's intention strategy. Ultimately, the air-ground robot team can achieve robust and accurate tracking despite changes in the target strategy.

Description

Technical Field

[0001] The present invention belongs to the field of heterogeneous robot collaborative adaptive tracking, and in particular relates to a demand-driven air-ground collaborative robust tracking method. Background Art

[0002] Robots, hailed as the "crown jewel of manufacturing," are gaining increasing attention worldwide for their intelligence and autonomy. However, the overall functionality, task execution efficiency, and fault tolerance of individual robots are relatively limited, making them incapable of adapting to complex tasks and dynamic environments. Consequently, many researchers are turning their attention to multi-robot swarm systems, which offer greater task execution efficiency and environmental adaptability. A heterogeneous multi-robot collaborative system, comprised of aerial and ground robots, is a typical example of a multi-robot swarm system. It combines the complementary strengths of aerial and ground swarms, extending the two-dimensional plane of ground robot motion to three-dimensional space. By leveraging the aerial robots' flexibility and wide field of view with the ground robots' high payload and robust computational reasoning capabilities, it demonstrates significant advantages in task execution efficiency, environmental adaptability, and fault tolerance. It has successfully replaced humans in tasks such as search and rescue, autonomous exploration, and patrol and maintenance in hazardous, dirty, and extreme temperature environments.

[0003] However, facing complex tasks and dynamic environments, current air-ground collaborative systems still face challenges such as low intelligence and weak autonomy. Therefore, applying artificial intelligence (AI) and natural swarm intelligence theory to air-ground collaborative systems is an effective solution for enhancing both individual and collective intelligence. Although artificial intelligence (AI), represented by deep learning, has been widely used in robotic perception, decision-making, and control in recent years, the high real-time performance requirements of air-ground robots make it difficult to directly deploy AI algorithms on air-ground terminals, necessitating the design of more efficient communication mechanisms. Inspired by the activities of natural swarm intelligence, researchers at home and abroad have begun to incorporate natural swarm intelligence into their research on multi-robot systems. However, due to the high heterogeneity of air-ground robots' capabilities, structures, and action spaces, multi-robot collaboration based on natural swarm intelligence must still consider additional challenges such as system complexity, dynamics, time-varying behavior, and coupling when applied to air-ground collaboration.

[0004] Numerous studies have investigated the problem of collaborative air-ground tracking. The earliest work approached this problem from a planning perspective, focusing on tracking targets in static environments based on classical control theory. These studies often overlooked the information sources of the perception layer and instead relied on pre-processed data. Furthermore, they employed the same model for both air and ground robots, employing a homogeneous strategy to address the heterogeneous air-ground robot collaboration problem. However, this approach fails to fully exploit the complementary strengths of air and ground robots. Multi-Agent Deep Reinforcement Learning (MADRL) has long been widely used in collaborative robots, particularly in decision-making and planning tasks. To address environmental instability, most work has followed a centralized training distributed execution (CTDE) framework. This approach requires global information to be acquired through real-time communication during training. However, this scenario is uncommon in the real world due to high bandwidth requirements. Furthermore, each agent's decision-making, based solely on local observations, limits the ability of multiple agents to collaborate. Recent research aims to improve collaborative efficiency through effective communication between agents. These efforts focus on reducing communication resource consumption during collaboration, primarily by selecting communication partners, learning communication content, and managing communication frequency. Although existing work has been able to persistently track targets with fixed strategies in a single environment, it is unable to adapt to changes in the target strategy, making the tracking effect less robust. Summary of the Invention

[0005] The technical problems to be solved by the present invention are:

[0006] In order to avoid the shortcomings of the existing technology, the present invention provides a demand-driven air-ground collaborative robust tracking method, which is used to achieve collaborative tracking between multiple intelligent agents and perform adaptive adjustments based on target requirements and their own needs to improve the robustness of tracking.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A demand-driven air-ground collaborative robust tracking method, characterized by comprising:

[0009] The aerial robot obtains a first semantic map and a first target direction in real time and inputs the first observation encoder; the first observation encoder obtains a first environmental feature based on the first semantic map and the first target direction and inputs the first demand predictor; the first demand predictor obtains a first target predicted demand based on the first environmental feature and inputs the first target predicted demand into a demand alignment module;

[0010] The ground robot obtains a second semantic map and a second target direction in real time and inputs the second observation encoder; the second observation encoder obtains a second environmental feature based on the second semantic map and the second target direction and inputs the second demand predictor; the second demand predictor obtains a second target predicted demand based on the second environmental feature and inputs the second target predicted demand into a demand alignment module;

[0011] The demand alignment module obtains a target demand based on the first target predicted demand and the second target predicted demand;

[0012] Inputting the target demand and the aerial robot's own demand into a first decision module, the first decision module outputting instructions to adjust the aerial robot's continuous movements;

[0013] The target requirement and the ground robot's own requirement are input into a second decision module, and the second decision module outputs instructions to adjust the continuous action of the ground robot.

[0014] A further technical solution of the present invention is as follows: the first observation encoder and the second encoder have the same structure, using a CNN-LSTM framework, including:

[0015] The first semantic map is input into the CNN module for encoding to obtain a hidden vector. The hidden vector and the target direction are combined by MLP to obtain encoded information. The encoded information is input into the LSTM module to obtain environmental features, as shown in the following formula:

[0016]

[0017] in, Indicates robot R i The cell state at time step t, Indicates aerial robot R i The cell state at time step t-1, represents the environmental characteristics at time step t, represents the environmental characteristics at time step t-1, Indicates coded information; the subscript i is A or G, representing an aerial robot or a ground robot, respectively.

[0018] A further technical solution of the present invention is that the first demand forecaster has the same structure as the first demand forecaster, and adopts three MLP neural networks to respectively predict three target demands; the three target demands include survival needs, relationship needs and growth and development needs.

[0019] A further technical solution of the present invention: the demand alignment module adopts an MLP neural network.

[0020] A further technical solution of the present invention is as follows: the first decision module and the second decision module have the same structure, including an actor and a critic, the actor is used to output continuous actions, and the critic is used to estimate the quality of the actions.

[0021] A computer system, characterized in that it includes: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0022] A computer-readable storage medium is characterized by storing computer-executable instructions, which are used to implement the above method when executed.

[0023] A computer program product, characterized by comprising computer executable instructions, wherein the instructions are used to implement the above method when executed.

[0024] The beneficial effects of the present invention are:

[0025] This paper provides a demand-driven robust tracking method for air-ground collaborative tracking. During the air-ground collaborative adaptive tracking process, the decision-making strategy of the air-ground robot is influenced by its own needs and the needs of the target. By drawing on the ERG demand theory, the air-ground robot can autonomously predict the target's strategy and formulate a corresponding strategy accordingly. Based on this ERG demand mechanism, even if the target changes its escape strategy, the air-ground robot can still accurately predict the target's intention strategy. Ultimately, the air-ground robot team can achieve robust and accurate tracking despite the target's strategy changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0027] Figure 1 This is a process diagram of the method framework in an example of the present invention.

[0028] Figure 2 This is a conceptual diagram of the human social needs theory in the example of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In order to enable those skilled in the art to better understand the present invention, the present invention is described in detail below with reference to specific embodiments.

[0032] This invention exploits the following principle: During air-ground collaborative adaptive tracking, the decision-making strategy of an air-ground robot is influenced by its own needs and the needs of its target. By drawing on ERG demand theory, the air-ground robot can autonomously predict the target's strategy and formulate a corresponding strategy accordingly. Based on this ERG demand mechanism, even if the target changes its escape strategy, the air-ground robot can still accurately predict the target's intention strategy. Ultimately, the air-ground robot team can achieve robust and accurate tracking despite changes in the target's strategy.

[0033] 1) Establishing a collaborative target tracking model for air-ground robots

[0034] The air-ground robot needs to continuously and stably monitor the dynamic target under the constraints of the air-ground robot's field of view and communication distance, ensuring that the target is always within the air-ground robot's field of view and minimizing communication resources as much as possible. The tracking process can be seen as two air-ground heterogeneous intelligent agents (the aerial robot R A , ground robot R G ) Collaborative pursuit of target R T , the objective function is:

[0035]

[0036] in, Indicates that within time T, R A and R G With R T The sum of the distances between is defined as:

[0037]

[0038] Where dist(·,·) represents the Euclidean distance between the two robots at time t.A , π G and π T Represent the strategies of the aerial robot, ground robot, and target robot respectively, θ A ,θ G and θ T Represents its strategy parameters respectively. Function are all calculated results at time t. Ensure aerial robot R A The visual range c1 must always cover the target R T , Ensure aerial robot R A and ground robot R G The distance between them must be less than their maximum communication distance c2, that is, there is a connection between them. Ensure ground robot R G The distance to the obstacle must be greater than its safety distance c3. The goal is to avoid being tracked by the air-ground robot as much as possible, and the entire tracking process is a zero-sum game.

[0039] 2) Reinforcement Learning Modeling

[0040] Based on the above-established air-ground robot collaborative tracking target model, after analyzing the problem, the entire air-ground collaborative tracking process can be modeled as a decentralized partially observable Markov decision process (Decentralized Partially Observable Markov Decision Process, Dec-POMDP) in Represent the state space, observation space, action space, immediate reward and state transition probability respectively. At each time step (superscript t represents time), the aerial robot R A With ground robot R G Get the observation state from the environment respectively and based on their respective strategies π A ,π G Take action Then the environment state will be determined according to the state transition probability Update to the next state s t +1 At the same time, the air-ground robot obtains immediate rewards from the environment Aerial Robot R A The goal is to maximize the expected reward To learn strategies, ground robot R G Same thing.

[0041] Air-ground robot observation: Aerial robot R A Get the top-down semantic map and its relationship with the target R T The relative direction of each semantic graph is represented by a one-hot vector to represent free space, obstacles and other objects. G Observation and aerial robot R A Similar, but with a wider field of view than the Aerial Robot R A To make the tracking process more challenging, the target R T All observations within the entire environment can be obtained.

[0042] High-level motion of air-ground robots: Air-ground robots all use continuous motion space a i . Ground Robot R G The McNamee wheels allow for free change of direction. The speed of the ground robot is a c,G It is a three-dimensional vector, including the angular velocity around the yaw axis and the linear velocity along the x and y axes. A The action space of the ground robot R G Similarly, both air-ground robots use high-level strategies, and low-level strategies are controlled by PID controllers.

[0043] Heterogeneous robot communication: Air-ground robots communicate through mobile self-organizing networks, and point-to-point information sharing can be carried out between robots.

[0044] Reward function: According to the ERG (Existence Relatedness Growth) demand theory, the target strategy of an air-ground robot is influenced by its own needs and the needs of others. The air-ground robot should maximize its own needs as much as possible. In other words, its own needs are the intrinsic motivation for its behavioral decisions. It seeks to maximize the reward function. Specifically, they are defined as:

[0045]

[0046] Among them, μ1,μ2>0,γ1,γ2>0,ε1,ε2>0 are hyperparameters. Represents the survival needs reward. The air-ground robot will develop other needs based on its own survival needs. When the distance between the air-ground robot and the obstacle is less than its safe distance, it will be punished. The specific definition is:

[0047]

[0048] Where n is the number of obstacles in the environment. Represents a mutual relationship reward. The air-ground robot will maintain a close relationship within its communication range. When the air-ground robot is within its communication range, it will be rewarded. The specific definition is:

[0049]

[0050] Among them, d safety is the minimum safe distance between air-ground robots. Represents the growth and development needs reward. The goal of the air-ground robot is to chase the target, so it must try to keep the target within its field of view. If the distance between the robot and the target exceeds its maximum field of view, it is meaningless. It is defined as:

[0051]

[0052] Among them, dist(R i ,R T ) represents the real-time distance between the air-ground robot and the target, D obs Indicates the maximum observation distance of the robot, d * Indicates the optimal observation distance.

[0053] At the same time, the target's behavior is also driven by its internal needs, so various levels of need rewards are also set for it. It is worth noting that the target's survival reward is the same as that of the air-ground robot, and both must ensure that they do not collide with obstacles. The definitions of the relationship reward and development need reward are as follows:

[0054]

[0055] The target's interrelationship requirement is to escape the air-ground robot's pursuit. The farther away from the air-ground robot, the more satisfied this requirement is, and the greater the reward. The target's growth and development requirement is to reach the destination as quickly as possible. The closer to the destination, the greater the reward. The pursuit-escape process between the target and the air-ground robot is a zero-sum game, so the target's overall reward can be defined as:

[0056]

[0057] Among them, μ3,γ3,ε3>0 are hyperparameters

[0058] 3) Independent reinforcement learning framework design

[0059] Each air-ground robot adopts a personalized model. A Includes observation encoder module and demand forecasting module.

[0060] The observation encoder module adopts the CNN-LSTM framework and consists of two parts: first, visual information That is, the semantic map will be input into the CNN module to encode the hidden vector features, the hidden vector features and the target direction Combined with the MLP to obtain the encoded information, the encoded information is passed to the LSTM module to obtain the environmental features, as shown in the formula:

[0061]

[0062] in, Indicates aerial robot R A The cell state at time step t, the environment characteristics Can implicitly represent the characteristics of the target, To encode information.

[0063] The demand forecasting module is used to predict target demand and adopts three MLP neural networks to predict three target demands respectively.

[0064] The demand alignment module is used to unify the target demand predicted by the aerial robot and the target demand predicted by the ground robot.

[0065] Environmental characteristics The unified target requirements are passed to the decision module, and the last continuous actor in the decision module is used to output continuous actions, and a critic is used to estimate the quality of the action.

[0066] With Aerial Robot R A Similarly, the ground robot R G The visual input is fed into the CNN and MLP modules to encode features and predict target requirements. The system then determines its behavior based on the environmental characteristics and the predicted target requirements. The reinforcement learning module then uses the target requirements and its own observations to output continuous actions.

[0067] During the training process, each robot is trained using the Proximal Policy Optimization algorithm.

[0068] Based on the above-mentioned air-ground robot collaborative tracking target model, reinforcement learning modeling is performed on the air-ground robot collaborative tracking target model, and an independent reinforcement learning framework is designed for the reinforcement learning model. The present invention provides a demand-driven air-ground collaborative robust tracking method. First, the air-ground robots respectively obtain their first-person observation states, and use the ERG (Existence Relatedness Growth) demand prediction module to predict the future demand of the target. Then, taking into account the heterogeneous characteristics of the air-ground robots, the target demand predicted by the air-ground robots will undergo a value unification process; then the unified target demand will be input into their respective decision modules, and the reinforcement learning module will determine its own actions based on the observation information and target demand. Figure 1 As shown, specifically including:

[0069] The aerial robot obtains a first semantic map and a first target direction in real time and inputs the first observation encoder; the first observation encoder obtains a first environmental feature based on the first semantic map and the first target direction and inputs the first demand predictor; the first demand predictor obtains a first target predicted demand based on the first environmental feature and inputs the first target predicted demand into a demand alignment module;

[0070] The ground robot obtains a second semantic map and a second target direction in real time and inputs the second observation encoder; the second observation encoder obtains a second environmental feature based on the second semantic map and the second target direction and inputs the second demand predictor; the second demand predictor obtains a second target predicted demand based on the second environmental feature and inputs the second target predicted demand into a demand alignment module;

[0071] The demand alignment module obtains a target demand based on the first target predicted demand and the second target predicted demand;

[0072] Inputting the target demand and the aerial robot's own demand into a first decision module, the first decision module outputting instructions to adjust the aerial robot's continuous movements;

[0073] The target requirement and the ground robot's own requirement are input into a second decision module, and the second decision module outputs instructions to adjust the continuous action of the ground robot.

[0074] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present invention, and these modifications or replacements should all be included in the scope of protection of the present invention.

Claims

1. A demand-driven air-ground collaborative robust tracking method, characterized by: include: The aerial robot obtains the first semantic map and the first target direction in real time and inputs them into a first observation encoder; The first observation encoder obtains a first environmental feature based on the first semantic map and the first target direction and inputs the first environmental feature into a first demand predictor; the first demand predictor obtains a first target predicted demand based on the first environmental feature and inputs the first target predicted demand into a demand alignment module; The ground robot obtains a second semantic map and a second target direction in real time and inputs the second observation encoder; the second observation encoder obtains a second environmental feature based on the second semantic map and the second target direction and inputs the second demand predictor; the second demand predictor obtains a second target predicted demand based on the second environmental feature and inputs the second target predicted demand into a demand alignment module; The demand alignment module obtains a target demand based on the first target predicted demand and the second target predicted demand; Inputting the target demand and the aerial robot's own demand into a first decision module, the first decision module outputting instructions to adjust the aerial robot's continuous movements; The target requirement and the ground robot's own requirement are input into a second decision module, and the second decision module outputs instructions to adjust the continuous action of the ground robot.

2. The demand-driven air-ground collaborative robust tracking method according to claim 1, characterized in that: include: The first observation encoder and the second encoder have the same structure and adopt the CNN-LSTM framework, including: The first semantic map is input into the CNN module for encoding to obtain a hidden vector. The hidden vector and the target direction are combined by MLP to obtain encoded information. The encoded information is input into the LSTM module to obtain environmental features, as shown in the following formula: in, Indicates robot R i The cell state at time step t, Indicates aerial robot R i The cell state at time step t-1, represents the environmental characteristics at time step t, represents the environmental characteristics at time step t-1, Indicates coded information; the subscript i is A or G, representing an aerial robot or a ground robot, respectively.

3. The demand-driven air-ground coordinated robust tracking method according to claim 1, characterized in that: The first demand forecaster has the same structure as the first demand forecaster, and adopts three MLP neural networks to respectively predict three target demands; the three target demands include survival demands, relationship demands and growth and development demands.

4. The demand-driven air-ground coordinated robust tracking method according to claim 1, characterized in that: The demand alignment module adopts an MLP neural network.

5. The demand-driven air-ground coordinated robust tracking method according to claim 1, characterized in that: The first decision module and the second decision module have the same structure, including an actor and a critic, the actor is used to output continuous actions, and the critic is used to estimate the quality of actions.

6. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

7. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

8. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when the instructions are executed.

Images