Dynamic target intelligent tracking method and system based on air-ground unmanned cooperative system

Abstract

The invention discloses a dynamic target intelligent tracking method and system based on an air-ground unmanned cooperative system, and the method comprises the steps: obtaining the kinematics information of air and ground equipment, constructing an environment map, and obtaining the initial position of a dynamic target; determining a system state vector according to kinematics information, establishing a state equation and an observation equation, and extracting a kinematics state equation; updating system state estimation and a covariance matrix by applying a prediction and correction process of extended Kalman filtering; establishing an error state equation and an error covariance matrix to obtain position and posture information of air and ground equipment; the air flight equipment determines a dynamic tracking path by combining the initial position, the ground environment information and a path planning method; air and ground equipment is controlled in real time according to the pose information and the dynamic tracking path, and air and ground cooperative tracking of the dynamic target is achieved. According to the method, the overall cooperative performance of the air-ground unmanned platform is remarkably improved, the efficiency loss caused by independent task execution of the platform is effectively reduced, and the tracking capability of the dynamic target and the task execution efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of automated control and robotics technology, and in particular to a method and system for intelligently tracking dynamic targets based on an air-to-ground unmanned collaborative system. Background Art

[0002] Dynamic target tracking refers to the technology of using various sensors and algorithms to continuously monitor, locate and track targets in motion in order to obtain information such as the target's real-time position, speed, motion trajectory, etc.

[0003] In the field of dynamic target tracking, it is difficult to fully describe the state of the target by using a single sensor to perform dynamic target tracking tasks in a complex environment. In addition, the behavior and state of dynamic targets vary, and the information provided by a single sensor is difficult to meet the needs of comprehensive understanding and tracking of the target. In the existing technology, drones and unmanned vehicles usually perform dynamic target tracking tasks alone. Drones have the advantage of high altitude and can obtain information about large areas from a macro perspective, and have a better grasp of the overall movement trend of the target and the surrounding environment. Unmanned vehicles are close to the ground and can observe the target at close range to obtain more detailed local information.

[0004] The lack of effective coordination mechanism between drones and unmanned vehicles results in low efficiency in tracking dynamic targets in complex environments. Summary of the invention

[0005] The technical purpose of this application is to solve the technical problem in the prior art that drones and unmanned vehicles lack an effective coordination mechanism in dynamic target tracking tasks, resulting in low tracking efficiency, and to propose a dynamic target intelligent tracking method and system based on an air-to-ground unmanned collaborative system.

[0006] To implement the above technical solution, this application adopts the following technical solution.

[0007] In a first aspect, an embodiment of the present application provides an air-to-ground unmanned cluster dynamic target intelligent tracking method, wherein the air-to-ground unmanned collaborative system includes an aerial flying device and a ground mobile device, and the method includes:

[0008] Acquiring kinematic information of the aerial flying device and the ground moving device;

[0009] Based on the collected ground image, an environmental map is constructed and the initial position of the dynamic target is obtained; the state vector of the air-ground unmanned collaborative system is determined according to the kinematic information; based on the state vector, the state equation and observation equation of the air-ground unmanned collaborative system are established; based on the state equation, the kinematic state equation is extracted; based on the kinematic state equation and the observation equation, the state estimation and state covariance matrix of the air-ground unmanned collaborative system are continuously updated through the prediction process and correction process of the extended Kalman filter; based on the kinematic state equation and the system error parameters, the error state equation of the air-ground unmanned collaborative system is established; based on the error state equation, an error covariance matrix is ​​established; based on the state estimation, the system error parameters and the error covariance matrix, the position and posture information of the aerial flight equipment and the ground mobile equipment is obtained;

[0010] The aerial flight device obtains the initial position and ground environment information of the dynamic target, and determines a dynamic tracking path in combination with a path planning method;

[0011] According to the position information of the aerial flying device and the ground mobile device and the dynamic tracking path, the aerial flying device and the ground mobile device are controlled in real time to achieve air-ground collaboration to complete the dynamic target tracking task.

[0012] In a second aspect, an embodiment of the present application provides a dynamic target intelligent tracking system based on an air-ground unmanned collaborative system, wherein the air-ground unmanned collaborative system includes an aerial flying device and a ground mobile device, and the system includes:

[0013] A parameter acquisition module, used to acquire kinematic information of the aerial flying device and the ground moving device;

[0014] A model building module, used to build an environmental map based on the collected ground image and obtain the initial position of the dynamic target; determine the state vector of the air-ground unmanned collaborative system according to the kinematic information; establish the state equation and observation equation of the air-ground unmanned collaborative system according to the state vector; extract the kinematic state equation based on the state equation; based on the kinematic state equation and the observation equation, continuously update the state estimation and state covariance matrix of the air-ground unmanned collaborative system through the prediction process and correction process of the extended Kalman filter; establish the error state equation of the air-ground unmanned collaborative system according to the kinematic state equation, establish the error covariance matrix according to the error state equation, solve the error covariance matrix, and thus determine the error of the state estimation; obtain the position and posture information of the aerial flight equipment and the ground mobile equipment according to the state estimation, the state covariance matrix, and the error covariance matrix;

[0015] A path planning module, used for the aerial flight device to obtain the initial position and ground environment information of the dynamic target, and determine a dynamic tracking path for the dynamic target in combination with a path planning method;

[0016] The collaborative control module is used to control the aerial device and the ground mobile device in real time according to the environmental map, the position information of the aerial device and the ground mobile device and the dynamic tracking path, so as to achieve air-ground collaborative completion of the dynamic target tracking task.

[0017] In a third aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements a dynamic target intelligent tracking method based on an air-to-ground unmanned collaborative system as provided in any possible implementation method of the first aspect.

[0018] Compared with the prior art, the dynamic target intelligent tracking method based on the air-ground unmanned collaborative system provided in the embodiment of the present application deeply considers the close collaborative relationship between aerial flying equipment and ground mobile equipment. Through the constructed air-ground unmanned collaborative system, the system can demonstrate better target tracking and path planning capabilities when facing dynamic target tracking tasks in complex environments or the challenge of limited field of view of a single platform; this improvement significantly improves the overall collaborative performance of the air-ground unmanned platform and effectively reduces the efficiency loss caused by the platform performing tasks alone. In addition, the method also innovatively adopts the extended Kalman filter algorithm, which can flexibly adjust the control strategy according to the real-time state parameters and observation data of the drone and unmanned vehicle. This precise control method, coupled with an effective compensation mechanism for positioning errors, jointly ensures that drones and unmanned vehicles can achieve high-precision collaborative navigation during the execution of tasks.

[0019] The dynamic target intelligent tracking system based on the air-ground unmanned collaborative system provided in the embodiment of the present application has the same beneficial technical effects as the above-mentioned dynamic target intelligent tracking method based on the air-ground unmanned collaborative system, which will not be elaborated on. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present application in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes and are used to help understand the present application. They do not specifically limit the shapes and proportional dimensions of the components of the present application. Under the guidance of the present application, those skilled in the art can select various possible shapes and proportional dimensions to implement the present application according to specific circumstances. In the drawings:

[0021] Figure 1 It is a schematic diagram of the process flow of a dynamic target intelligent tracking method based on an air-to-ground unmanned collaborative system provided in an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of the structure of the aerial flying device and the ground mobile device in the embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of the unmanned vehicle positioning process of this application;

[0024] Figure 4 This is a flowchart of the collaborative task between a drone and a ground intelligent vehicle and an air-to-ground robot in another embodiment of the present application;

[0025] Figure 5 is the UAV simulation model parameter in one embodiment of the present application;

[0026] Figure 6 This is the hardware control process of the unmanned vehicle in another embodiment of the present application;

[0027] Figure 7 This is a drone simulation scene (extracted part) in another embodiment of the present application;

[0028] Figure 8 It is the position of the QR code from the perspective of the downward-looking camera of the drone in yet another embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0030] This application aims to address the technical problem in the prior art that drones and unmanned vehicles lack an effective coordination mechanism in dynamic target tracking tasks, resulting in low tracking efficiency, and proposes a dynamic target intelligent tracking method and system based on an air-to-ground unmanned collaborative system.

[0031] The dynamic target intelligent tracking method based on the air-ground unmanned collaborative system provided by this application adopts the collaborative cooperation between drones and unmanned vehicles. This method can not only take advantage of the drone's camera to explore the high-altitude field of view, thereby obtaining field of view information that the unmanned vehicle cannot perceive; but also take advantage of the drone's flying over obstacles to further ensure the unmanned vehicle's field of view while providing a global perspective. Therefore, through the drone-unmanned vehicle collaboration, the advantages of both are combined to make up for each other's shortcomings, so that the drone can work more effectively and significantly improve work efficiency.

[0032] In complex situations, unmanned vehicles often have difficulty successfully tracking dynamic targets due to limited field of view and uncertainty of dynamic targets, resulting in failure of the tracking mission. In contrast, drones have advantages such as a wide field of view and flexible maneuverability, and can perform dynamic target tracking tasks more effectively. When an unmanned vehicle loses a dynamic target during the tracking process, the drone can provide the unmanned vehicle with surrounding environment information and dynamic target coordinate information, allowing it to continue the tracking mission. The dynamic target tracking mission starts from the first frame of the tracking, manually selecting the target, and using the corresponding tracking algorithm to enable the drone or unmanned vehicle to autonomously track the target.

[0033] The present application is further described below in conjunction with the accompanying drawings and specific embodiments.

[0034] The embodiment provides a dynamic target intelligent tracking method based on an air-ground unmanned collaborative system, wherein the air-ground unmanned collaborative system includes an aerial flying device and a ground mobile device, such as Figure 1 As shown, the method includes:

[0035] Step 1: Obtain kinematic information of aerial flying equipment and ground mobile equipment;

[0036] Step 2: Build an environmental map based on the collected ground images and obtain the initial position of the dynamic target; determine the state vector of the air-ground unmanned collaborative system based on the kinematic information; establish the state equation and observation equation of the air-ground unmanned collaborative system based on the state vector; extract the kinematic state equation based on the state equation; continuously update the state estimation and state covariance matrix of the air-ground unmanned collaborative system through the prediction process and correction process of the extended Kalman filter based on the kinematic state equation and observation method; establish the error state equation of the air-ground unmanned collaborative system based on the kinematic state equation and system error parameters; establish the error covariance matrix based on the error state equation; obtain the position and posture information of the aerial flight equipment and the ground mobile equipment based on the state estimation, system error parameters and error covariance matrix;

[0037] Step 3: The aerial flight equipment obtains the initial position and ground environment information of the dynamic target, and determines the dynamic tracking path in combination with the path planning method;

[0038] Step 4: Based on the environmental map, the position information and dynamic tracking path of the aerial flight equipment and ground mobile equipment, the aerial flight equipment and ground mobile equipment are controlled in real time to achieve air-ground collaboration to complete the dynamic target tracking task.

[0039] In the embodiment, the structures of the aerial flying device and the ground mobile device are as follows: Figure 2As shown. Aerial flying equipment (such as drones) and ground mobile equipment (such as unmanned vehicles) transmit data through a communication data link. The ground mobile equipment is equipped with a ground processor, a first camera, a controller and a sensor, and the controller controls the first motor. The aerial flying equipment is equipped with a second camera, an aerial processor, a second motor and others (sensors, etc.). In the embodiment, the sensors carried by the aerial flying equipment and the ground mobile equipment are such as IMU inertial sensors, visual odometers, EKF positioning modules, T265 positioning cameras and / or laser radars, etc.

[0040] Before the method is implemented, tasks can be allocated according to target tracking requirements, and aerial flying equipment and ground mobile equipment can be selected to track the dynamic target to be tracked.

[0041] The sensors installed on the aerial flying equipment and the ground mobile equipment can be used to collect the three-axis position, three-axis speed, three-axis attitude, acceleration and acceleration, so as to monitor its operating status in real time.

[0042] In the embodiment, the state vector in step 2 is It can be expressed as follows:

[0043]

[0044] in, Represents the position of the three axes in the world coordinate system. Represents the speed of the three axes in the world coordinate system, Represents the posture of the three axes in the world coordinate system. and There are three degrees of freedom (x, y, z). represents the angular velocity deviation, b a Represents the acceleration deviation.

[0045] In a specific embodiment, the angular velocity and acceleration can be measured by using an IMU inertial sensor.

[0046] According to the state vector, the state equation and observation equation of the air-ground unmanned collaborative system are established, which are the state equation and observation equation of the nonlinear system of the traditional EKF and are expressed as:

[0047]

[0048] Among them, s k is the state vector of the system at time k, s k-1 is the state vector of the system at time k-1, is the process noise, γ k To measure noise, represents the system dynamic equation, u k is the control input at time k, is the observation equation, ∈ k is the observation vector at time k.

[0049] The kinematic state equation is extracted based on the state equation. For example, the differential of the position can get the velocity, so the following kinematic state equation can be established:

[0050]

[0051] in, For the quaternion The corresponding rotation matrix is, express The attitude of the three axes under acceleration a and in the world coordinate system The corresponding rotation matrix, g represents the acceleration constant, n bω With n ba represent the random walk noise of acceleration and angular velocity respectively, and ω is the angular velocity.

[0052] In an embodiment, the prediction process of the extended Kalman filter is used to predict the system state at the next moment based on the current state estimate and the kinematic state equation, and update the state covariance matrix to estimate the uncertainty of the state estimate at the next moment;

[0053] The correction process of the extended Kalman filter is used to calculate the predicted observation value based on the predicted system state at the next moment, compare it with the actual measurement value, and calculate the Kalman gain using the state covariance matrix based on the comparison result, and then update the state estimate and the state covariance matrix to make the state estimate more accurate and timely reflect the latest uncertainty.

[0054] The specific prediction and correction steps are as follows:

[0055] 1) Prediction

[0056]

[0057] 2) Calibration

[0058]

[0059] Among them, the state transfer matrix For Jacobian, Observation equation Linearize to get the Jacobian matrix

[0060] The extended Kalman filter is used to update the state estimate to achieve the pose update, and the updated pose information is used to locate the flying device in the air. The prediction and correction formula based on the extended Kalman filter is used to iteratively calculate and determine the optimal state estimate of the nonlinear system.

[0061] In some embodiments, a QR code (Aurco code) is provided on the outside of the top surface of the ground mobile device; Figure 8 The specific location of the QR code can be clearly seen in the downward camera view of the drone. The aerial device uses the camera to collect pictures including the top outer surface of the ground mobile device; the picture is subjected to target detection. If the Aurco code is detected, the internal code of the Aurco code is read and the position and attitude information of the ground mobile device relative to the aerial device camera is obtained with the help of the visual odometer to determine the observation value, and then determine the observation function, which is used for state update.

[0062] In a specific embodiment, in the prediction process of the extended Kalman filter, the state covariance matrix is ​​updated using the covariance matrix of the noise, specifically including: using an IMU inertial sensor to measure the acceleration and angular velocity of the aerial flight equipment and the ground mobile equipment; there is noise in the measurement value of the IMU inertial sensor, so the relationship between its true value and the measurement value is expressed as:

[0063]

[0064] Among them, ω is the true angular velocity, ω m is the measured angular velocity, b w is the angular velocity deviation (of IMU), n w is the random walk noise of angular velocity, a is the real acceleration, a m is the measured acceleration, b a is the acceleration deviation (of IMU), n a is the random walk noise of acceleration.

[0065] In view of the continuous existence of disturbances and noise in actual motion, estimated values ​​are usually used instead of true values ​​for calculation. Based on this, the above formula can be transformed into the following expression:

[0066]

[0067] According to the relationship and the kinematic state equation, the difference between the state value and the true value is taken as the state error

[0068]

[0069] The differential equation for the continuous-time error is thus determined as follows:

[0070]

[0071] Solving the partial derivatives of the differential equation, we can get the error state equation, which includes the state transfer matrix and the noise matrix; the error state equation is continuous and linear, as shown below:

[0072]

[0073] In the formula, is the state transfer matrix, G c is the noise matrix, and the specific parameters of the two matrices are as follows:

[0074]

[0075] According to the error state equation, the noise covariance matrix Q is obtained c :

[0076]

[0077] In order to reduce the computational burden and simplify the complexity of the time model, specific variables and Q c A further discretization process was implemented, using △t as the integration time interval, and the specific expression is as follows:

[0078]

[0079] Discretization is performed based on the state transfer matrix and the noise matrix respectively, and then the state covariance matrix corresponding to a given time point is calculated using the Riccati equation as the updated state covariance matrix to achieve effective estimation of the state quantity. The specific process is as follows:

[0080]

[0081] In practical applications, the update rate of IMU is significantly higher than that of visual odometer. However, the error of IMU will accumulate over time, resulting in a gradual decrease in estimation accuracy. To solve this problem, in some embodiments, the pose calculated by visual odometer is used as the observation value of extended Kalman filter (EKF), and Gaussian white noise is introduced into the pose of the robot to simulate the measurement noise of the robot.

[0082] As an example, the method further includes: the system error parameter includes a measurement position error and a posture estimation error; the method further includes: introducing Gaussian measurement noise into the position information and the posture information to obtain the measured values ​​of the position measurement value and the posture measurement value, and using the position measurement value and the posture measurement value as the observation value; wherein the introduction of Gaussian white noise is expressed as:

[0083]

[0084] Among them, n m is a Gaussian white noise vector, n p is the position measurement white noise, n q is the attitude measurement white noise, n p The mean is 0 and the variance is Gaussian distribution, n q The mean is 0 and the variance is Gaussian distribution.

[0085] Since the coordinate systems of the cameras carried by the aerial flight equipment and the ground mobile equipment in the air-ground unmanned collaborative system coincide with the coordinate systems of the aerial flight equipment and the ground mobile equipment, the position of the camera can be directly expressed by the corresponding representation symbols:

[0086]

[0087] ∈ p is the position measurement value. Therefore, the measured position error As shown below:

[0088]

[0089] right beg The partial derivative of the measured position error can be obtained

[0090]

[0091] Similarly, we can get ∈ q , and the partial derivative of the measured attitude error

[0092]

[0093] In the formula, ∈ q is the attitude measurement value, To measure the attitude error, is the partial derivative of the measured attitude error.

[0094] In the embodiment, the partial derivative of the measured position error is obtained above and the partial derivative of the measured attitude error After that, the measurement value is updated as follows, that is, according to the state estimation, the system error parameter (measurement error ) and the error covariance matrix to obtain the position and posture information of the aerial flight equipment and the ground mobile equipment, including:

[0095] The gradient vector and gradient-related covariance matrix are solved based on the state estimate, system error parameter, and partial derivative of the system error parameter. The expression is as follows:

[0096] Solve the Kalman gain based on the gradient vector and the gradient-related covariance matrix:

[0097] By Kalman gain and error estimate Solve for the posterior state estimator:

[0098] Solve the error covariance matrix, the expression is:

[0099]

[0100] The error of the posterior state estimate is obtained, and finally the position and posture information of the aerial flight equipment and the ground mobile equipment is obtained.

[0101] After the above calculations, the binocular visual odometers of drones and unmanned vehicles can be fused with the IMU inertial measurement sensors. Compared with the positioning algorithm that relies on only a single sensor, this fused positioning algorithm shows higher stability and the accuracy of the positioning results is significantly improved.

[0102] In step 2, the collected ground image information is processed to build an environmental map and obtain the preliminary position of the target. By processing and analyzing these sensor data and using the extended Kalman filter (EKF) algorithm for data fusion, accurate attitude solution is achieved and the positioning of the aerial drone is realized.

[0103] In some embodiments, the collaborative task flow chart of the aerial drone and the unmanned vehicle (ground smart car) is as follows: Figure 4 As shown, in step three, the aerial flying device (drone) relies on the onboard positioning camera and lidar to obtain the dynamic target and the initial position of the dynamic target and the position and shape of obstacles and other ground environment information, and transmits it to the processor on the aerial drone. The path is planned in combination with the graph theory algorithm, and dynamic tracking is achieved through Wi-Fi communication based on obstacle avoidance and target heading behavior.

[0104] In the embodiment, stable ranging and high-precision point cloud mapping can be achieved with the help of laser radar. The specific simulation model parameters of the drone are as follows: Figure 5 As shown, with the corresponding path planning software and program, use the communication equipment to transmit the navigation file to the flight control, select the best path for the UAV according to the target location and environmental information, and execute the command to reach the designated location;

[0105] In the actual implementation process, as one of the embodiments, in step 4, the drone and the unmanned vehicle can achieve stable communication between the sensors inside the drone and with the flight control based on the MAVROS tool in the ROS system environment. The hardware control process of the unmanned vehicle here is clearly shown in Figure 6 In, from Figure 6 As can be seen in the figure, the hardware control of the unmanned vehicle is an orderly and complex process, covering multiple links such as signal input, processing and execution. The unmanned vehicle obtains the image information of the target location through the camera and sends it to the main control unit. For the simulation scene of the drone, please refer to Figure 7 . Figure 7 The various settings and operating conditions of the drone in the virtual environment are presented. This information is crucial for simulating the real flight status of the drone in the simulation, and can help technicians evaluate and optimize the performance and tracking strategy of the drone in advance. In the entire air-ground collaborative system, Python language is used in combination with relevant SDKs for control and communication design, so as to achieve air-ground collaborative completion of dynamic target tracking tasks.

[0106] In some embodiments, during the game process, the strategy combination of the two robots jointly affects their benefits. According to the basic principles of game theory, the participating entities are drones and unmanned vehicles, with special attention paid to the observation of unmanned vehicles by drones. The drone receives information from unmanned vehicles or other collaborative devices through Wi-Fi communication (using socket sockets to establish connections) to assist in tracking tasks. In the face of target movement, the drone flexibly adjusts the flight attitude and path based on the built-in obstacle avoidance mechanism (calculating weights based on the instant distance to obstacles to avoid collisions) and the goal-oriented flight strategy (continuously flying towards the target point) to ensure continuous tracking of the target. The unmanned vehicle evaluates the positioning optimization effect of this information and then decides whether to use this information. Given that all participants make decisions at the same time, the game is defined as a complete information static game.

[0107] The positioning process of car-free people can be found in Figure 3 , the positioning of the UAV to the unmanned vehicle under the complete information game is expressed as:

[0108] T g,G =T g,A +Ξ g,A ·(T A,c +Ξ A,c ·T c,G );

[0109] Among them, T g,G refers to the coordinates of the unmanned vehicle, T g,A refers to the coordinates of the drone, R g,A Represents the rotation matrix of the drone itself in world coordinates, which can be solved by the official OnBoard SDK and can be regarded as a constant.A,c Represents the coordinate conversion between the drone and the camera it carries. A,c Represents the rotation change between the drone and its own camera, T c,G Refers to the coordinate transformation from camera to unmanned vehicle.

[0110] In the co-location scenario, due to R g,A , T A,c , R A,c , T c,G These parameters are constant, so when implementing collaborative positioning, the covariance matrix of the unmanned vehicle can directly use the covariance matrix of the drone itself. In addition, according to the specific application scenario targeted by this patent, only the coordinate calculation of the x-axis and y-axis is concerned. Therefore, in this case, T g,A Simplified to At the same time, the covariance matrix of the unmanned vehicle is simplified to a 2x2 matrix The value of this matrix is ​​equivalent to the corresponding part of the drone covariance matrix

[0111] The dynamic target intelligent tracking method based on the air-ground unmanned collaborative system provided by the embodiment of the present application shows significant advantages in positioning in the air-ground collaborative system compared to the way that the unmanned vehicle is positioned separately. The drone can observe the unmanned vehicle with itself as the center and generate corresponding observation information, which is used to update the state of the unmanned vehicle, thereby improving the positioning accuracy. However, when the unmanned vehicle receives the observation information provided by the drone, it may encounter inconsistencies with its own positioning information or even conflicts. This phenomenon not only increases the computational overhead, but also may cause the divergence of the EKF algorithm. In order to solve the above problems, an innovative solution is proposed, that is, to integrate the complete information static game theory into the positioning algorithm, so as to make an intelligent choice of whether the unmanned vehicle is collaboratively positioned. In addition, considering that the unmanned vehicle studied in this paper only runs on horizontal ground, its Z-axis direction coordinate is always kept at 0, so in practical applications, only the accuracy of the unmanned vehicle in the X-axis and Y-axis directions needs to be paid attention to, and the update in the Z-axis direction can be abandoned. This optimization makes the positioning framework of the unmanned vehicle more efficient and accurate.

[0112] It is understandable that the steps in the above embodiments are only for illustrating the embodiments and cannot be understood as limiting the execution steps. In some embodiments, the order of implementing the steps is not limited to the above step sequence.

[0113] Based on the dynamic target intelligent tracking method based on the air-to-ground unmanned collaborative system provided in the above embodiments, the embodiments of the present application also provide a dynamic target intelligent tracking system based on the air-to-ground unmanned collaborative system, including a parameter acquisition module, a model building module, a path planning module and a collaborative control module.

[0114] The parameter acquisition module is used to obtain the kinematic information of the aerial flying device and the ground moving device.

[0115] The model building module is used to build an environmental map based on the collected ground images and obtain the initial position of the dynamic target; determine the state vector of the air-ground unmanned collaborative system based on the kinematic information; establish the state equation and observation equation of the air-ground unmanned collaborative system based on the state vector; extract the kinematic state equation based on the state equation; based on the kinematic state equation and the observation equation, continuously update the state estimation and state covariance matrix of the air-ground unmanned collaborative system through the prediction process and correction process of the extended Kalman filter; establish the error state equation of the air-ground unmanned collaborative system based on the kinematic state equation, establish the error covariance matrix based on the error state equation, solve the error covariance matrix, and thus determine the error of the state estimation; obtain the position and posture information of the aerial flight equipment and the ground mobile equipment based on the state estimation, the state covariance matrix, and the error covariance matrix.

[0116] The path planning module is used for the aerial flight equipment to obtain the initial position and ground environment information of the dynamic target, and combines the path planning method to determine the dynamic tracking path for the dynamic target.

[0117] The collaborative control module is used to control the aerial equipment and the ground mobile equipment in real time according to the environmental map, the position information of the aerial equipment and the ground mobile equipment, and the dynamic tracking path, so as to realize the air-ground collaboration to complete the dynamic target tracking task.

[0118] The dynamic target intelligent tracking system based on the air-ground unmanned collaborative system provided in the embodiment of the present application builds an environmental perception architecture for the target area and shapes the target model with carefully selected sensor data. With the help of the positioning camera and laser radar carried by the drone, the target position is determined using cutting-edge image recognition and positioning methods, and then an efficient path and collaborative control solution is planned. In this way, dynamic targets can be accurately tracked, breaking through the shackles of traditional tracking and ensuring that the system operates stably and reliably in complex environments.

[0119] The dynamic target intelligent tracking method designed in this application is innovative and operational for achieving efficient target tracking. It demonstrates good tracking effects in actual scenarios through a unique combination of sensors and algorithm applications. It has certain reference significance for other related unmanned system research and solving the dynamic target intelligent tracking problem of air-to-ground unmanned collaborative systems, and provides ideas and directions for subsequent technical improvements and expansions.

[0120] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned dynamic target intelligent tracking method based on the air-to-ground unmanned collaborative system.

[0121] It can be understood that the execution steps of each module in the above dynamic target intelligent tracking system based on the air-ground unmanned collaborative system can refer to the various steps of the dynamic target intelligent tracking method based on the air-ground unmanned collaborative system provided in the above embodiments, and will not be repeated here.

[0122] The above is a detailed introduction to the dynamic target intelligent tracking method and system based on the air-to-ground unmanned collaborative system provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the concept of the present application and should not be understood as limiting the scope of protection of the present application.

Claims

1. A dynamic target intelligent tracking method based on an air-ground unmanned collaborative system, characterized in that: The air-ground unmanned collaborative system includes an aerial flying device and a ground mobile device, and the method includes: Acquiring kinematic information of the aerial flying device and the ground moving device; Based on the collected ground image, an environmental map is constructed and the initial position of the dynamic target is obtained; the state vector of the air-ground unmanned collaborative system is determined according to the kinematic information; based on the state vector, the state equation and observation equation of the air-ground unmanned collaborative system are established; based on the state equation, the kinematic state equation is extracted; based on the kinematic state equation and the observation equation, the state estimation and state covariance matrix of the air-ground unmanned collaborative system are continuously updated through the prediction process and correction process of the extended Kalman filter; based on the kinematic state equation and the system error parameters, the error state equation of the air-ground unmanned collaborative system is established; based on the error state equation, an error covariance matrix is ​​established; based on the state estimation, the system error parameters and the error covariance matrix, the position and posture information of the aerial flight equipment and the ground mobile equipment is obtained; The aerial flight device obtains the initial position and ground environment information of the dynamic target, and determines a dynamic tracking path in combination with a path planning method; According to the environmental map, the position information of the aerial flying device and the ground mobile device and the dynamic tracking path, the aerial flying device and the ground mobile device are controlled in real time to achieve air-ground collaboration to complete the dynamic target tracking task.

2. The dynamic target intelligent tracking method based on the air-ground unmanned collaborative system according to claim 1 is characterized in that: The method further comprises: The outer side of the top surface of the ground mobile device is provided with an AURCO code; The aerial flying device collects pictures including the outer side of the top surface of the ground mobile device through a camera; Target detection is performed on the image. If the Aurco code is detected, the observation value is determined by reading the internal code of the Aurco code and according to the position information and attitude information of the ground mobile device relative to the camera of the aerial device obtained by using the visual odometer, thereby determining the observation equation.

3. The dynamic target intelligent tracking method based on the air-ground unmanned collaborative system according to claim 2 is characterized in that: The system error parameters include a measurement position error and an attitude estimation error; the method further includes: introducing Gaussian measurement noise into the position information and the attitude information to obtain the measured values ​​of the position measurement value and the attitude measurement value, and using the position measurement value and the attitude measurement value as observation values; The Gaussian white noise introduced is expressed as: n m =[n p n q ] T ; Among them, n m is a Gaussian white noise vector, n p is the position measurement white noise, n q is the attitude measurement white noise, n p The mean is 0 and the variance is Gaussian distribution, n q The mean is 0 and the variance is Gaussian distribution.

4. The dynamic target intelligent tracking method based on the air-ground unmanned collaborative system according to claim 1 is characterized in that: The prediction process of the extended Kalman filter is used to predict the system state at the next moment based on the current state estimate and the kinematic state equation, and update the state covariance matrix to estimate the uncertainty of the state estimate at the next moment; The correction process of the extended Kalman filter is used to calculate the predicted observation value based on the predicted system state at the next moment, compare it with the actual measurement value, calculate the Kalman gain using the state covariance matrix according to the comparison result, and then update the state estimation and the state covariance matrix to make the state estimation more accurate and timely reflect the latest uncertainty.

5. The dynamic target intelligent tracking method according to claim 4 is characterized in that: The method comprises: in the prediction process of the extended Kalman filter, using the covariance matrix of the noise to update the state covariance matrix, specifically comprising: Using IMU inertial sensors to measure the acceleration and angular velocity of the aerial flying device and the ground moving device; Determine the relationship between the true value and the measured value of the acceleration and angular velocity, expressed as: Among them, ω is the true angular velocity, ω m is the measured angular velocity, b Θ is the angular velocity deviation, n Θ is the random walk noise of angular velocity, a is the real acceleration, a m is the measured acceleration, b a is the acceleration deviation, n a is the random walk noise of acceleration; According to the relationship and the kinematic state equation, the difference between the state value and the true value is taken as the state error; Determine a differential equation of a continuous-time error, solve the partial derivatives of the differential equation, and obtain an error state equation, wherein the error state equation includes a state transfer matrix and a noise matrix; According to the error state equation, the covariance matrix of the noise is obtained; △t is used as the integration time interval, and discretization processing is performed based on the state transfer matrix and the noise matrix respectively. Then, the Riccati equation is used to calculate the state covariance matrix corresponding to a given time point and used as the updated state covariance matrix, so as to achieve effective estimation of the state quantity.

6. The dynamic target intelligent tracking method based on the air-ground unmanned collaborative system according to claim 1 is characterized in that: For the aerial flying device or the ground mobile device, the state vector is as follows: in, is the state vector, Represents the position of the three axes in the world coordinate system. Represents the speed of the three axes in the world coordinate system, Represents the posture of the three axes in the world coordinate system. and There are three degrees of freedom (x, y, z). represents the angular velocity deviation, b a Represents the acceleration deviation.

7. The method for intelligent tracking of dynamic targets based on an air-ground unmanned collaborative system according to claim 6 is characterized in that: The kinematic state equation is expressed as: in, For the quaternion The corresponding rotation matrix is, Indicated in The attitude of the three axes under acceleration a and in the world coordinate system The corresponding rotation matrix, g represents the acceleration constant, n bω With n ba represent the random walk noise of acceleration and angular velocity respectively, and ω is the angular velocity.

8. The dynamic target intelligent tracking method according to claim 1, characterized in that: Obtaining the position and posture information of the aerial flight device and the ground mobile device according to the state estimation, the system error parameter, and the error covariance matrix, including: Solving the gradient vector and the gradient-related covariance matrix according to the state estimate, the system error parameter, and the partial derivative of the system error parameter; Solving the Kalman gain according to the gradient vector and the gradient-related covariance matrix; The a posteriori state estimator is solved according to the Kalman gain and the error estimate, the error covariance matrix is ​​solved to obtain the error of the a posteriori state estimator, and finally the position and posture information of the aerial flying device and the ground mobile device is obtained.

9. Dynamic target intelligent tracking system based on air-ground unmanned collaborative system, characterized by: The air-ground unmanned collaborative system includes an aerial flying device and a ground mobile device, and the system includes: A parameter acquisition module, used to acquire kinematic information of the aerial flying device and the ground moving device; A model building module, used to build an environmental map based on the collected ground image and obtain the initial position of the dynamic target; determine the state vector of the air-ground unmanned collaborative system according to the kinematic information; establish the state equation and observation equation of the air-ground unmanned collaborative system according to the state vector; extract the kinematic state equation based on the state equation; based on the kinematic state equation and the observation equation, continuously update the state estimation and state covariance matrix of the air-ground unmanned collaborative system through the prediction process and correction process of the extended Kalman filter; establish the error state equation of the air-ground unmanned collaborative system according to the kinematic state equation and the system error parameters; establish the error covariance matrix according to the error state equation; obtain the position and posture information of the aerial flight equipment and the ground mobile equipment according to the state estimation, the system error parameters and the error covariance matrix; A path planning module, used for the aerial flight device to obtain the initial position and ground environment information of the dynamic target, and determine a dynamic tracking path in combination with a path planning method; The collaborative control module is used to control the aerial device and the ground mobile device in real time according to the environmental map, the position information of the aerial device and the ground mobile device and the dynamic tracking path, so as to achieve air-ground collaborative completion of the dynamic target tracking task.

10. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, a dynamic target intelligent tracking method based on an air-to-ground unmanned collaborative system as described in any one of claims 1-8 is implemented.

Images