Unmanned aerial vehicle and unmanned vehicle collaborative operation method and system

Abstract

The invention belongs to the technical field of air-ground cooperative control, and particularly relates to an unmanned aerial vehicle and unmanned vehicle cooperative operation method and system. When the unmanned aerial vehicle preferentially finds the target, guiding the unmanned vehicle to run to the target; when the unmanned vehicle preferentially finds the target, according to the coordinate of the target in the camera coordinate system of the unmanned vehicle, carrying out coordinate conversion on the target according to the sequence of the camera coordinate system of the unmanned vehicle, a vehicle body coordinate system, an east-north-sky coordinate system taking the initial position of the unmanned vehicle as the origin of coordinates and an ECEF coordinate system to obtain the coordinate of the target in the ECEF coordinate system; the unmanned aerial vehicle obtains coordinates of the unmanned aerial vehicle in a global coordinate system and converts the coordinates into an ECEF coordinate system, coordinates of the target in an east-north-sky coordinate system with the initial position of the unmanned aerial vehicle as the origin of coordinates are calculated by combining the coordinates of the target in the ECEF coordinate system, and the unmanned aerial vehicle flies to the target according to the coordinates. According to the invention, bidirectional cooperation is realized, the unmanned vehicle can guide the unmanned aerial vehicle, the unmanned aerial vehicle can also guide the unmanned vehicle, and the robustness and task completion efficiency of the system are improved.

Description

Technical Field

[0001] This invention belongs to the field of air-ground collaborative control technology, specifically relating to a method and system for collaborative operation of unmanned aerial vehicles and unmanned vehicles. Background Technology

[0002] In recent years, with the rapid development of unmanned vehicle and drone technologies, air-ground collaborative systems have shown great potential in disaster relief, logistics delivery, military reconnaissance, and smart cities. In complex and ever-changing operational environments, drones need to acquire environmental information over a wide area to perform macro-situational awareness of target areas, while unmanned vehicles need to perceive targets at close range with high precision to complete detailed tasks such as material delivery and target tracking. This places higher demands on the system's multimodal perception capabilities and the mutual positioning of drones and unmanned vehicles. However, existing technologies have some shortcomings in achieving mutual positioning between unmanned vehicles and drones to complete collaborative operations, limiting the application effectiveness and efficiency of air-ground collaborative operations.

[0003] First, existing technologies lack effective mutual positioning mechanisms, making it difficult for drones and unmanned vehicles to obtain each other's position and motion status information in real time and accurately. This leads to positional deviations during collaborative operations, hindering accurate coordination. Second, under traditional navigation methods, the dynamic boundary information generated by drones and unmanned vehicles during environmental exploration and operations cannot be effectively integrated, resulting in a lack of information exchange and collaborative optimization, and consequently, low operational efficiency.

[0004] Moreover, most existing technologies involving collaboration between drones and unmanned vehicles (UAVs) involve the drone as the main control unit, constructing a map and identifying targets before transmitting the target's location and path information to the UAV. The UAV then uses this information for autonomous navigation, which is a drone-guided UAV operation mode. For example, the air-ground collaborative intelligent assisted driving navigation method disclosed in Chinese invention patent application CN118408545A, published on July 30, 2024, is such a method. Specifically, the drone creates a 3D map in a world coordinate system and converts it into a 2D grid map; the drone detects whether a specified target exists in the ground image, and if so, sends the 2D grid map and the first location information of the specified target in the 2D grid map to the UAV; based on the first location information and the UAV's second location information in the 2D grid map, the UAV determines a navigation path to the specified target using a path planning algorithm; the UAV then controls itself to travel along the navigation path using a trajectory tracking algorithm. Thus, in complex environments, the cooperation between drones and unmanned vehicles (UAVs) enables UAVs to quickly and autonomously navigate to designated target locations, improving the efficiency of UAV mission execution. Clearly, the reverse approach—where UAVs guide drones—is rarely used, but it is currently essential in security and emergency response scenarios. For example, UAVs can be responsible for assessing and initially locating dangerous areas, while drones perform the tasks; this combination improves the efficiency of safety and emergency response missions.

[0005] Therefore, it is essential to find a way to effectively enable unmanned vehicles to guide drones. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for collaborative operation of unmanned aerial vehicles (UAVs) and unmanned vehicles (UAVs), in order to solve the problem of low task efficiency caused by the existing operation mode of using only UAVs to guide UAVs.

[0007] To address the aforementioned technical problems, this invention provides a technical solution for a collaborative operation method between unmanned aerial vehicles (UAVs) and unmanned vehicles (UAVs), specifically including: The present invention provides a method for collaborative operation of unmanned aerial vehicles and unmanned vehicles, the method comprising: When the drone detects the target first, it guides the unmanned vehicle to the target. When the autonomous vehicle detects a target, it transforms the target's coordinates in the autonomous vehicle's camera coordinate system, then in the vehicle's coordinate system, then in the northeast-sky coordinate system with the autonomous vehicle's initial position as the origin, and finally in the geocentric coordinate system, to obtain the target's coordinates in the geocentric coordinate system. The UAV then acquires its own coordinates in the global coordinate system and transforms them to the geocentric coordinate system. It then combines these coordinates with the target's coordinates in the geocentric coordinate system to calculate the target's coordinates in the northeast-sky coordinate system with the UAV's initial position as the origin. Finally, the UAV flies to the target based on these coordinates in the northeast-sky coordinate system with the UAV's initial position as the origin.

[0008] The beneficial effects of the above technical solution are as follows: The UAV and unmanned vehicle collaborative operation method of the present invention can not only realize the operation mode of UAV guiding unmanned vehicle, but also realize the operation mode of unmanned vehicle guiding UAV, and can take into account the collaborative closed loop between systems. In the specific implementation process, after the unmanned vehicle first detects the target, it obtains the target's coordinates in the geocentric coordinate system through layers of coordinate transformation. Then, the UAV can obtain the target's coordinates in the global coordinate system and transform them to the geocentric coordinate system. Combined with the target's coordinates in the geocentric coordinate system, the target's coordinates in the northeast-central coordinate system with the UAV itself as the origin can be obtained, realizing centimeter-level relative positioning and guiding it to accurately drive to the target position. This method can still maintain stable spatial perception and navigation capabilities in complex outdoor environments. Thus, a two-way collaborative mechanism is realized as a whole. The unmanned vehicle can guide the UAV, and the UAV can also guide the unmanned vehicle, improving the robustness of the system and the efficiency of task completion.

[0009] Furthermore, the autonomous vehicle transforms the target from the vehicle coordinate system to the Northeast-Sky coordinate system with the initial position of the autonomous vehicle as the origin by: transforming the target from the vehicle coordinate system to the Northeast-Sky coordinate system with the initial position of the autonomous vehicle as the origin based on the IMU pose and GPS data of the autonomous vehicle.

[0010] Furthermore, a publish-subscribe communication mechanism is used between the drone and the unmanned vehicle. Accordingly, the unmanned vehicle publishes the coordinates of the target in the geocentric coordinate system to the corresponding topic for the drone to receive.

[0011] Furthermore, when the drone detects the target first, the method for guiding the unmanned vehicle to the target is as follows: when the drone locks onto the target, it hovers above the target and then transforms its own coordinates in the Northeast Celestial Coordinate System with its initial position as the origin to the Earth-centered Earth-fixed Coordinate System, and sends them to the unmanned vehicle; the unmanned vehicle obtains its own coordinates in the global coordinate system and transforms them to the Earth-centered Earth-fixed Coordinate System, calculates the coordinate difference between the unmanned vehicle and the drone in the Earth-centered Earth-fixed Coordinate System, and then converts this coordinate difference into an offset in the Northeast Celestial Coordinate System with the unmanned vehicle's initial position as the origin. Based on this offset, the drone's coordinates in the Northeast Celestial Coordinate System with the unmanned vehicle's initial position as the origin are obtained and used as the target to guide the unmanned vehicle.

[0012] Furthermore, the method also includes: after the UAV takes off, the UAV creates a rectangular area in front of its own orientation and divides the rectangular area into grids, storing the coordinates of the center point of each grid in the UAV's local coordinate system; then the UAV performs path planning based on the stored coordinates of the center points arranged in an S-shaped path; the UAV's local coordinate system is a northeast-sky coordinate system with the UAV's initial position as the origin.

[0013] Furthermore, the drone locks onto the target by using the AprilTag visual recognition algorithm. When the drone detects the target and the error between the target's position in the image and the image center is within the required threshold range, the target is determined to be locked.

[0014] Furthermore, when the UAV detects a target but the error between the target's position in the image and the image center is outside the required threshold range, PID single closed-loop control is used to adjust the UAV's pose until the error between the target's position in the image and the image center is within the required threshold range.

[0015] To address the aforementioned technical problems, the present invention also provides a technical solution for a collaborative operation system of unmanned aerial vehicles and unmanned vehicles, specifically including: The present invention discloses a collaborative operation system of unmanned aerial vehicles (UAVs) and unmanned vehicles (UAVs), which are connected by communication. The UAV is used to guide the UAV to the target when it has priority in detecting the target. The UAV is also used to obtain its own coordinates in a global coordinate system and transform them to a geocentric coordinate system when the UAV has priority in detecting the target. Then, it calculates the target's coordinates in a northeast-sky coordinate system with the UAV's initial position as the origin by combining the target's coordinates in the geocentric coordinate system with the target's coordinates in the northeast-sky coordinate system with the UAV's initial position as the origin, and flies to the target according to the target's coordinates in the northeast-sky coordinate system with the UAV's initial position as the origin. When the unmanned vehicle first detects a target, it performs coordinate transformation on the target in the following order: unmanned vehicle camera coordinate system, vehicle body coordinate system, northeast-sky coordinate system with the unmanned vehicle's initial position as the origin, and geocentric coordinate system, to obtain the target's coordinates in the geocentric coordinate system.

[0016] The beneficial effects of the above technical solution are as follows: The UAV and unmanned vehicle collaborative operation system of the present invention can not only realize the operation mode of UAV guiding unmanned vehicle, but also realize the operation mode of unmanned vehicle guiding UAV, and can take into account the collaborative closed loop between systems. In the specific implementation process, after the unmanned vehicle first detects the target, it obtains the target's coordinates in the geocentric coordinate system through layers of coordinate transformation. Then, the UAV can obtain the target's coordinates in the global coordinate system and transform them to the geocentric coordinate system. Combined with the target's coordinates in the geocentric coordinate system, the target's coordinates in the northeast-central coordinate system with the UAV itself as the origin can be obtained, realizing centimeter-level relative positioning and guiding it to accurately drive to the target position. This method can still maintain stable spatial perception and navigation capabilities in complex outdoor environments. Thus, a two-way collaborative mechanism is realized as a whole. The unmanned vehicle can guide the UAV, and the UAV can also guide the unmanned vehicle, improving the robustness of the system and the efficiency of task completion.

[0017] Furthermore, the drones and unmanned vehicles communicate with each other via RTK base stations and wireless communication links.

[0018] Furthermore, the wireless communication between the drones and unmanned vehicles adopts a publish-subscribe communication mechanism. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the unmanned vehicle in the system of the present invention; Figure 2 This is a schematic diagram of the drone in the system of the present invention; Figure 3 This is a schematic diagram of RTK positioning in the system of the present invention; Figure 4 This is a schematic diagram of the remote control for the unmanned vehicle in the system of the present invention; Figure 5 This is a schematic diagram of the drone remote controller in the system of the present invention; Figure 6 This is a schematic diagram of the Minihome system in this invention; Figure 7 This is a schematic diagram of the AR code used in the system of the present invention; Figure 8 This is a flowchart of the collaborative operation method of drones and unmanned vehicles of the present invention; Figure 9 This is a control block diagram of the PID control of the present invention; Figure 10 This is the control loop diagram of the PID control of the present invention; Figure 11 This is a communication diagram of the UAV and unmanned vehicle nodes of the present invention; Figure 12 This is a schematic diagram of coordinate transformation according to the present invention; Figure 13-1 This is a target image detected by the drone of this invention; Figure 13-2 This is a target lock diagram of the UAV of the present invention; Figure 14-1 This is the unmanned vehicle path planning diagram of the present invention; Figure 14-2 This is a diagram showing the path planning results of the unmanned vehicle according to the present invention; Figure 15-1 This is the target recognition image for unmanned vehicles according to the present invention; Figure 15-2 This is a diagram showing the target recognition results of the unmanned vehicle according to the present invention; Figure 16-1 This is the drone flight planning diagram of the present invention; Figure 16-2 This is a diagram showing the UAV flight planning results of the present invention. Detailed Implementation

[0020] The UAV and unmanned vehicle collaborative operation method and system of the present invention can not only realize the operation mode of UAV guiding unmanned vehicle, but also realize the operation mode of unmanned vehicle guiding UAV, realizing a fundamental leap from "one-way navigation" to "two-way collaboration", and can take into account the collaborative closed loop between systems. When the UAV first detects the target, it guides the unmanned vehicle to drive to the target, and when the unmanned vehicle first detects the target, it guides the UAV to fly to the target, thus constructing a two-way mutually guiding air-ground collaborative framework. This allows the unmanned vehicle to also act as a perception and guidance node, actively sending target information back to the UAV, thereby guiding it to fly to the target area. This achieves equal interaction between information flow and control flow, improving the robustness of the system and the efficiency of task completion. In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0021] An implementation method for a collaborative operation system of drones and unmanned vehicles: The entire drone and unmanned vehicle collaborative operation system deploys drones and unmanned vehicles, which communicate wirelessly. The drones are controlled by gimbals for attitude adjustment and are equipped with cameras capable of capturing images and identifying targets within their angular field of view. Similarly, the unmanned vehicles are also equipped with cameras, capable of capturing images and identifying targets within their angular field of view. Figures 1-7The diagram shows the various components of the entire system, including the unmanned vehicle, etc. Figure 1 As shown, the drone Figure 2 As shown, Minihome is as follows Figure 6 As shown.

[0022] To implement the method of this invention, a target object (or target) is set up, mainly for detection, and can be encoded using AprilTag, such as... Figure 7 As shown, this invention provides a method for collaborative operation between drones and unmanned vehicles (UAVs) to locate targets. The system primarily controls drones and UAVs to automatically cruise, search for, and locate targets within a defined area. Once one detects a target, it sends a signal to the other for coordinated air-ground operations. The system constructs a bidirectional, mutually guiding air-ground system framework, allowing the UAV to also act as a perception and guidance node, actively sending target information back to the drone to guide it to the target area. This achieves equal interaction between information flow and control flow, improving system robustness and task completion efficiency.

[0023] To enable wireless communication between drones and unmanned vehicles, the system is equipped with RTK base stations (real-time dynamic differential base stations) and wireless communication links. An RTK positioning diagram is shown below. Figure 3 As shown. The role of the RTK base station is to correct the GPS data and improve the positioning accuracy of the drone and the unmanned vehicle; the wireless communication link is used to enable communication between the drone and the unmanned vehicle. By establishing a real-time pose information exchange mechanism between the drone and the unmanned vehicle through the RTK base station and the wireless communication link, centimeter-level relative positioning between the drone and the unmanned vehicle can be achieved, enabling both parties to accurately obtain each other's positions in dynamic environments and supporting stable collaborative operations.

[0024] Furthermore, in order to allow for manual intervention when necessary, the system is also equipped with a remote control. The diagrams of the unmanned vehicle and the drone are shown below. Figure 4 , 5 As shown, the remote control is used to make human intervention when drones and unmanned vehicles are in danger.

[0025] This system employs a unified node management, message subscription, and publish mechanism design within the ROS and MAVROS frameworks, ensuring high synergy and scalability among its modules (perception, pathfinding, recognition, and execution), facilitating engineering implementation and algorithm iteration. The system utilizes coordinate systems including global coordinates (GPS coordinates, latitude, longitude, and altitude), geocentric coordinates (ECEF coordinates), and camera coordinates. The entire system possesses the following functions: System initialization and communication mechanism: The program initializes ROS nodes, sets up multiple subscribers and publishers, and enables UAVs and unmanned vehicles to acquire each other's operational pose, position information, and target detection results in real time, and publish motion control, target coordinates, gimbal control, and search completion signals, thus establishing a stable data interaction framework. Furthermore, communication between UAVs and unmanned vehicles adopts a publish-subscribe communication mechanism to achieve data interaction. Figure 11 This is a communication diagram between drone and unmanned vehicle nodes. From... Figure 11 As can be seen, drones and unmanned vehicles can communicate and exchange information and topics. The drone is controlled by target_search ( Figure 11 (Top left part) Find the target and send it via / search / result, / distributegoal transforms the coordinates, move_base controls path planning, and cmd_vel performs the movement.

[0026] Parameter Loading and Map Division: The UAV reads map dimensions, camera field of view, and flight altitude from the parameter server and calculates a reasonable map division method. An "S"-shaped path strategy is used to divide the flight area into several sub-blocks, and the center point coordinates of each block are calculated (based on the current UAV attitude transformation). The UAV autonomously explores forward in the direction of UAV flight. The "S"-shaped coverage path algorithm used here ensures full coverage of the flight area and path continuity. Existing technologies that use FAST-LIO to create a 3D map followed by filtering and rasterization, while offering spatial accuracy, have high computational complexity and are unsuitable for lightweight operation scenarios. This invention employs a task parameter-driven "S"-shaped path division method to partition the flight area into gridded sub-blocks, balancing path coverage and real-time operation. With simultaneous mapping and obstacle avoidance capabilities, dynamic search and target recognition are achieved, effectively reducing system computational resource consumption and improving overall operational efficiency and task execution stability.

[0027] Waypoint planning and path generation: After the drone (unmanned vehicle) discovers the target and obtains the target coordinate information, it sends it to the unmanned vehicle (drone). The unmanned vehicle will autonomously plan a path and avoid obstacles to drive to the target (the drone will autonomously plan waypoints and avoid obstacles to fly to the target).

[0028] Target detection and dynamic locking mechanism: The UAV uses the AprilTag visual recognition algorithm to detect targets. Once a target is detected, if its position in the image is centered (within a set threshold error), the target is considered locked, and the UAV hovers above it, sending a detection success signal to the vehicle. If the target deviates from the image center, PID control adjusts the flight attitude, guiding the UAV to continuously adjust its position to center the target, improving positioning accuracy and stability. The mathematical expression for PID control is: , u ( t ) represents the control output (the drone's speeds Vx and Vy). e ( t ) indicates positional deviation. e ( t )=0.5- c ( t ), c ( t ) indicates the target location (c x ,c y ), K p This represents a scaling factor used for rapid response. K i Indicates the integral coefficient, used to eliminate steady-state error. K d This represents the derivative coefficient, used to suppress overshoot. The overall control block diagram and control loop are shown below. Figure 9 and Figure 10 As shown. It should be noted that the calculation of positional deviation... e ( t The 0.5 value used is the ideal center position of the image center in normalized coordinates. In the image coordinate system, pixel coordinates are usually normalized to the [0,1] interval, so the coordinates of the image center point are (X... center ,Y center )=(0.5,0.5). Therefore, when the target detection coordinates When the value is close to (0.5, 0.5), it indicates that the target is located in the center of the image, and the error is minimal. e ( t The value () represents the degree of target deviation from the center. After the autonomous vehicle detects the target, it displays the coordinate system and the target's pose information, sending this pose information to the drone to confirm successful detection. Here, visual center deviation detection combined with a PID attitude control algorithm is used, enabling the drone to achieve fine adjustments and stable hovering while locking onto the AprilTag target. That is, the drone-vehicle collaborative operation falls into two categories: the first is where the drone prioritizes target detection and the autonomous vehicle autonomously drives to the target point; the second is where the autonomous vehicle prioritizes target detection and the drone autonomously drives to the target point. The processes for both scenarios are as follows: Figure 8 As shown in the diagram, the coordinate transformation involved is illustrated below. Figure 12 As shown in the figure, this diagram illustrates coordinate transformations for various sensors, including cameras, radar, and IMUs, to the map coordinate system (GPS coordinate system).

[0029] The first scenario: After the drone powers on, it receives initial GPS coordinates, which serve as the origin of the ENU coordinate system. Within this ENU coordinate system, the drone flies in an S-shaped pattern. After takeoff, it creates a rectangular area in front of itself and divides this area into a grid, storing the grid points in a two-dimensional array. These grid points are then arranged in an S-shaped pattern and packaged into a `geometry_msgs::PoseArray` for drone path planning and publishing, ensuring path continuity and coverage integrity. The points stored and published here are all world coordinates within the ENU coordinate system, with the drone's initial position as the origin. When the drone, flying in an S-shaped pattern and controlled by the gimbal, locks onto a target, it hovers above the target. At this point, an ENU coordinate system is generated, with the drone's initial ENU coordinates as the origin and the target lock offset as the offset. This coordinate system is converted to ECEF (the target ECEF) and published to the drone, ending the drone's operation. After receiving the ECEF information, the autonomous vehicle uses GeographicLib (a C++ library for solving geospatial computing problems) to convert its GPS (latitude, longitude, and altitude) coordinates (i.e., coordinates in the global coordinate system) into three-dimensional coordinates in the ECEF coordinate system. Then, it calculates the coordinate difference between the target's ECEF coordinates and the autonomous vehicle's own ECEF coordinates, further converting this difference into an offset in the ENU coordinate system with the autonomous vehicle's initial position as the origin, ultimately obtaining the target's local coordinates relative to the autonomous vehicle. For example... Figure 13-1 , 13-2 As shown in Figures 14-1 and 14-2, this process utilizes a high-precision RTK base station in conjunction with GeographicLib to achieve a triple coordinate system transformation from GPS to ECEF to ENU, enabling centimeter-level relative positioning. This method is far superior to existing technologies that obtain target location information by transforming between the camera coordinate system and the world coordinate system, the latter being inaccurate under conditions such as strong light and obstruction. The solution of this invention can maintain stable spatial perception and navigation capabilities in complex outdoor environments.

[0030] The second scenario: The autonomous vehicle explores autonomously using vision. When the camera detects a target, it uses camera intrinsics to estimate the target's pose and plots a 3D coordinate system to display the pose. The target's marked position (coordinates in the camera coordinate system) is published to the `aruco_position` topic in `geometry_msgs::Point` type. The autonomous vehicle fuses the target's pose information, its IMU pose information, and its GPS data. Using the vehicle's pose information, it transforms the marker from the vehicle coordinate system to the ENU coordinate system (with the initial position of the autonomous vehicle as the origin, which is also the power-on position). After obtaining the ENU coordinates, it transforms them to the ECEF coordinate system and publishes them to the ` / aruco_global_position` topic in `sensor_msgs::NavSatFix` format for the drone to receive. The drone subscribes to topics related to its current GPS, target ECEF, local pose, and flight status. It then converts the drone's GPS coordinates to the ECEF coordinate system, calculates the coordinate difference between the target's ECEF and the drone's ECEF, and further converts this difference into an offset in the ENU coordinate system with the drone's own startup position as the origin. This yields the target's local coordinates relative to the drone. After obtaining these local coordinates, the drone flies to the target in OFFBOARD mode (a flight mode in the PX4 flight controller). Figure 15-1 , 15-2 As shown in Figures 16-1 and 16-2. The local pose refers to the direction the drone's nose faces when it reaches the target area; subscribing to the unmanned vehicle's pose ensures the drone and vehicle face the same direction. Flight state refers to the drone's flight towards the target in OFFBOARD mode among various modes (stationary, manual, self-stabilized, OFFBOARD, etc.).

[0031] An implementation method for collaborative operation of drones and unmanned vehicles: This invention discloses a method for collaborative operation between unmanned aerial vehicles (UAVs) and unmanned vehicles (UAVs). The core of this method lies in achieving high-precision mutual positioning and task collaborative control between heterogeneous robotic systems (including UAVs and UAVs). This method establishes a real-time pose information interaction mechanism between the UAV and the UAV through an RTK base station and a wireless communication link, enabling both to accurately acquire each other's positions in dynamic environments and supporting stable collaborative operation. The UAV performs map division and "S"-shaped path planning based on task area parameters, performs a comprehensive search, and detects target objects (AprilTags) in real time. Once a target is locked, its pose information is sent to the UAV to assist in path planning and navigation. Conversely, when the UAV detects a target first, it can also feed back the target pose to the UAV, guiding it to fly precisely to the target area. Through this two-way pose perception and mutual guidance mechanism, precise collaboration and task division between the UAV and the UAV are achieved, significantly enhancing the system's autonomous perception, navigation, and dynamic response capabilities in complex scenarios.

[0032] In the first scenario, upon powering on, the drone receives initial GPS coordinates, which serve as the origin of the ENU coordinate system. Within this ENU coordinate system, the drone flies in an S-shaped pattern. After takeoff, it creates a rectangular area in front of itself and divides this area into a grid, storing the grid points in a two-dimensional array. These grid points are then arranged in an S-shaped pattern and packaged into a `geometry_msgs::PoseArray` for drone path planning and deployment. The points stored and deployed here are all world coordinates within the ENU coordinate system, with the drone's initial position as the origin. When the drone, flying in an S-shaped pattern and controlling the gimbal, detects a target, it flies over the target. At this point, an ENU coordinate system is generated, with the drone's initial ENU coordinates as the origin and the target's offset as the offset. This coordinate system is converted to ECEF (the target ECEF) and deployed to the drone, ending the drone's operation. After receiving the ECEF information, the autonomous vehicle uses GeographicLib (a C++ library for solving geospatial computing problems) to convert its GPS (latitude, longitude, and altitude) coordinates (i.e., coordinates in the global coordinate system) into three-dimensional coordinates in the ECEF coordinate system. Then, it calculates the coordinate difference between the target's ECEF coordinates and the autonomous vehicle's own ECEF coordinates, and further converts this coordinate difference into an offset in the ENU coordinate system with the autonomous vehicle as the origin, finally obtaining the target's local coordinates relative to the autonomous vehicle.

[0033] In the second scenario, the autonomous vehicle (RV) explores autonomously using vision. When the camera detects a target, it uses camera intrinsics to estimate the target's pose and plots a 3D coordinate system to display the pose. The location of the target marker is then published to the `aruco_position` topic in `geometry_msgs::Point` format. The RV fuses the target's pose information, its IMU pose information, and its GPS data. Using the RV's pose information, it transforms the marker from the vehicle coordinate system to the ENU (East-North-Up) coordinate system and publishes it to the ` / aruco_global_position` topic in `sensor_msgs::NavSatFix` format for the drone to receive. The drone subscribes to topics related to its current GPS, target ECEF, local pose, and flight status. It then converts the drone's GPS coordinates to the ECEF coordinate system, calculates the coordinate difference between the target ECEF and the drone's ECEF, and further converts this difference into an offset in the ENU coordinate system with the drone's own power-on position as the origin. This gives the target's local coordinates relative to the drone. Once the drone obtains these local coordinates, it flies to the target in OFFBOARD mode (a flight mode in the PX4 flight controller).

[0034] In summary, this invention not only achieves a fundamental leap from "one-way navigation" to "two-way collaboration" in terms of cooperation methods, but also forms a unique and practical technical system in key dimensions such as pose fusion, map processing, control mechanisms, and software architecture. It can be widely applied to multi-robot collaborative task scenarios, such as disaster relief, material search, and precision agriculture. The proposed technical solution effectively enhances the autonomous perception and collaborative navigation capabilities of the air-ground collaborative system in dynamic and unstructured environments, significantly improving the information fusion efficiency and path planning accuracy between multiple platforms. While ensuring real-time performance, it optimizes path redundancy and computational resource consumption during collaborative task execution, overcoming the response latency issues of existing air-ground collaborative systems.

Claims

1. A method for collaborative operation of unmanned aerial vehicles (UAVs) and unmanned vehicles, characterized in that, The method includes: When the drone detects the target first, it guides the unmanned vehicle to the target. When the autonomous vehicle detects a target, it transforms the target's coordinates in the autonomous vehicle's camera coordinate system, then in the vehicle's coordinate system, then in the northeast-sky coordinate system with the autonomous vehicle's initial position as the origin, and finally in the geocentric coordinate system, to obtain the target's coordinates in the geocentric coordinate system. The UAV then acquires its own coordinates in the global coordinate system and transforms them to the geocentric coordinate system. It then combines these coordinates with the target's coordinates in the geocentric coordinate system to calculate the target's coordinates in the northeast-sky coordinate system with the UAV's initial position as the origin. Finally, the UAV flies to the target based on these coordinates in the northeast-sky coordinate system with the UAV's initial position as the origin.

2. The method for collaborative operation of unmanned aerial vehicles and unmanned vehicles according to claim 1, characterized in that, The autonomous vehicle transforms the target from the vehicle coordinate system to the Northeast-Sky coordinate system with the initial position of the autonomous vehicle as the origin by: transforming the target from the vehicle coordinate system to the Northeast-Sky coordinate system with the initial position of the autonomous vehicle as the origin based on the IMU pose and GPS data of the autonomous vehicle.

3. The method for collaborative operation of unmanned aerial vehicles and unmanned vehicles according to claim 1, characterized in that, The communication mechanism between the drone and the unmanned vehicle adopts a publish-subscribe mechanism. Accordingly, the unmanned vehicle publishes the coordinates of the target in the geocentric coordinate system to the corresponding topic, which is received by the drone.

4. The method for collaborative operation of unmanned aerial vehicles and unmanned vehicles according to claim 1, characterized in that, When the drone detects the target first, it guides the unmanned vehicle to the target in the following way: when the drone locks onto the target, it hovers above the target and then transforms its own coordinates in the Northeast Sky Coordinate System with its initial position as the origin to the Earth-centered Earth-fixed Coordinate System and sends them to the unmanned vehicle. The autonomous vehicle acquires its own coordinates in the global coordinate system and transforms them to the geocentric coordinate system. It calculates the coordinate difference between the autonomous vehicle and the drone in the geocentric coordinate system, and then converts the coordinate difference into the offset in the northeast-sky coordinate system with the initial position of the autonomous vehicle as the origin. Based on the offset, it obtains the coordinates of the drone in the northeast-sky coordinate system with the initial position of the autonomous vehicle as the origin and uses it as a target to guide the autonomous vehicle.

5. The method for collaborative operation of unmanned aerial vehicles and unmanned vehicles according to claim 4, characterized in that, The method further includes: after the UAV takes off, the UAV creates a rectangular area in front of its own orientation and divides the rectangular area into grids, storing the coordinates of the center point of each grid in the UAV's local coordinate system; then the UAV plans its path based on the stored coordinates of the center points arranged in an S-shaped path; the UAV's local coordinate system is a northeast-sky coordinate system with the UAV's initial position as the origin.

6. The method for collaborative operation of unmanned aerial vehicles and unmanned vehicles according to claim 4, characterized in that, The drone locks onto the target by using the AprilTag visual recognition algorithm. When the drone detects the target and the error between the target's position in the image and the center of the image is within the required threshold range, the target is determined to be locked.

7. The method for collaborative operation of unmanned aerial vehicles and unmanned vehicles according to claim 6, characterized in that, When the UAV detects a target but the error between the target's position in the image and the image center is outside the required threshold range, PID single closed-loop control is used to adjust the UAV's pose until the error between the target's position in the image and the image center is within the required threshold range.

8. A collaborative operation system for unmanned aerial vehicles (UAVs) and unmanned vehicles (UAVs), comprising a UAV and an UAV, wherein the UAV and the UAV are connected by communication, characterized in that, The drone is used to guide the unmanned vehicle to the target when it has priority in detecting the target; it is also used to obtain its own coordinates in the global coordinate system and transform them to the geocentric coordinate system when the unmanned vehicle has priority in detecting the target, and then calculate the coordinates of the target in the northeast-sky coordinate system with the initial position of the drone as the origin by combining the coordinates of the target in the geocentric coordinate system, and fly to the target according to the coordinates of the target in the northeast-sky coordinate system with the initial position of the drone as the origin. When the unmanned vehicle first detects a target, it performs coordinate transformation on the target in the following order: unmanned vehicle camera coordinate system, vehicle body coordinate system, northeast-sky coordinate system with the unmanned vehicle's initial position as the origin, and geocentric coordinate system, to obtain the target's coordinates in the geocentric coordinate system.

9. The UAV and unmanned vehicle collaborative operation system according to claim 8, characterized in that, The drones and unmanned vehicles communicate with each other via RTK base stations and wireless communication links.

10. The UAV and unmanned vehicle collaborative operation system according to claim 9, characterized in that, The wireless communication between drones and unmanned vehicles adopts a subscription-publishing communication mechanism.

Images