Air-ground visual cooperative target positioning method and system

Abstract

The present application relates to a kind of air-ground vision cooperative target positioning method and system.By respectively obtaining the pose parameter in the uniform coordinate system of airborne, ground imaging unit, double-end image detection matching locks the same target to be observed and extracts two-way observation angle information, conversion generates double observation ray in the uniform coordinate system and completes space intersection to solve target global position estimate, combined with the relative position of target space of aircraft coordinate calculation, it is helpful to rely on air-ground double monocular cooperative observation to realize long-distance target high-precision positioning under the constraint of rigid platform, break through the limitation of insufficient single airborne visual range finding precision, improve the reliability of low-altitude aircraft airspace target perception and obstacle avoidance decision, provide stable and reliable technical support for low-altitude aircraft autonomous flight, airspace intrusion early warning and active obstacle avoidance control.

Description

Technical Field

[0001] This invention relates to the field of visual perception and air-to-ground cooperative positioning technology for low-altitude aircraft, and more specifically, to a ground-to-air visual cooperative target positioning method and system. Background Technology

[0002] With the rapid development of the low-altitude flight industry, the application scenarios of drones, eVTOL, and various low-altitude aircraft are continuously expanding. Throughout operations, these aircraft need to perceive various targets in the airspace in real time and locate them to support basic flight functions such as path planning, visual obstacle avoidance, risk warning, and autonomous flight control. The ability to perceive and locate targets in the airspace directly determines the operational safety of low-altitude aircraft. Camera-based visual perception solutions, with their convenient deployment and controllable hardware costs, have become the mainstream method for monitoring targets in the airspace.

[0003] Currently, airborne visual perception is mainly divided into two categories: monocular vision solutions and airborne binocular / multi-view vision solutions. Monocular cameras are lightweight, have few deployment restrictions, and offer high angle measurement accuracy, making them widely used in various small aircraft. However, monocular hardware lacks direct depth observation capabilities. Airborne binocular and multi-view vision systems rely on the parallax principle to recover target depth information, representing another mainstream ranging and positioning method.

[0004] In the process of realizing this invention, the inventors discovered that both existing airborne visual positioning schemes have inherent defects: monocular vision cannot directly obtain the target distance. When estimating the distance by relying on target size, motion parallax, deep learning, etc., it is easily affected by the target's own attributes, the observation environment, and the aircraft's motion state, resulting in poor distance estimation accuracy and reduced reliability of obstacle avoidance decisions; airborne binocular and multi-view vision have strict constraints on the rigidity of the installation platform, baseline length, camera synchronization and calibration accuracy, and small, lightweight aircraft are limited by load and space, resulting in poor long-distance ranging performance. Both types of schemes are difficult to meet the actual use requirements of lightweight deployment and long-distance high-precision target positioning. Summary of the Invention

[0005] Based on this, and to address the aforementioned problems, this invention provides a ground-to-air visual collaborative target localization method and system. By acquiring the pose parameters of the airborne and ground imaging units in a unified coordinate system, dual-end image detection and matching locks the same target to be observed and extracts the observation angle information from both paths. The system then converts and generates dual observation rays in a unified coordinate system and completes spatial intersection to estimate the global position of the target. Combined with the aircraft coordinates, the system calculates the target's spatial relative position. This helps to achieve high-precision localization of long-distance targets without rigid platform constraints by relying on ground-to-air dual monocular collaborative observation, overcomes the limitations of insufficient accuracy in single-airborne visual ranging, improves the reliability of airspace target perception and obstacle avoidance decision-making for low-altitude aircraft, and provides stable and reliable technical support for autonomous flight, airspace intrusion warning, and active obstacle avoidance control of low-altitude aircraft.

[0006] In a first aspect, the present invention provides a ground-to-air visual cooperative target localization method, the method comprising: acquiring a first pose parameter of a first imaging unit mounted on an aircraft in a unified coordinate system and a second pose parameter of a second imaging unit deployed on the ground in a unified coordinate system; performing target detection on a scene image using the first imaging unit and the second imaging unit to lock onto the same target to be observed; and extracting a first observation angle information of the target to be observed relative to the first imaging unit and a second observation angle information relative to the second imaging unit; based on the first pose parameter and the second pose parameter, converting the first observation angle information into a first observation ray, and converting the second observation angle information into a second observation ray. A second observation ray is generated; the first and second observation rays are spatially intersected in a unified coordinate system to output a global position estimate of the target; based on the global position estimate and the current position of the aircraft carrying the first imaging unit in the unified coordinate system, the position vector of the target relative to the aircraft is obtained; based on the position vector, the distance, azimuth, and altitude difference of the target relative to the aircraft are determined, and the velocity of the target relative to the aircraft and its approach trend relative to the aircraft are determined based on multi-frame continuous global position estimates, and the spatial position of the target relative to the aircraft is generated.

[0007] Optionally, obtaining the first attitude parameters of the first imaging unit mounted on the aircraft in a unified coordinate system and the second attitude parameters of the second imaging unit deployed on the ground in a unified coordinate system includes: pre-constructing a unified coordinate system, wherein the unified coordinate system is selected as a global geodetic coordinate system or a local navigation coordinate system; performing data fusion through multiple types of sensors equipped with the first imaging unit mounted on the aircraft to obtain the first attitude parameters including position, attitude, and timestamp; performing data fusion through ground positioning sensing equipment equipped with the second imaging unit deployed on the ground to obtain the second attitude parameters including position, attitude, and timestamp; and uniformly converting the first attitude parameters and the second attitude parameters to a unified coordinate system and storing them. By pre-constructing a global geodetic coordinate system or a local navigation coordinate system as a unified reference, and fusing data from multiple types of airborne and ground-based sensors, the pose parameters with timestamps at both ends are obtained. After completing a unified coordinate transformation and storage, the position and attitude information of the imaging units at both ends of the airspace and ground are normalized to the same spatial reference. This eliminates the observation reference deviation caused by the inconsistency of the coordinate systems of ground and air devices, ensures the geometric consistency of subsequent observation ray transformation and spatial intersection calculation, provides an accurate and synchronous spatial coordinate basis for ground-air dual-view collaborative positioning, effectively reduces positioning system errors caused by cross-platform observation, and improves the accuracy of global position calculation of airspace targets.

[0008] Optionally, a first imaging unit and a second imaging unit are used to perform target detection on the scene images to lock onto the same target. This includes: acquiring a first spatial scene image through the first imaging unit and a second spatial scene image through the second imaging unit; performing target detection on the first and second spatial scene images respectively, and outputting detection data containing pixel center coordinates; the first and second imaging units interact with the detection data, and pairing and filtering the detected spatial targets based on multi-class matching constraints to lock onto the same target, and reading the pixel center coordinates of the target in the first and second spatial scene images respectively. By independently acquiring spatial images and performing target detection separately through airborne and ground-based imaging units to output the target pixel center coordinates, and then interacting with the detection data across ends and relying on multi-class matching constraints to complete target pairing and filtering, the same target under the dual ground and air perspectives can be accurately locked. This enables the association and matching of the same source target under the condition of uncoupled dual cameras in different locations, avoiding the problem of target confusion and mismatch from different perspectives, and providing the same source target pixel observation data for subsequent angle conversion, observation ray construction and spatial intersection calculation, ensuring the data consistency and positioning robustness of ground-air cooperative positioning.

[0009] In the above implementation process, the detection data exchanged between the first and second imaging units is lightweight detection data, without transmitting the complete original airspace scene image. The lightweight detection data includes target pixel center, observation angle information, timestamp, and imaging unit pose-related parameters. The air and ground ends only exchange lightweight detection data such as target pixel center, observation angle, timestamp, and imaging unit pose, discarding the transmission of the bulky complete original airspace scene image. This can significantly reduce the data transmission volume of the air-to-ground wireless communication link, effectively reduce communication bandwidth occupation and data transmission latency, and improve the real-time performance of target matching and positioning calculation. At the same time, this lightweight data only retains the core parameters required for positioning, adapting to various air-to-ground communication links such as narrowband data transmission and wireless ad hoc networks, lowering the threshold for system hardware communication deployment.

[0010] Optionally, the first observation angle information of the target relative to the first imaging unit and the second observation angle information relative to the second imaging unit are extracted. This includes: retrieving the calibration parameters of the first imaging unit and, in conjunction with the pixel center coordinates of the target in the first spatial scene image, converting them to generate the first observation angle information; retrieving the calibration parameters corresponding to the second imaging unit and, in conjunction with the pixel center coordinates of the target in the second spatial scene image, converting them to generate the second observation angle information. By relying on the pre-stored calibration parameters of the imaging unit and combining them with the target pixel center coordinates to obtain the corresponding observation angle information, the pitch and azimuth observation angles of the target relative to the camera optical axis can be accurately restored based on the geometric relationship of pinhole imaging, giving full play to the inherent advantage of the high angle measurement accuracy of monocular cameras; and by relying on the camera calibration parameters to complete the standardized conversion from pixels to observation angles, eliminating the angle calculation errors caused by lens distortion and imaging installation deviations, providing high-precision angle observation input for subsequent construction of observation rays and spatial rendezvous positioning, thus ensuring the calculation accuracy of ground-air cooperative target positioning from the source.

[0011] Optionally, based on the first pose parameter and the second pose parameter, the first observation angle information is converted into a first observation ray, and the second observation angle information is converted into a second observation ray, including: converting the first observation angle information into a first unit observation ray in the first imaging unit coordinate system according to the pixel coordinates of the target in the first spatial scene image and the calibration parameters of the first imaging unit; converting the second observation angle information into a second unit observation ray in the second imaging unit coordinate system according to the pixel coordinates of the target in the second spatial scene image and the calibration parameters of the second imaging unit; and converting the first unit observation ray and the second unit observation ray to a unified coordinate system according to the first pose parameter and the second pose parameter to obtain the first observation ray and the second observation ray. First, the pixel coordinates are transformed to the device's own coordinate system based on the imaging unit calibration parameters to generate the unit observation ray. Then, the ray coordinate system transformation is completed by combining the pose parameters of the air and ground ends. The observation vectors under the two independent device coordinate systems are uniformly mapped to the same global reference. The step-by-step transformation completely restores the imaging geometry relationship, eliminates the problem of observation reference fragmentation caused by the differences in spatial position and attitude of airborne and ground imaging units, and enables the two observation rays to have a unified spatial geometric calculation basis. This ensures the accuracy of subsequent dual-line spatial intersection solution of geometric models, gives full play to the positioning advantages of ground-air dual-vision collaborative observation, and reduces the cumulative calculation error caused by multi-layer coordinate transformation.

[0012] The above implementation process also includes dynamically allocating confidence weights for the observation rays: real-time acquisition of seven evaluation indicators, including imaging unit resolution, target detection confidence in the current frame, target imaging pixel size, target centering within the imaging frame, pose sensing output accuracy, data time synchronization error between the air and ground ends, and the health status of the imaging unit and its supporting sensing equipment; pre-setting segmented quantitative scoring standards for each indicator, independently scoring the first and second observation rays, and normalizing the weighted fusion of the scores of each indicator to generate the real-time dynamic weights for the corresponding observation rays; inputting the dynamic weights into the spatial intersection solution stage, and using weighted least squares optimal estimation to calculate the global position estimate of the target to be observed.

[0013] The above implementation process also includes emergency tracking for single-end target loss: if any imaging unit temporarily loses the target or fails to detect it, the observation ray at that end is stopped from being updated, and only the observation ray at the end that effectively captures the target is retained; the global position estimate of the target in historical multi-frames is retrieved, and the short-term motion trajectory of the target is deduced using a uniform or uniformly accelerated motion prediction model, and the target is continuously tracked in combination with the single effective observation ray; after both ends are rematched to the same target to be observed, the two observation rays are recalculated and the dynamic weights are updated, and the dual-ray collaborative intersection solution process is restored.

[0014] Optionally, a spatial intersection solution is performed on the first and second observation rays in a unified coordinate system to output the global position estimate of the target to be observed. This includes: constructing spatial straight line equations along the directions of the first and second observation rays, respectively, with the position of the first imaging unit in the unified coordinate system as the starting point of the first ray and the position of the second imaging unit in the unified coordinate system as the starting point of the second ray; if the first and second observation rays intersect, the intersection point is taken as the global position estimate of the target to be observed; if the first and second observation rays do not intersect strictly due to observation errors, the optimal estimation algorithm for non-plane intersection is used to determine the estimated point where the distance between the two observation rays is minimized, which is taken as the global position estimate of the target to be observed. A spatial straight-line model is constructed using the spatial points of two imaging units in a unified coordinate system as the starting points of the rays. To address the situation where rays have no intersection due to various observation errors, the shortest distance between the two rays is obtained as the global position estimate of the target through the optimal estimation algorithm of intersecting rays. This method can adapt to various interference conditions such as synchronization error, detection error, and attitude measurement error during ground-air collaborative observation, and weaken the impact of observation deviation on the positioning results. The three-dimensional position calculation is completed based on the spatial geometric constraints of the dual rays, which makes up for the deficiency that a single visual observation cannot directly obtain the target depth and stably outputs a high-precision global position estimate.

[0015] Optionally, based on the global position estimate and the coordinates of the first imaging unit, the spatial position of the target relative to the first imaging unit is determined. This includes: obtaining the position vector of the target relative to the aircraft based on the global position estimate and the current position of the aircraft carrying the first imaging unit in a unified coordinate system; determining the distance, azimuth, and altitude difference of the target relative to the aircraft based on the position vector; and determining the target's velocity and its approach trend relative to the aircraft based on multi-frame continuous global position estimates, integrating these to generate the spatial position of the target relative to the aircraft. By combining the target's global position estimate in a unified coordinate system with the aircraft's real-time coordinates to construct a relative position vector, not only are the static distance, azimuth, and altitude difference of the target relative to the aircraft calculated, but the target's velocity and relative approach trend are also mined through multi-frame time-series estimates and integrated into a complete spatial relative position. This not only intuitively reflects the target's current relative distribution in the airspace but also predicts the collision risk trend between the target and the aircraft, overcoming the limitation that dynamic conflicts cannot be assessed by relying solely on single-frame static coordinates.

[0016] Optionally, after determining the spatial relative position of the target to be observed, the method further includes: generating at least one of the following based on the position vector of the target to be observed relative to the aircraft, the speed and approach trend of the target to be observed, and a preset safety threshold; capable of directly converting the target spatial positioning result into a landable flight response output, thus establishing a link between target positioning and autonomous obstacle avoidance control of the aircraft; and capable of outputting corresponding response information according to the dynamic risk level, thereby achieving automated identification and proactive intervention of airspace intrusion target risks, reducing the lag of manual response, and comprehensively improving the proactive safety protection capability of low-altitude aircraft operation.

[0017] Secondly, the present invention also provides a ground-to-air visual cooperative target localization system, comprising: a pose acquisition module for acquiring first pose parameters and second pose parameters of a first imaging unit mounted on an aircraft and a second imaging unit deployed on the ground in a unified coordinate system; a target detection and matching module for performing target detection on a scene image using the first and second imaging units to lock onto the same target to be observed; and extracting first observation angle information of the target relative to the first imaging unit and second observation angle information relative to the second imaging unit; and an observation ray conversion module for converting the first observation angle information into a first observation ray based on the first pose parameters and the second pose parameters, and converting the second observation angle information into a second observation ray. The system generates a second observation ray through information conversion; a spatial intersection solution module is used to perform spatial intersection solution on the first and second observation rays in a unified coordinate system, and output a global position estimate of the target to be observed; a relative position calculation module is used to obtain the position vector of the target to be observed relative to the aircraft based on the global position estimate and the current position of the aircraft carrying the first imaging unit in the unified coordinate system; based on the position vector, the system determines the distance, azimuth, and altitude difference of the target to be observed relative to the aircraft, and determines the motion speed of the target to be observed and its approach trend relative to the aircraft based on multi-frame continuous global position estimates, and integrates them to generate the spatial position of the target to be observed relative to the aircraft.

[0018] According to the technical solution of the present invention, by acquiring the pose parameters of the airborne and ground imaging units in a unified coordinate system, the dual-end image detection and matching locks the same target to be observed and extracts the observation angle information of the two paths, converts and generates dual observation rays in a unified coordinate system and completes spatial intersection to solve the global position estimate of the target, and combines the aircraft coordinates to solve the spatial relative position of the target, it helps to achieve high-precision positioning of long-distance targets without rigid platform constraints by relying on ground-air dual monocular collaborative observation, overcomes the limitation of insufficient accuracy of single airborne visual ranging, improves the reliability of airspace target perception and obstacle avoidance decision of low-altitude aircraft, and provides stable and reliable technical support for autonomous flight, airspace intrusion early warning and active obstacle avoidance control of low-altitude aircraft. Attached Figure Description

[0019] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the ground-air visual cooperative target localization method provided in the embodiments of the present invention; Figure 2This is a flowchart of the pose acquisition method provided in the embodiments of the present invention; Figure 3 This is a flowchart of the dual-target matching method provided in the embodiments of the present invention; Figure 4 This is a flowchart of the observation angle calculation method provided in the embodiments of the present invention; Figure 5 This is a flowchart of the observation ray conversion method provided in the embodiments of the present invention; Figure 6 This is a flowchart of the spatial intersection and positioning method provided in the embodiments of the present invention; Figure 7 This is a flowchart of the relative position calculation method provided in the embodiments of the present invention; Figure 8 This is a schematic diagram of the structure of the ground-air visual cooperative target positioning system provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will now be described with reference to the accompanying drawings. For example, the flowcharts and block diagrams in the drawings illustrate the architecture, functions, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0022] Please refer to Figure 1 , Figure 1 This is a flowchart of the ground-to-air visual cooperative target localization method provided in an embodiment of the present invention. The ground-to-air visual cooperative target localization method includes: Step S100: Obtain the first pose parameter of the first imaging unit mounted on the aircraft in a unified coordinate system and the second pose parameter of the second imaging unit deployed on the ground in a unified coordinate system.

[0023] In step S100 above, a global geodetic coordinate system or a local navigation coordinate system is pre-selected as a unified coordinate system to complete the benchmark construction; the first imaging unit mounted on the aircraft relies on multiple sensors such as GPS, inertial navigation unit, barometer, and magnetometer to carry out data fusion processing to generate the first pose parameter containing spatial position, attitude, and timestamp information; the second imaging unit deployed on the ground uses ground GPS, inertial navigation, gimbal encoder and other positioning sensing devices to fuse and collect data to obtain the second pose parameter carrying position, attitude, and timestamp; the two sets of pose parameters are uniformly converted into the pre-built unified coordinate system and stored to realize the benchmark normalization of the status data of the equipment at both the air and ground ends.

[0024] Step S200: Use the first imaging unit and the second imaging unit to perform target detection on the scene image to lock the same target to be observed; and extract the first observation angle information of the target to be observed relative to the first imaging unit and the second observation angle information relative to the second imaging unit.

[0025] In step S200 above, the first imaging unit acquires a first spatial scene image, and the second imaging unit simultaneously acquires a second spatial scene image. Target detection operations are performed independently on the two scene images, outputting complete detection results for all targets within the image, including target detection boxes, target categories, target confidence scores, target tracking numbers, and target pixel center coordinates. The devices at both the air and ground ends exchange lightweight detection data, transmitting only target pixel centers, target categories, confidence scores, tracking numbers, observation angles, timestamps, and imaging unit pose-related parameters, without transmitting the bulky complete original spatial scene image, thus reducing communication bandwidth usage. Subsequently, the two ends align based on timestamps and target... Multiple matching constraints, including target category consistency, observation angle matching, motion continuity, and spatial intersection rationality, are used to perform pairing and screening on all targets detected at both ends. This accurately locks the same target under the dual ground and air perspectives and reads the pixel center coordinates of the matched target in the two images. The calibration parameters, such as the intrinsic parameters, distortion parameters, and carrier installation extrinsic parameters, are retrieved from the two sets of imaging units. Combined with the pixel center coordinates of the target, geometric conversion is performed to generate the first observation angle information of the target relative to the first imaging unit and the second observation angle information relative to the second imaging unit. The angle information includes at least the azimuth angle and can also output the pitch angle simultaneously.

[0026] Step S300: Based on the first pose parameter and the second pose parameter, the first observation angle information is converted into a first observation ray, and the second observation angle information is converted into a second observation ray.

[0027] In step S300 above, the imaging unit calibration parameters include camera intrinsic parameters, lens distortion parameters, and imaging unit mounting extrinsic parameters relative to the carrier. The intrinsic parameters include focal length and principal point pixel coordinates, the distortion parameters include radial distortion coefficient and tangential distortion coefficient, and the mounting extrinsic parameters are the rotation and translation transformation parameters of the imaging unit relative to the aircraft / ground gimbal. First, based on the above set of calibration parameters and the pixel coordinates of the target to be observed in the corresponding image, the image distortion is corrected. Then, the first observation angle information is transformed to the first imaging unit's own coordinate system to obtain the first unit observation ray with a modulus of 1. Simultaneously, the second observation angle information is transformed to the second imaging unit's coordinate system to obtain the second unit observation ray. Then, combined with the first pose parameter and the second pose parameter after unified coordinate system transformation and storage, coordinate transformation operations are performed on the two sets of unit observation rays through coordinate rotation and translation matrices. The rays in the local coordinate system of the device are uniformly mapped to the global unified coordinate system to obtain the first observation ray and the second observation ray that can be used for intersection calculation in the unified coordinate system.

[0028] Furthermore, dynamic allocation of confidence weights for observation rays can be performed. During implementation, seven evaluation indicators are collected in real time: imaging unit resolution, target detection confidence in the current frame, target imaging pixel size, target centering within the imaging frame, pose sensing output accuracy, data time synchronization error between the air and ground ends, and the health status of the imaging unit and its supporting sensing equipment. For each indicator, a segmented quantitative scoring standard is preset, and the first and second observation rays are scored independently. The scores of each indicator are weighted, fused, and normalized to generate the corresponding real-time dynamic weight for the observation ray. The better the observation quality, the higher the weight value is assigned, and the corresponding weight is automatically reduced when there are obvious defects in the observation. This dynamic weight is synchronously transmitted to the subsequent spatial intersection solution stage for weighted least squares optimal estimation calculation, weakening the interference of low-quality observation rays on global position estimation and improving the anti-interference capability of the positioning results.

[0029] Simultaneously, an emergency handling mechanism for single-end target loss is set up. If a target is temporarily lost or detection fails in any imaging unit during operation, the update of the observation ray at that end is suspended, and only the effective observation ray at the end that normally captures the target is retained. The global position estimate of the target that has been calculated in the previous multiple frames is retrieved, and the short-term spatial motion trajectory of the target is deduced through a uniform / uniform acceleration motion prediction model. The target is continuously tracked by relying on a single effective observation ray in conjunction with the trajectory prediction model, ensuring that the positioning output is uninterrupted. After the imaging units at both ends synchronously re-detect and match the same target to be observed, the above complete conversion of the two observation rays is re-executed, and the dynamic weights of the two observation rays are updated synchronously, restoring the ground-to-air dual-ray collaborative rendezvous process.

[0030] Step S400: Perform spatial intersection solution on the first and second observation rays in a unified coordinate system to output the global position estimate of the target to be observed.

[0031] In step S400 above, the spatial point of the first imaging unit in the unified coordinate system is taken as the starting point of the first ray, and the point of the second imaging unit is taken as the starting point of the second ray. Spatial straight line equations are constructed along the extension directions of the two observation rays respectively. Under ideal conditions with no observation errors, the two spatial straight lines intersect, and the intersection point is directly taken as the global position estimate of the target to be observed. In actual scenarios, due to the influence of sensor measurement errors, image target detection errors, time synchronization errors at both ends of the air and ground, and carrier attitude estimation errors, the two rays are often in a state of non-planar straight lines without precise intersection points. At this time, the non-planar intersection optimal estimation algorithm is used to obtain the two non-planar straight lines. The midpoint of the shortest common perpendicular or the weighted estimation point is used as the instantaneous global position estimate of the target. To eliminate random errors in single-frame observations and improve positioning continuity, extended Kalman filtering, unscented Kalman filtering, or particle filtering can be used to perform temporal smoothing filtering on multi-frame continuous intersection results. The filtering state variables include the target's position, velocity, and acceleration in a unified coordinate system. The observations are fused with the two observation angles and the single-frame intersection position estimate, and the global position estimate of the target is continuously iterated and corrected. At the same time, weighted least squares optimization is performed in combination with the set dynamic weights of the observation rays to further reduce the interference of low-quality observation data on the positioning results.

[0032] Step S500: Based on the global position estimate and the current position of the aircraft carrying the first imaging unit in the unified coordinate system, obtain the position vector of the target to be observed relative to the aircraft; based on the position vector, determine the distance, azimuth angle and altitude difference of the target to be observed relative to the aircraft, and determine the motion speed of the target to be observed and its approach trend relative to the aircraft based on multi-frame continuous global position estimates, and integrate them to generate the spatial position of the target to be observed relative to the aircraft.

[0033] In step S500 above, the global position estimate of the target under observation, after filtering and optimization, is retrieved in a unified coordinate system. Simultaneously, the real-time three-dimensional coordinates of the aircraft equipped with the first imaging unit in the unified coordinate system are read. The global position estimate of the target is subtracted from the real-time coordinates of the aircraft to complete the vector difference calculation, thus obtaining the three-dimensional position vector of the target relative to the aircraft. Spatial geometric analytical operations are performed on this three-dimensional position vector. The straight-line distance between the target and the aircraft is calculated using the vector magnitude. The horizontal azimuth angle between the target and the aircraft is solved by combining the horizontal plane projection components. The height difference between the target and the aircraft is obtained through the vertical component difference, thereby completing the static spatial... The system calculates relative parameters; continuously caches the global position estimates of the target over multiple frames in time sequence; fits the target's three-dimensional motion velocity based on the position change between adjacent frames and the time interval between frames; combines the target velocity vector with the aircraft's own motion vector to perform relative motion calculation; quantifies different types of relative approach trends, such as the target moving closer to the aircraft, moving away from the aircraft, and passing by laterally; and outputs the estimated collision duration between the target and the aircraft; integrates and encapsulates static distance, azimuth, and altitude difference parameters, as well as dynamic motion velocity, relative approach trend, and estimated collision duration to form complete spatial relative position data of the observed target that includes static spatial information and dynamic motion characteristics.

[0034] Furthermore, after calculating the relative position of the target in space, a tiered airspace safety threshold is preset. This threshold includes a near-distance warning threshold, a dangerous collision threshold, a lateral safety distance threshold, a vertical height safety threshold, and a critical approach speed threshold. The target's distance to the aircraft, altitude difference, relative approach speed, and estimated collision duration are compared with each preset safety threshold to determine the risk level. Based on the risk level, corresponding handling information is output: in low-risk conditions, only target alarm information is output, with the target's bearing, distance, and approach speed indicated for aircraft operators; in medium-risk conditions, obstacle avoidance suggestions such as heading adjustment, altitude gain / loss, and deceleration are output simultaneously, clearly specifying the recommended adjustment range and duration; in high-risk collision conditions, at least one flight control command is directly generated, including deceleration, changing flight heading, increasing / decreasing flight altitude, hovering in place, and returning to base. These commands can be directly issued to the aircraft's flight control execution unit to complete autonomous avoidance actions. The entire process achieves automatic identification and tiered handling of airspace intrusion target risks, supporting all-weather autonomous obstacle avoidance and full-domain airspace safety management for low-altitude aircraft.

[0035] Please refer to Figure 2 , Figure 2 This is a flowchart of a pose acquisition method provided in an embodiment of the present invention; the pose acquisition method includes: Step S20: Pre-construct a unified coordinate system, which may be either a global geodetic coordinate system or a local navigation coordinate system.

[0036] In step S20 above, a suitable reference coordinate system is selected according to the operational airspace range of the aircraft. For large-scale cross-regional flight scenarios, the global geodetic coordinate system is used as the unified reference, while for close-range, small-scale fixed-point observation scenarios, the local navigation coordinate system is used as the unified reference. After the coordinate system is constructed, the origin, three-axis positive definition, and unit scale standard are calibrated. The coordinate system transformation reference parameters are fully recorded and stored. This provides a fixed spatial calculation reference for the unified conversion of the pose, observation ray, and target position at both ends of the airspace and ground, eliminating the systematic positioning deviation caused by the inconsistent reference of the separate equipment.

[0037] Step S21: Data fusion is performed using the first imaging unit mounted on the aircraft and equipped with multiple sensors to obtain the first attitude parameter, which includes position, attitude, and timestamp.

[0038] In step S21 above, the first imaging unit on the aircraft is equipped with a variety of airborne sensing devices, including a satellite positioning sensor, an inertial measurement unit, a magnetometer, and an altitude sensor. Each sensor synchronously collects raw positioning, attitude, and altitude data and carries a local hardware synchronization timestamp. The Kalman fusion algorithm is used to jointly solve the raw data from multiple sources, remove noise and abrupt changes from individual sensors, and fuse and output high-precision three-dimensional spatial position and roll / pitch / yaw three-axis attitude angles. The fused position and attitude information is bound to the unified timestamp of the current frame to generate complete first-order attitude parameters, and the spatial state of the airborne imaging unit at every moment is fully recorded.

[0039] Step S22: Data fusion is performed by the second imaging unit deployed on the ground and the ground positioning sensing device to obtain the second pose parameters including position, attitude and timestamp.

[0040] In step S22 above, the second imaging unit deployed on the ground is equipped with a ground satellite positioning module, a gimbal attitude encoder, and a ground inertial navigation sensor to form a ground positioning sensing device. This device collects the coordinates of the installation point of the ground imaging unit and the raw data of the gimbal rotation attitude in real time. The raw data from the multiple ground sensors is smoothed, filtered, and fused to eliminate attitude measurement disturbances caused by ground vibration and gimbal jitter. The device is then synchronized with the unified timing timestamp synchronized with the airborne terminal. The fused ground terminal position, attitude, and timestamp are bound together to generate standardized second pose parameters, thus completely recording the real-time spatial installation attitude information of the ground imaging unit.

[0041] Step S23: Convert the first pose parameter and the second pose parameter to a unified coordinate system and store them.

[0042] In step S23 above, the pre-constructed unified coordinate system transformation matrix and reference parameters are retrieved, and coordinate reference transformation operations are performed on the position coordinates and attitude angles within the first pose parameter and the second pose parameter, respectively, to map all pose data belonging to the two independent equipment coordinate systems of the airborne end and the ground end to the same unified coordinate system. After the transformation is completed, the two sets of pose parameters are timestamped for alignment verification, and invalid pose data with time deviations exceeding the threshold are removed. The verified and coordinate-normalized first pose parameter and second pose parameter are packaged and stored in time sequence to provide synchronous and referenced original pose data support for subsequent observation angle conversion, unified conversion of observation rays, and spatial intersection positioning.

[0043] Please refer to Figure 3 , Figure 3 This is a flowchart of a two-end target matching method provided in an embodiment of the present invention; the two-end target matching method includes: Step S30: Acquire a first spatial scene image through the first imaging unit, and acquire a second spatial scene image through the second imaging unit.

[0044] In step S30 above, the first imaging unit on the airborne end and the second imaging unit on the ground end are controlled to synchronously trigger image acquisition actions at the same acquisition frame rate to generate corresponding airspace scene images. During the acquisition process, the acquisition timestamp corresponding to each frame image is recorded synchronously to ensure that the two images are time-series aligned and reduce the time synchronization error of subsequent target matching. The first imaging unit outputs the first airspace scene image, and the second imaging unit synchronously outputs the second airspace scene image. The two images correspond to the same observation airspace under the aircraft's aerial view and the ground fixed observation view, respectively.

[0045] Step S31: Perform target detection on the first spatial scene image and the second spatial scene image respectively, and output detection data containing pixel center coordinates.

[0046] In step S31 above, a preset target detection algorithm is run independently on the first and second spatial scene images to identify various targets in the spatial domain. For each detected target, the pixel center coordinates of the target detection box are calculated, and the target detection box size, target category, detection confidence, and target tracking number are output simultaneously and integrated to form a complete single-frame detection data. Each imaging unit only locally caches all target detection results of the current frame and does not transmit the complete original spatial scene image to the outside. Only the lightweight detection fields are retained for subsequent cross-end interaction.

[0047] Step S32: The first imaging unit and the second imaging unit interact to detect data, perform pairing and filtering on the detected spatial targets, lock the same target to be observed, and read the pixel center coordinates of the target to be observed in the first spatial scene image and the second spatial scene image respectively.

[0048] In step S32 above, the first imaging unit and the second imaging unit only exchange lightweight detection data. The lightweight detection data does not contain the complete original image, but only carries the target pixel center coordinates, target category, detection confidence, tracking number, image acquisition timestamp, and real-time pose parameters of the imaging unit. After the interaction between the two ends is completed, matching discrimination conditions are constructed based on multiple matching constraints such as timestamp matching, target category consistency, observation angle approximation, spatial geometric constraints, and motion continuity. All spatial targets output by the two ends are cross-paired and filtered one by one to eliminate false matching targets caused by perspective differences and accurately lock the same target to be observed under the dual ground and air perspectives. After the matching is completed, the pixel center coordinates of the same target to be observed in the first spatial scene image and the second spatial scene image are extracted separately as the basic input data for subsequent conversion of observation angle information.

[0049] Please refer to Figure 4 , Figure 4 This is a flowchart of the observation angle calculation method provided in the embodiments of the present invention; the observation angle calculation method includes: Step S40: Retrieve the calibration parameters of the first imaging unit, and combine them with the pixel center coordinates of the target to be observed in the first spatial scene image to generate the first observation angle information.

[0050] In step S40 above, the complete set of calibration parameters of the first imaging unit, which are pre-stored locally, are read. These calibration parameters include imaging intrinsic parameters, lens radial and tangential distortion coefficients, and airborne installation extrinsic parameters. The intrinsic parameters include focal length and image principal pixel coordinates. First, distortion correction is performed on the center coordinates of the target pixel using the distortion coefficients to eliminate pixel offset errors caused by lens deformation, and the corrected standard pixel coordinates are obtained. Then, based on the pinhole imaging geometric model, the horizontal azimuth angle and vertical pitch angle of the target relative to the imaging unit are calculated using the optical axis of the imaging unit as a reference, combined with the focal length and the corrected pixel coordinates. The two sets of angle values ​​are integrated and encapsulated into the first observation angle information, which fully represents the spatial orientation of the target relative to the optical axis of the first imaging unit.

[0051] Step S41: Retrieve the calibration parameters corresponding to the second imaging unit, and combine them with the pixel center coordinates of the target to be observed in the second spatial scene image to generate the second observation angle information.

[0052] In step S41 above, the pre-stored calibration parameters specific to the second imaging unit are retrieved. These parameters include the imaging intrinsic parameters of the device, the lens distortion coefficient, and the extrinsic parameters of the ground gimbal installation. The center coordinates of the target pixels are corrected using a unified distortion correction logic to remove pixel deviations caused by lens distortion of the ground observation device. Based on the standard pinhole imaging geometric conversion relationship, the azimuth and elevation angles of the target relative to the second imaging unit are calculated by combining the corrected pixel coordinates and the calibration focal length. The two sets of angle data are combined to form the second observation angle information, which completely records the observation direction of the same target relative to the ground imaging unit, providing standardized angle input for the subsequent construction of two observation rays.

[0053] Please refer to Figure 5 , Figure 5 This is a flowchart of an observation ray conversion method provided in an embodiment of the present invention; the observation ray conversion method includes: Step S50: Based on the pixel coordinates of the target to be observed in the first spatial scene image and the calibration parameters of the first imaging unit, convert the first observation angle information into the first unit observation ray in the coordinate system of the first imaging unit.

[0054] In step S50 above, the complete calibration parameters of the first imaging unit are retrieved. First, the distortion correction of the target pixel coordinates is completed based on the distortion coefficient. Then, the azimuth and pitch angles contained in the first observation angle information are combined to carry out three-dimensional vector conversion. Taking the origin of the first imaging unit's own coordinate system as the reference, the original spatial vector pointing to the target to be observed is constructed. The original vector is normalized to a normalized modulus, which can constrain the vector length to 1, generating the first unit observation ray that only represents spatial direction and has no distance dimension. This ray completely describes the observation direction of the target in the local coordinate system of the airborne imaging unit.

[0055] Step S51: Based on the pixel coordinates of the target to be observed in the second spatial scene image and the calibration parameters of the second imaging unit, convert the second observation angle information into the second unit observation ray in the coordinate system of the second imaging unit.

[0056] In step S51 above, the calibration parameters of the second imaging unit are read, and the target pixel coordinates are corrected using a consistent distortion correction process. The original spatial vector in the local coordinate system of the ground imaging unit is constructed by combining the azimuth and pitch angles in the second observation angle information. The vector is normalized to obtain the second unit observation ray with a modulus of 1. This ray independently represents the observation direction of the target relative to the ground end imaging unit. It belongs to two independent local coordinate systems with the first unit observation ray and does not yet have unified calculation conditions.

[0057] Step S52: Based on the first pose parameter and the second pose parameter, transform the first unit observation ray and the second unit observation ray to a unified coordinate system to obtain the first observation ray and the second observation ray.

[0058] In step S52 above, the first pose parameter and the second pose parameter after unified coordinate system transformation and time-series alignment storage are retrieved, and the rotation matrix and translation amount corresponding to the transformation from the device coordinate system to the unified coordinate system are extracted from the two sets of pose parameters respectively; firstly, the attitude rotation transformation of the first unit observation ray and the second unit observation ray is completed by the rotation matrix to match the real-time spatial orientation of the airborne and ground equipment; then, the spatial origin offset of the equipment in the unified coordinate system is superimposed to complete the complete mapping from the local coordinate system to the global unified coordinate system; after the transformation is completed, the standardized first observation ray and the second observation ray in the unified coordinate system are obtained, and the two observation rays share the same spatial reference.

[0059] Please refer to Figure 6 , Figure 6 This is a flowchart of a spatial rendezvous and positioning method provided in an embodiment of the present invention; the spatial rendezvous and positioning method includes: Step S60: Using the position of the first imaging unit in the unified coordinate system as the starting point of the first ray and the position of the second imaging unit in the unified coordinate system as the starting point of the second ray, construct spatial straight line equations along the directions of the first observation ray and the second observation ray, respectively.

[0060] In step S60 above, the three-dimensional coordinates of the first imaging unit stored in the unified coordinate system are read as the origin and starting point of the first spatial line, and the three-dimensional coordinates of the second imaging unit are read as the origin and starting point of the second spatial line. The normalized first and second observation rays are used as the direction vectors of the two lines, and the equations of the two spatial lines are constructed in parametric standard form to fully quantify the geometric expression of the two observation lines in the global unified space. The dynamic weight parameters of the two observation rays generated are simultaneously introduced during the construction process to reserve the calculation parameter interface for subsequent weighted optimal estimation, and complete the unified mathematical modeling before solving the target position.

[0061] Step S61: If the first observation ray and the second observation ray intersect, then take the intersection point as the global position estimate of the target to be observed.

[0062] In step S61 above, the two sets of spatial straight line equations are solved simultaneously to determine whether there is a unique common intersection point between the two sets of straight lines. Under ideal conditions with no sensor noise, detection bias, or timing synchronization error, the two spatial straight lines have a unique intersection point, which is the true three-dimensional spatial position of the target to be observed. The three-dimensional coordinates of the intersection point are directly assigned as the estimated instantaneous global position of the target to be observed. This result has no approximate fitting bias and can be directly used for subsequent relative position calculation and timing filtering update.

[0063] Step S62: If the first and second observation rays do not intersect strictly due to observation errors, the optimal estimation algorithm for non-plane intersection is used to determine the estimated point where the distance between the two observation rays is minimized, which is used as the global position estimate of the target to be observed.

[0064] In step S62 above, due to engineering realities such as pose measurement error, image target detection deviation, timing synchronization deviation at both ends of the air and ground, and residual lens calibration error, the two spatial straight lines are in a non-planar state and do not have a precise intersection point. At this time, the non-planar intersection optimal estimation algorithm is started. Based on the parametric equations of the two non-planar straight lines, the shortest common perpendicular of the two lines is solved, and the midpoint of the common perpendicular is taken as the optimal estimated point. At the same time, combined with the dynamic weights configured in advance of the two observation rays, weighted least squares correction is performed. The ray with higher observation quality corresponds to a larger weight, which further reduces the coordinate offset caused by low-precision observation. The coordinates of the midpoint of the shortest distance after weighted correction are used as the global position estimate of the target to be observed and output to the subsequent relative position calculation process to adapt to various actual operation scenarios with observation disturbances.

[0065] Please refer to Figure 7 , Figure 7 This is a flowchart of a relative position calculation method provided in an embodiment of the present invention; the relative position calculation method includes: Step S70: Based on the global position estimate and the current position of the aircraft carrying the first imaging unit in the unified coordinate system, obtain the position vector of the target to be observed relative to the aircraft.

[0066] In step S70 above, the estimated global three-dimensional position of the target under observation, which is output by spatial intersection and can carry temporal filtering correction, is retrieved, and the real-time three-dimensional coordinates of the aircraft under a unified coordinate system are read simultaneously. The global position estimate of the target is subtracted from the real-time coordinates of the aircraft by vector subtraction operation to calculate the three-dimensional difference vector. This difference vector is the position vector of the target under observation relative to the aircraft. The three-dimensional components of the vector correspond to the spatial offsets in the horizontal, vertical and vertical dimensions, respectively.

[0067] Step S71: Based on the position vector, determine the distance, azimuth, and altitude difference between the target to be observed and the aircraft, and determine the motion speed of the target to be observed and its approach trend relative to the aircraft based on multi-frame continuous global position estimates, and integrate them to generate the spatial position of the target to be observed relative to the aircraft.

[0068] In step S71 above, geometric analytical calculations are performed on the position vector to obtain the vector magnitude and the straight-line distance between the target and the aircraft. The vector is then projected onto the horizontal plane to calculate the horizontal azimuth angle. The vertical component difference is extracted to obtain the height difference between the target and the aircraft, thus completing the static relative spatial parameter solution. The global position estimate of the target is continuously cached for multiple consecutive frames. The absolute three-dimensional motion velocity of the target is calculated by combining the inter-frame time interval. The relative motion relationship between the two is then calculated by integrating the aircraft's own motion vector. Different approach trends, such as the target approaching, moving away, and passing by laterally, are quantitatively distinguished. The estimated collision duration is calculated simultaneously. The static distance, azimuth angle, height difference, dynamic motion velocity, relative approach trend, and estimated collision duration are all integrated and encapsulated to form a complete spatial position of the observed target relative to the aircraft, which simultaneously contains static spatial distribution information and dynamic motion characteristics.

[0069] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the ground-to-air visual cooperative target localization system provided in an embodiment of the present invention. The system includes a pose acquisition module 10, a target detection and matching module 20, an observation ray conversion module 30, a spatial intersection solution module 40, and a relative position calculation module 50. The pose acquisition module 10 is used to acquire the first pose parameter of the first imaging unit mounted on the aircraft in a unified coordinate system and the second pose parameter of the second imaging unit deployed on the ground in a unified coordinate system. The target detection and matching module 20 is used to perform target detection on the scene image using the first and second imaging units to lock onto the same target to be observed; and to extract the first observation angle information of the target relative to the first imaging unit and the second observation angle information relative to the second imaging unit. The observation ray conversion module 30 is used to calculate the relative position based on the first pose parameter and the second observation ray conversion module 50. The two pose parameters are used to convert the first observation angle information into a first observation ray and the second observation angle information into a second observation ray. The spatial intersection solution module 40 is used to perform spatial intersection solution on the first and second observation rays in a unified coordinate system and output the global position estimate of the target to be observed. The relative position calculation module 50 is used to obtain the position vector of the target to be observed relative to the aircraft based on the global position estimate and the current position of the aircraft carrying the first imaging unit in the unified coordinate system. Based on the position vector, the distance, azimuth angle and altitude difference of the target to be observed relative to the aircraft are determined, and the motion speed of the target to be observed and the approach trend relative to the aircraft are determined based on the multi-frame continuous global position estimates. The spatial position of the target to be observed relative to the aircraft is then integrated to generate the spatial position of the target to be observed relative to the aircraft.

[0070] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the steps in any of the above implementations.

[0071] Based on the same inventive concept, the present invention also provides a computer program product, which includes a computer program / instruction that, when executed by a processor, implements the steps in any of the above implementation methods.

[0072] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A ground-to-air visual cooperative target localization method, characterized in that, include: Acquire the first attitude parameters of the first imaging unit mounted on the aircraft in the same coordinate system and the second attitude parameters of the second imaging unit deployed on the ground in the same coordinate system. The first imaging unit and the second imaging unit are used to perform target detection on the scene image to lock the same target to be observed; and the first observation angle information of the target to be observed relative to the first imaging unit and the second observation angle information relative to the second imaging unit are extracted; wherein, the interactive detection data of the first imaging unit and the second imaging unit is lightweight detection data, and the complete original spatial scene image is not transmitted; the lightweight detection data includes the target pixel center, observation angle information, timestamp, and imaging unit pose-related parameters; Based on the first pose parameter and the second pose parameter, the first observation angle information is converted into a first observation ray, and the second observation angle information is converted into a second observation ray. In the unified coordinate system, the first and second observation rays are spatially intersected to solve for the estimated global position of the target to be observed. Based on the global position estimate and the current position of the aircraft carrying the first imaging unit in the unified coordinate system, the position vector of the target to be observed relative to the aircraft is obtained; Based on the position vector, the distance, azimuth, and altitude difference of the target to be observed relative to the aircraft are determined, and the motion speed of the target to be observed and its approach trend relative to the aircraft are determined based on multi-frame continuous global position estimation. The spatial position of the target to be observed relative to the aircraft is then generated.

2. The method according to claim 1, characterized in that, The acquisition of the first attitude parameters of the first imaging unit mounted on the aircraft in a unified coordinate system and the second attitude parameters of the second imaging unit deployed on the ground in the same coordinate system includes: A unified coordinate system is pre-constructed, which may be a global geodetic coordinate system or a local navigation coordinate system. Data fusion is performed using the first imaging unit mounted on the aircraft and equipped with multiple types of sensors to obtain the first attitude parameter, which includes position, attitude, and timestamp. Data fusion is performed by the second imaging unit deployed on the ground and the ground positioning sensing device to obtain a second pose parameter including position, attitude and timestamp. The first pose parameter and the second pose parameter are uniformly converted to the unified coordinate system and stored.

3. The method according to claim 1, characterized in that, The step of using the first imaging unit and the second imaging unit to perform target detection on the scene image and lock onto the same target to be observed includes: A first spatial scene image is acquired through the first imaging unit, and a second spatial scene image is acquired through the second imaging unit; Target detection is performed on the first spatial scene image and the second spatial scene image respectively, and detection data including pixel center coordinates is output. The first imaging unit and the second imaging unit interact with the detection data, perform pairing and filtering of the detected spatial targets based on multi-class matching constraints, lock the same target to be observed, and read the pixel center coordinates of the target to be observed in the first spatial scene image and the second spatial scene image respectively.

4. The method according to claim 3, characterized in that, The extraction of the first observation angle information of the target relative to the first imaging unit and the second observation angle information relative to the second imaging unit includes: The calibration parameters of the first imaging unit are retrieved, and combined with the pixel center coordinates of the target to be observed in the first spatial scene image, the first observation angle information is generated. The calibration parameters corresponding to the second imaging unit are retrieved, and combined with the pixel center coordinates of the target in the second spatial scene image, the second observation angle information is generated.

5. The method according to claim 1, characterized in that, The step of converting the first observation angle information into a first observation ray and the second observation angle information into a second observation ray based on the first and second pose parameters includes: Based on the pixel coordinates of the target to be observed in the first spatial scene image and the calibration parameters of the first imaging unit, the first observation angle information is converted into the first unit observation ray in the first imaging unit coordinate system. Based on the pixel coordinates of the target to be observed in the second spatial scene image and the calibration parameters of the second imaging unit, the second observation angle information is converted into a second unit observation ray in the second imaging unit coordinate system; Based on the first pose parameter and the second pose parameter, the first unit observation ray and the second unit observation ray are respectively transformed to the unified coordinate system to obtain the first observation ray and the second observation ray.

6. The method according to claim 1, characterized in that, The step of performing spatial intersection calculations on the first and second observation rays in the unified coordinate system to output the global position estimate of the target to be observed includes: Using the position of the first imaging unit in the unified coordinate system as the starting point of the first ray and the position of the second imaging unit in the unified coordinate system as the starting point of the second ray, spatial straight line equations are constructed along the directions of the first observation ray and the second observation ray, respectively. If the first observation ray and the second observation ray intersect, the intersection point is taken as the global position estimate of the target to be observed; If the first observation ray and the second observation ray do not intersect strictly due to observation errors, the optimal estimation algorithm for non-plane intersection is used to determine the estimated point where the distance between the two observation rays is minimized, which is used as the global position estimate of the target to be observed.

7. The method according to claim 1, characterized in that, After determining the spatial relative position of the target to be observed, the method further includes: Based on the position vector of the target relative to the aircraft, the speed and approach trend of the target, and in conjunction with a preset safety threshold, at least one of the following is generated: target alarm information, obstacle avoidance suggestions, or flight control commands.

8. A ground-to-air visual cooperative target localization system, characterized in that, include: The pose acquisition module is used to acquire the first pose parameter of the first imaging unit mounted on the aircraft in a unified coordinate system and the second pose parameter of the second imaging unit deployed on the ground in the same coordinate system. The target detection and matching module is used to perform target detection on the scene image using the first imaging unit and the second imaging unit to lock the same target to be observed; and to extract the first observation angle information of the target to be observed relative to the first imaging unit and the second observation angle information relative to the second imaging unit. The observation ray conversion module is used to convert the first observation angle information into a first observation ray based on the first pose parameter and the second pose parameter, and to convert the second observation angle information into a second observation ray. The spatial intersection solution module is used to perform spatial intersection solution on the first observation ray and the second observation ray in the unified coordinate system, and output the global position estimate of the target to be observed; The relative position calculation module is used to obtain the position vector of the target to be observed relative to the aircraft based on the global position estimate and the current position of the aircraft carrying the first imaging unit in a unified coordinate system; determine the distance, azimuth angle and altitude difference of the target to be observed relative to the aircraft based on the position vector; determine the motion speed of the target to be observed and the approach trend relative to the aircraft based on the multi-frame continuous global position estimate; and integrate to generate the spatial position of the target to be observed relative to the aircraft.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, perform the steps of the method according to any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the steps of the method according to any one of claims 1-x.

Images