Multi-camera low-altitude unmanned aerial vehicle tracking method, system and electronic device

Abstract

The application relates to the field of unmanned aerial vehicle positioning and tracking, and discloses a multi-camera low-altitude unmanned aerial vehicle tracking method, system and electronic equipment. In the method, a camera array composed of multiple cameras deployed on the ground is taken as a reference to establish a three-dimensional overhead grid coordinate system, and a mapping relationship between a pixel coordinate system of each camera in the multiple cameras and the overhead grid coordinate system is determined; based on the mapping relationship of each camera, image feature maps synchronously collected by the multiple cameras are projected into the overhead grid coordinate system to generate a globally unified overhead grid feature map; and based on the position and motion state of a target detected in the overhead grid feature map by a detection head, a continuous motion trajectory of the target in the overhead grid coordinate system is generated. Through the above method, accurate three-dimensional positioning, stable and continuous tracking and reliable management and control of multiple targets of a low-altitude unmanned aerial vehicle are realized.

Description

Technical Field

[0001] This application relates to the field of drone positioning and tracking, and in particular to a multi-camera low-altitude drone tracking method, system, and electronic equipment. Background Technology

[0002] With the rapid development of the low-altitude economy, security threats such as unauthorized drone flights, intrusion into controlled airspace, and privacy breaches have become increasingly prominent. Ground-based multi-camera visual tracking, with its advantages of low cost and wide coverage, has become a core technology for low-altitude security control. This technology achieves three-dimensional positioning and continuous tracking of low-altitude drones through simultaneous acquisition by multiple cameras, target detection, and geometric stereo matching.

[0003] The relevant ground-based multi-camera low-altitude target tracking technology has significant shortcomings. Each camera processes data independently in an independent pixel coordinate system, which easily leads to target association confusion and tracking interruption. The positioning accuracy for distant small targets is low, and it is easily affected by complex sky backgrounds and sudden changes in light. The multi-camera data is only fused at the positioning result level, resulting in poor tracking robustness in dense multi-target scenarios, and it cannot meet the complex low-altitude security needs of cities. Summary of the Invention

[0004] The purpose of this application is to provide a multi-camera low-altitude UAV tracking method, system, and electronic device to achieve accurate three-dimensional positioning, stable continuous tracking, and reliable parallel control of multiple targets for low-altitude UAVs.

[0005] To address the aforementioned technical problems, embodiments of this application provide a multi-camera low-altitude UAV tracking method, comprising: establishing a three-dimensional aerial upward-looking grid coordinate system based on a camera array composed of multiple cameras deployed on the ground, and determining the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system; based on the mapping relationship of each camera, projecting the image feature maps synchronously acquired by the multiple cameras onto the aerial upward-looking grid coordinate system to generate a globally unified upward-looking grid feature map; and generating a continuous motion trajectory of the target in the aerial upward-looking grid coordinate system based on the position and motion state of the target detected by the detection head in the upward-looking grid feature map.

[0006] An embodiment of this application also provides a multi-camera low-altitude UAV tracking system, the system comprising: a ground-based multi-camera array module, including multiple cameras, for acquiring image feature maps; a calibration and spatial alignment module, for establishing a three-dimensional aerial upward-looking grid coordinate system based on the camera array composed of multiple cameras deployed on the ground, and determining the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system; a feature extraction and projection module, for projecting the image feature maps synchronously acquired by the multiple cameras onto the aerial upward-looking grid coordinate system based on the mapping relationship of each camera, generating a globally unified upward-looking grid feature map; and a perception and tracking module, for detecting the position and motion state of the target in the upward-looking grid feature map by a detection head, and generating a continuous motion trajectory of the target in the aerial upward-looking grid coordinate system.

[0007] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the multi-camera low-altitude UAV tracking method as described above.

[0008] In one or more embodiments of this application, technical problems such as error-prone multi-camera target association, low positioning accuracy of small targets at low altitudes at long distances, and poor robustness of tracking in occluded scenes are addressed. To address these problems, this application establishes an aerial upward-looking grid coordinate system and determines the mapping relationship between the pixel coordinate system of each camera and this aerial upward-looking grid coordinate system. This provides a unified spatial reference for multiple cameras, eliminates association deviations caused by independent coordinate systems, avoids target association confusion, and improves tracking continuity in occluded and single-camera failure scenarios. Furthermore, this application projects the image feature maps of multiple cameras onto the aerial upward-looking grid coordinate system to generate a global upward-looking grid feature map. This aggregates multi-camera features in a unified global space, eliminates perspective distortion and scale ambiguity in pixel coordinate systems, improves the 3D positioning accuracy of small targets at long distances, and enhances anti-interference capabilities. Additionally, this application detects targets in the global upward-looking grid feature map based on the detection head and generates continuous motion trajectories. By completing target detection and trajectory association in a unified 3D space, it improves the robustness of tracking in dense multi-target scenes and can adapt to complex low-altitude environments. Attached Figure Description

[0009] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0010] Figure 1This is a flowchart of a multi-camera low-altitude UAV tracking method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the aerial look-up grid (WEV) involved in the embodiments of this application; Figure 3 This is another flowchart of the multi-camera low-altitude UAV tracking method provided in the embodiments of this application; Figure 4 This is a schematic diagram of a multi-camera low-altitude UAV tracking system provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] To more clearly illustrate the technical solutions of the embodiments in this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the content described below are some examples or embodiments of this specification. For those skilled in the art, without creative effort, the technical solutions or means disclosed in this specification can be applied to other scenarios based on this technical content.

[0012] It should be understood that the terms "system," "device," "unit," and / or "module" used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other words can achieve the same purpose, they may be replaced by other expressions.

[0013] Unless otherwise specified, the technical terms used to describe components, elements, etc. in this specification are not singular but may include plural. Generally speaking, terms such as "comprising" or "including" only indicate that explicitly identified steps, elements, or components are included, and these steps, elements, and components do not constitute an exclusive list, as the described method or apparatus may also include other steps or components.

[0014] This specification uses flowcharts to illustrate the operational steps performed by the apparatus or system of related embodiments. However, unless otherwise specified, the order in which these steps are described should not be construed as a limitation on the order of execution. Those skilled in the art can adjust the order of these steps based on the knowledge and information conveyed by the embodiments in this specification. Adjustments include, but are not limited to, reversing the order of steps, merging multiple steps, and splitting a step.

[0015] As the background technology shows, relevant ground-based multi-camera low-altitude target tracking technologies mostly employ multi-camera simultaneous acquisition, single-frame target detection, and geometric stereo matching to achieve UAV 3D positioning and tracking, solving the problems of low positioning accuracy and easy tracking loss with monocular cameras. However, significant shortcomings exist: Multi-camera images are processed independently in their respective pixel coordinate systems, lacking a globally unified spatial coordinate system constraint. When the target is partially occluded or single-camera detection fails, target association errors, frequent ID switching, and tracking interruptions are very likely to occur. Relying solely on geometric triangulation matching of single-frame images results in poor positioning accuracy for small, low-altitude targets at long distances, and is susceptible to interference from complex sky backgrounds, sudden changes in lighting, and backlighting. Multi-camera data fusion is only performed at the post-fusion level of positioning results, failing to achieve unified spatial perception at the feature level. This results in severely insufficient robustness for tracking multi-target and dense target scenes, making it unsuitable for complex low-altitude urban environments.

[0016] Bird's-Eye-View (BEV) perception architecture is a mainstream advanced perception solution in the field of autonomous driving. It can project multi-view images onto a global top-down coordinate system centered on the vehicle, achieving unified modeling and perception of the ground environment. However, the core of the existing BEV architecture, which focuses on top-down perception of ground targets, heavily relies on prior knowledge of the ground plane. Its projection logic, coordinate system design, and feature modeling methods cannot adapt to the upward-looking scenarios of ground equipment detecting aerial targets. Currently, there is no technology that can flip the perspective of the BEV architecture to build a unified perception solution with an aerial upward-looking grid adapted to ground-to-air scenarios, nor is there a mature technology that can be applied to ground-based multi-view low-altitude target tracking scenarios.

[0017] In view of this, and in response to the technical problems in related technologies such as the susceptibility of multi-camera target association to errors, the low positioning accuracy of small targets at low altitudes at long distances, and the poor robustness of tracking in occluded scenes, as well as the limitation of traditional BEV architectures that are only adapted to ground-view scenarios, this application provides a multi-camera low-altitude UAV tracking method. This method establishes an aerial upward-looking grid coordinate system and determines the mapping relationship between the pixel coordinate system of each camera and this aerial upward-looking grid coordinate system. This provides a unified spatial reference for multiple cameras, eliminates the correlation deviation caused by independent coordinate systems, avoids target association confusion, and improves tracking continuity in occlusion and single-camera failure scenarios. Furthermore, this application projects the image feature maps of multiple cameras onto the aerial upward-looking grid coordinate system to generate a global upward-looking grid feature map. By aggregating the features of multiple cameras in a unified global space, it eliminates perspective distortion and scale ambiguity of pixel coordinate systems, improves the 3D positioning accuracy of distant small targets, and enhances anti-interference capability. In addition, this application detects targets in the global upward-looking grid feature map based on the detection head and generates continuous motion trajectories. By completing target detection and trajectory association in a unified 3D space, it improves the tracking robustness of multi-target dense scenes and can adapt to complex low-altitude environments.

[0018] Figure 1 This is an exemplary flowchart illustrating a multi-camera low-altitude UAV tracking method according to some embodiments of this application. Figure 1 As shown, in some embodiments, the multi-camera low-altitude UAV tracking method includes the following steps.

[0019] Step 110: Using a camera array consisting of multiple cameras deployed on the ground as a reference, establish a three-dimensional aerial upward-looking grid coordinate system, and determine the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system.

[0020] In step 110, a global aerial view (Worm's-Eye View, WEV) unified coordinate system is constructed based on a camera array consisting of multiple cameras deployed on the ground. Specifically, as follows... Figure 2 As shown, this application uses the geometric center of the camera array as the origin of the coordinate system, the horizontal plane as the XY plane, and the exact center of the XY plane as the origin of the coordinate system; in the illustration, the vertically upward direction is the positive Z-axis, and all aerial grid layers are arranged upward along the positive Z-axis with this origin as the reference, forming a bottom-up three-dimensional space; bottom-up grid units (such as...) are set up. Figure 2 The “□” and “○” in the diagram represent the grid cells where the target is located. The resolution is 0.1m to 1m. The grid cells are arranged in layers from near the ground to high altitude along the positive Z-axis to form a three-dimensional grid array. This array covers the low-altitude control airspace with a horizontal range of ±500m (X-axis direction) and a vertical range of 0m to 500m (Z-axis direction), thus completing the three-dimensional gridded modeling of the low-altitude airspace.

[0021] Please see Figure 2 , Figure 2 The "●" in the figure represents the unique origin of the coordinate system. This origin is strictly located on the Z=0m ground reference plane, which is both the exact center of the XY horizontal plane and the geometric center of the ground multi-view camera array. The three coincide completely, forming the unique spatial reference for the global upward-looking grid. All aerial grid layers are arranged upward along the positive Z-axis with this origin as the reference. Figure 2 The XY plane is the ground reference horizontal plane, covering a horizontal range of ±500m, and is the horizontal projection reference plane for looking up at the grid; the Z axis is vertically upward as the positive direction, covering a vertical height from 0m to 500m, corresponding to the low-altitude control airspace; Figure 2 In the diagram, "□" represents a single upward-looking grid cell with a spatial resolution of 0.1m to 1m, arranged in layers along the positive Z-axis from near the ground to high altitude, forming a three-dimensional grid array; "○" represents a grid cell containing a low-altitude target (such as a drone), which stores information such as the target's three-dimensional coordinates and features. In summary, Figure 2 The aerial grid shown represents an upward-looking perspective from the ground, perfectly matching the perception logic of ground-based multi-camera aerial detection.

[0022] In one example, this application deploys multiple (at least two, the specific number depending on the controlled airspace range and accuracy requirements) high-definition visible light cameras with servo pan-tilt units on the ground in the target control area, arranged in a regular array (such as rectangular or circular, with the array size adapted to the camera's field of view and control range). The pitch angle of a single camera is adjustable from 0° to 90°, the horizontal field of view is 90°, and the vertical field of view is 60°, collectively covering a low-altitude control airspace of ±500m horizontally and 0m to 300m vertically. The cameras have a resolution of no less than 2 million pixels, and the specific pixel specifications can be flexibly selected according to the requirements for detecting small targets at long distances.

[0023] Step 110 further completes the calibration of the intrinsic parameters and distortion coefficients of multiple cameras, as well as the joint calibration of the extrinsic parameters between cameras, and establishes the spatial mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system, providing a reference for subsequent unified feature projection. This mapping relationship is used to indicate the mathematical transformation from pixel coordinates to grid coordinates (e.g., which grid cell (X,Y,Z) in the air corresponds to a certain pixel (u,v)).

[0024] Step 120: Based on the mapping relationship of each camera, project the image feature maps of multiple cameras synchronously acquired onto the aerial upward-looking grid coordinate system to generate a globally unified upward-looking grid feature map.

[0025] In one example, this application performs intrinsic parameter calibration (including focal length, principal point, and distortion coefficient) and distortion coefficient calibration for each camera, and completes joint extrinsic parameter calibration of multiple cameras through a 3D calibration board, unifying their extrinsic parameters under a world coordinate system (aerial upward-looking grid coordinate system), and ensuring that their relative pose relationships are self-consistent, that is, camera A and camera B simultaneously photograph the same calibration object; further, the relative pose relationships between each camera are determined, thereby establishing the projection mapping relationship from all camera pixel coordinate systems to the global upward-looking grid coordinate system.

[0026] In a specific example, the mapping relationship is established through the following steps: by performing intrinsic parameter calibration on each camera, the transformation relationship between the pixel coordinate system of each camera and the camera coordinate system of the camera itself is determined; by performing joint extrinsic parameter calibration on multiple cameras simultaneously, the pose transformation relationship between the camera coordinate system of each camera and the aerial upward-looking grid coordinate system is determined; based on the transformation relationship and the pose transformation relationship, the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system is established.

[0027] Specifically, the intrinsic parameters of multiple cameras are calibrated one by one to obtain the internal parameters and distortion coefficients of each camera, and to determine the transformation relationship between the pixel coordinate system of a single camera and its own camera coordinate system. In particular, the intrinsic parameters such as the camera focal length (the distance from the center of the lens to the imaging plane, usually in pixels), principal point coordinates (the intersection of the optical axis and the imaging plane, i.e. the center point of the photo), radial and tangential distortion (image deformation caused by lens optical distortion) coefficients are obtained through calibration. The projection transformation rules from the two-dimensional pixel coordinates of the camera to the three-dimensional camera coordinate system are established to eliminate the camera's own imaging distortion and pixel coordinate deviation. Further, joint extrinsic parameter calibration was performed simultaneously on multiple cameras to determine the pose transformation relationship between each camera's coordinate system and the global aerial view grid (WEV) coordinate system. Specifically, based on a 3D calibration board, the relative pose of multiple cameras was jointly calculated to obtain the rotation matrix and translation vector between each camera. Simultaneously, the relative pose relationship (such as rotation and translation) of each camera's coordinate system relative to the global aerial view grid coordinate system was determined, and all cameras were uniformly aligned to a global spatial reference, achieving global normalization of the spatial positions of multiple cameras. Finally, based on the camera coordinate system transformation relationship and pose transformation relationship, a complete mapping relationship from each camera's pixel coordinates to the global aerial view grid coordinates was established through cascaded transformation calculations. This mapping relationship provides the spatial transformation basis for subsequent unified projection of image features into the global aerial view grid space, ensuring that multi-camera features can be aggregated to the same global spatial reference.

[0028] After establishing the projection mapping relationship between all camera pixel coordinate systems and the global upward-looking grid coordinate system, multiple cameras are controlled to synchronously acquire continuous video frames at the same frame rate. For each continuous video frame acquired by each camera, multi-scale visual features of each frame are extracted to generate an image feature map for each camera. In a specific example, this application uses hardware triggering to achieve synchronous acquisition of multiple cameras to obtain continuous video frames in the low-altitude airspace. Furthermore, a backbone network (such as ResNet50) combined with a feature pyramid network (such as Feature Pyramid Network, FPN) is used to extract multi-scale visual features of each frame, outputting the corresponding number of image feature maps for each camera, thereby preserving the detailed and semantic features of small targets at low altitudes.

[0029] Specifically, this application uses hardware triggering to control multiple ground cameras to synchronously acquire continuous video frames in the low-altitude airspace at the same frame rate, ensuring that the timestamps of all video frames are aligned, eliminating timing discrepancies between multiple cameras, and providing a precise timing basis for subsequent feature space alignment and unified projection of multiple cameras. The hardware triggering method employs one of the following: external synchronization pulse triggering, camera cascade triggering, or industrial control computer digital I / O triggering, providing a unified acquisition trigger signal for multiple cameras. For each frame acquired by each camera, a backbone network (such as ResNet50) combined with a feature pyramid network (such as FeaturePyramid Network, FPN) is used to extract multi-scale visual features of the image, fully preserving the detailed and semantic features of small low-altitude targets, ultimately generating an image feature map corresponding to each camera.

[0030] After generating the image feature map for each camera, based on the mapping relationship established in step 110, the image feature maps of each camera are uniformly projected onto the aerial upward grid coordinate system through the upward view frustum projection transformation (upward projection) or the Transformer learnable projection scheme, and the features of multiple cameras are spatially aggregated. Then, based on the feature denoising and enhancement algorithm, the upward grid feature map is denoised and enhanced. For example, feature denoising and enhancement are completed through one or more algorithms such as 3D convolution, channel attention or multi-scale feature fusion, thereby generating a globally unified upward grid feature map.

[0031] In a specific example, a globally unified upward-looking raster feature map is generated through a scheme of upward-looking frustum projection transformation (upward projection). This is determined by the following steps: based on the transformation relationship, the depth of each pixel in the image feature map along the line of sight in the camera coordinate system is predicted, and the two-dimensional image features of each pixel are up-sized into a three-dimensional frustum feature volume in the camera coordinate system; based on the pose transformation relationship, the three-dimensional frustum feature volume is transformed from the camera coordinate system to the aerial upward-looking raster coordinate system, and the features of multiple cameras within the same raster are accumulated and aggregated to obtain a globally unified upward-looking raster feature map.

[0032] Specifically, this application employs the LSS (Lift-Splat-Shoot) dimensionality-upgrading projection scheme based on the transformation relationship between the pixel coordinate system of a single camera and its camera coordinate system. For each pixel in the image feature map, it predicts the depth probability distribution along the line-of-sight direction within the height range (0m to 300m) in the camera coordinate system (i.e., each pixel predicts multiple possible depths and their probabilities). The two-dimensional pixel-level image features are then dimensionally upgraded to generate a three-dimensional frustum feature volume (a set of feature data with a three-dimensional spatial grid structure, i.e., voxelization) that matches the camera's upward field of view, completing the feature dimensionality-upgrading modeling from the two-dimensional image plane to the three-dimensional camera space. Subsequently, based on the pose transformation relationship between the camera coordinate system and the global aerial upward grid coordinate system, the three-dimensional frustum feature volumes generated by each camera are uniformly transformed to the aerial upward grid coordinate system and projected into the corresponding three-dimensional grid cells. The feature data from multiple cameras falling into the same grid cell are accumulated and aggregated to achieve global spatial fusion of multi-camera visual features. Finally, the projected upward-looking grid feature map is denoised and enhanced by 3D convolution (such as 3 layers of 3×3×3 3D convolution) and channel attention to generate a globally unified (256 channels) aerial upward-looking grid feature map.

[0033] In another specific example, a globally unified upward-looking grid feature map is generated using a Transformer-learnable projection scheme. This is achieved through the following steps: A set of query vectors is generated based on an aerial upward-looking grid coordinate system. Each query vector corresponds to a 3D grid cell in the aerial upward-looking grid coordinate system, and the query vector contains the center 3D coordinates of the corresponding 3D grid cell. Based on transformation and pose transformation relationships, the center 3D coordinates of each 3D grid cell are projected onto the pixel coordinate system of each camera, and the corresponding 2D pixel reference point coordinates are calculated. Using the 2D pixel reference point coordinates as the center, local feature sampling is performed on the feature maps of the images acquired by each camera, and the sampled features are used as key and value vectors. Spatial cross-attention calculation is performed between the query vectors and the sampled key vectors to obtain the attention weights of each 3D grid cell and the corresponding pixel regions in each camera image. The features of the corresponding pixel regions are then weighted and aggregated according to the attention weights to generate a globally unified upward-looking grid feature map.

[0034] Specifically, this example implicitly maps pixel features to the upward-looking grid space by performing cross-attention computation between a learnable query vector (WEV Query) and multi-camera image features, without requiring explicit depth estimation. In this example, this application predefines a set of learnable WEV Query tensors, whose spatial dimensions are completely consistent with the dimensions of the global aerial upward-looking grid array defined in step 110. For example, if the size of the global upward-looking grid is W×H×D (where W is the number of grids in the X-axis direction, H is the number of grids in the Y-axis direction, and D is the number of height layers in the Z-axis direction), then the shape of the WEV Query is W×H×D×C (C is the number of feature channels). Each element in this query vector uniquely corresponds to a three-dimensional grid cell in the global upward-looking grid array, serving as the "identity identifier" of that grid cell in the feature space. The WEV Query is continuously optimized during training, eventually learning the spatial location and semantic features corresponding to each grid cell. The multi-scale image feature maps obtained in step 120 are transformed into unified-dimensional Key and Value features through linear projection, preserving local details and global semantic information. The WEV Query corresponding to each WEV grid is used as the Query, and the Key and Value obtained from the multi-camera image feature encoding are used as input to perform spatial cross-attention calculation. During the attention calculation, the WEV Query corresponding to each grid cell is automatically associated and matched with all pixels in all camera image features, calculating the attention weight between the grid cell and each image pixel. The higher the weight, the greater the probability that the image pixel's feature corresponds to the spatial location of the corresponding grid cell. Based on the calculated attention weights, the pixel features of all camera images are weighted and summed, and the sum is assigned to the corresponding WEV grid cell. After all grid cells have completed feature assignment, a globally unified aerial upward-looking grid feature map is generated. This scheme does not require explicit prediction of the depth value of each pixel. The mapping relationship from pixel to upward-looking grid space is implicitly learned by the model through data-driven means. The depth information is implicit in the distribution of attention weights, effectively avoiding the projection bias caused by explicit depth estimation errors.

[0035] Step 130: Based on the position and motion state of the target detected by the detection head in the upward-looking grid feature map, generate the continuous motion trajectory of the target in the upward-looking grid coordinate system in the air.

[0036] Specifically, the global upward-looking grid feature map is input into a Transformer detector or a 3DCNN detector to complete the detection, classification, and 3D bounding box regression of low-altitude UAV targets, obtaining the target's 3D coordinates, physical dimensions, motion speed, heading angle, and other motion states in the upward-looking grid coordinate system. Any tracking algorithm, such as SimpleOnline and Realtime Tracking (SORT), Deep SimpleOnline and Realtime Tracking (Deep SORT), Byte Tracking, or Observation-Centric SORT (OC-SORT), is used to match and associate the detected targets within the global upward-looking grid 3D space. The trajectory is smoothed and completed by combining the target's historical motion states, generating a continuous 3D motion trajectory of the target in the global coordinate system.

[0037] Please see Figure 3 After step 130, step 140 is also included: based on the continuous motion trajectory of the target and based on the gimbal control algorithm, control multiple cameras to work in sync, so that the target is always within the overlapping field of view of multiple cameras, and update the spatial mapping relationship of the cameras accordingly.

[0038] Specifically, based on the continuous three-dimensional motion trajectory of the detected target, gimbal control algorithms such as PID (Proportion Integral Differential) or model predictive control (MPC) are used to synchronously control the servo gimbals of multiple cameras. By combining the historical motion trajectory of the detected target to predict its spatial position at the next moment, the pitch and azimuth angles of the servo gimbals are adjusted in advance to ensure that the target is always within the overlapping area of ​​the upward-looking fields of view of multiple cameras, guaranteeing continuous synchronous observation of the target by multiple cameras. After the servo gimbal pose is adjusted, the extrinsic parameters of each camera are calculated and updated in real time, and the spatial mapping relationship between the pixel coordinate system of each camera and the global aerial upward-looking grid coordinate system is synchronously corrected. This ensures that the spatial reference for subsequent image feature projection, target detection, and trajectory tracking remains accurate and effective, forming a closed-loop continuous tracking system and achieving long-term stable locking of low-altitude targets.

[0039] Through one or more of the above embodiments, a multi-camera low-altitude UAV tracking method is implemented. This method proposes a unified aerial view grid (WEV) coordinate system and constructs a bottom-up global airspace perception model, thereby completely eliminating the ground plane prior constraints relied upon by the traditional BEV architecture, adapting to the scenario requirements of ground-based multi-eye air tracking, and filling the gap in existing technologies. Based on this, this method achieves unified fusion of multi-camera images in a global view space at the feature level. With the strong constraint of the global 3D coordinate system, it solves the problems of target association errors and frequent ID switching caused by independent processing of multiple cameras. In scenarios with partial occlusion or single-camera failure, the target tracking continuity is improved. Simultaneously, this application completes target detection, 3D localization, and trajectory tracking in the global view grid space, effectively eliminating the inherent perspective distortion and scale ambiguity of the pixel coordinate system, significantly improving the 3D localization accuracy for distant low-altitude small targets, and significantly enhancing the ability to resist sudden changes in light, backlighting, and complex sky background interference. Furthermore, the globally unified 3D airspace perception capability provided by this method can simultaneously support the stable parallel tracking of dozens of dense low-altitude targets. It is widely adaptable to complex urban low-altitude environments and security control needs in various scenarios such as industrial parks, airports, ports, and prisons, demonstrating strong scene generalization capabilities. Finally, this method is fully compatible with existing multi-camera aerial tracking hardware solutions. Deployment can be achieved simply by upgrading the algorithm on the existing camera array and servo gimbal hardware, requiring no additional hardware investment. It boasts significant advantages such as low deployment cost, practicality, and strong scalability.

[0040] This application specification also provides a multi-camera low-altitude unmanned aerial vehicle (UAV) tracking system, such as... Figure 4 As shown, the system includes: a ground-based multi-camera array module containing multiple cameras for acquiring image feature maps; a calibration and spatial alignment module for establishing a three-dimensional aerial upward-looking grid coordinate system based on the camera array composed of multiple cameras deployed on the ground, and determining the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system; a feature extraction and projection module for projecting the image feature maps acquired synchronously by multiple cameras onto the aerial upward-looking grid coordinate system based on the mapping relationship of each camera, generating a globally unified upward-looking grid feature map; and a perception and tracking module for detecting the position and motion state of the target in the upward-looking grid feature map by a detection head, and generating a continuous motion trajectory of the target in the aerial upward-looking grid coordinate system. In some embodiments, the system also includes a gimbal linkage control module for controlling multiple cameras to perform synchronous linkage based on the continuous motion trajectory of the target and the gimbal control algorithm, so that the target is within the overlapping area of ​​the field of view of multiple cameras, and updating the mapping relationship.

[0041] It is not difficult to see that the multi-camera low-altitude UAV tracking system is a system embodiment corresponding to the aforementioned multi-camera low-altitude UAV tracking method. This embodiment can be implemented in conjunction with the aforementioned method. The relevant technical details mentioned in the aforementioned method are still valid in this embodiment, and will not be repeated here to avoid repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the aforementioned method.

[0042] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problem proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0043] This application also provides an electronic device, such as Figure 5 As shown, it includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the multi-camera low-altitude UAV tracking method as described above.

[0044] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0045] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0046] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A multi-camera low-altitude UAV tracking method, characterized in that, include: Using a camera array consisting of multiple cameras deployed on the ground as a reference, a three-dimensional aerial upward-looking grid coordinate system is established, and the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system is determined. Based on the mapping relationship of each camera, the image feature maps of the multiple cameras are simultaneously acquired and projected into the aerial upward-looking grid coordinate system to generate a globally unified upward-looking grid feature map. Based on the position and motion state of the target detected by the detection head in the upward-looking grid feature map, the continuous motion trajectory of the target in the upward-looking grid coordinate system in the air is generated.

2. The multi-camera low-altitude UAV tracking method according to claim 1, characterized in that, The continuous motion trajectory is generated based on a tracking algorithm; After generating the continuous motion trajectory, the method further includes: Based on the target's continuous motion trajectory and the gimbal control algorithm, the multiple cameras are controlled to work synchronously, so that the target is within the overlapping field of view of the multiple cameras, and the mapping relationship is updated.

3. The multi-camera low-altitude UAV tracking method according to claim 1, characterized in that, Determining the mapping relationship between the pixel coordinate system of each of the multiple cameras and the aerial upward-looking grid coordinate system includes: By calibrating the intrinsic parameters of each camera, the transformation relationship between the pixel coordinate system of each camera and the camera coordinate system of the camera itself is determined; By simultaneously performing joint extrinsic parameter calibration on the multiple cameras, the pose transformation relationship between the camera coordinate system of each camera and the aerial upward-looking grid coordinate system is determined. Based on the transformation relationship and the pose transformation relationship, a mapping relationship is established between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system; wherein, The intrinsic parameter calibration of each camera includes: focal length calibration, principal point calibration, and distortion coefficient calibration.

4. The multi-camera low-altitude UAV tracking method according to claim 3, characterized in that, The step of projecting the image feature maps synchronously acquired by the multiple cameras onto the aerial upward-looking grid coordinate system to generate a globally unified upward-looking grid feature map includes: Based on the transformation relationship, the depth of each pixel in the image feature map along the viewing direction in the camera coordinate system is predicted, and the two-dimensional image features of each pixel are upgraded to three-dimensional view frustum features in the camera coordinate system. Based on the pose transformation relationship, the three-dimensional view frustum feature volume is transformed from the camera coordinate system to the aerial upward-looking grid coordinate system, and the features of multiple cameras within the same grid are accumulated and aggregated to obtain the globally unified upward-looking grid feature map.

5. The multi-camera low-altitude UAV tracking method according to claim 3, characterized in that, The step of projecting the image feature maps synchronously acquired by the multiple cameras onto the aerial upward-looking grid coordinate system to generate a globally unified upward-looking grid feature map includes: Based on the aerial upward-looking grid coordinate system, a set of query vectors is generated, wherein each query vector corresponds to a three-dimensional grid cell in the aerial upward-looking grid coordinate system, and the query vector contains the center three-dimensional coordinates of the corresponding three-dimensional grid cell. Based on the transformation relationship and the pose transformation relationship, the center three-dimensional coordinates of each three-dimensional grid unit are projected into the pixel coordinate system of each camera, and the corresponding two-dimensional pixel reference point coordinates are calculated. Using the coordinates of the two-dimensional pixel reference point as the center, local feature sampling is performed in the image feature maps acquired by each camera, and the sampled features are used as key vectors and value vectors. Spatial cross-attention calculation is performed on the query vector and the sampled key vector to obtain the attention weight of each three-dimensional raster unit and the corresponding pixel region in each camera image. The features of the corresponding pixel region are then weighted and aggregated according to the attention weight to generate the globally unified upward-looking raster feature map.

6. The multi-camera low-altitude UAV tracking method according to claim 1, characterized in that, The image feature maps acquired simultaneously by the multiple cameras are obtained through the following methods: Control the multiple cameras to synchronously acquire continuous video frames at the same frame rate; For each continuous video frame captured by each camera, multi-scale visual features of each frame are extracted to generate an image feature map for each camera.

7. The multi-camera low-altitude UAV tracking method according to any one of claims 1 to 6, characterized in that, The establishment of a three-dimensional aerial upward-looking grid coordinate system, based on a camera array consisting of multiple cameras deployed on the ground, includes: A coordinate system is established with the geometric center of the camera array as the origin, the horizontal plane as the XY plane, and the vertical upward direction as the positive Z-axis, to form a three-dimensional grid array distributed along the Z-axis. The grid spatial resolution of the three-dimensional grid array is 0.1m to 1m, and the coverage range of the three-dimensional grid array is: ±500m in the X-axis direction, ±500m in the Y-axis direction, and 0m to 500m in the Z-axis direction.

8. The multi-camera low-altitude UAV tracking method according to claim 2, characterized in that, The detection head is either a Transformer or a 3D CNN. The tracking algorithm is one of SORT, DeepSORT, ByteTrack, or OC-SORT. The gimbal control algorithm is either PID or MPC. After generating a globally uniform upward-looking raster feature map, the method further includes: Based on the feature denoising and enhancement algorithm, the upward-looking grid feature map is denoised and enhanced. The feature denoising and enhancement algorithm is one of 3D convolution, channel attention, or multi-scale feature fusion.

9. A multi-camera low-altitude unmanned aerial vehicle (UAV) tracking system, characterized in that, The system includes: A ground-based multi-camera array module, comprising multiple cameras, is used to acquire image feature maps; The calibration and spatial alignment module is used to establish a three-dimensional aerial upward-looking grid coordinate system based on a camera array consisting of multiple cameras deployed on the ground, and to determine the mapping relationship between the pixel coordinate system of each camera and the aerial upward-looking grid coordinate system. The feature extraction and projection module is used to project the image feature map synchronously acquired by the multiple cameras onto the aerial upward-looking grid coordinate system based on the mapping relationship of each camera, so as to generate a globally unified upward-looking grid feature map. The perception and tracking module detects the position and motion state of the target in the upward-looking grid feature map by the detection head, and generates the continuous motion trajectory of the target in the upward-looking grid coordinate system in the air.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the multi-camera low-altitude UAV tracking method as described in any one of claims 1 to 8.

Images