Unmanned aerial vehicle three-dimensional target perception method based on dynamic parameter adaptation

Abstract

The application discloses a kind of unmanned aerial vehicle three-dimensional target perception methods based on dynamic parameter self-adaption, it includes extracting current frame unmanned aerial vehicle image feature, according to the height of current frame flight generation physical height center adaptive update candidate height layer, and predict the height probability of each feature position;According to the attitude information of inertial measurement unit, correct pixel back-projection ray, combined with flight height and candidate height layer, construct the local three-dimensional feature point set of current frame;Voxelization and vertical aggregation are carried out to the three-dimensional feature point set, generate bird's eye view feature map, and output target confidence, center offset, geometric size and yaw angle, obtain the local three-dimensional target frame of current frame.This method can reduce the influence of flight height and attitude change on three-dimensional projection and target frame estimation.

Description

Technical Field

[0001] This invention belongs to the field of UAV image perception and computer vision technology, specifically relating to a UAV 3D target perception method based on dynamic parameter adaptation. Background Technology

[0002] Unmanned aerial vehicle (UAV) visual target perception typically relies on images captured by airborne cameras to detect and locate roads, vehicles, pedestrians, facilities, or other ground targets. Compared to fixed ground cameras or vehicle-mounted cameras, UAVs offer advantages such as flexible viewing angles, wide coverage, and convenient deployment, enabling them to acquire information about targets and their surrounding environment from overhead or oblique perspectives. With the development of computer vision and deep learning technologies, visual perception methods based on image feature extraction, depth or height cue estimation, bird's-eye view spatial representation, and 3D bounding box regression are increasingly being used to estimate information such as target spatial location, size, and orientation.

[0003] Under monocular or limited visual sensor conditions, recovering the 3D spatial information of a target from a 2D image typically relies on scene geometry, target appearance cues, and feature space transformations. For UAV-based ground observation scenarios, the target's scale, orientation, occlusion status, and imaging deformation in the image vary with flight status, shooting angle, ground undulations, and scene layout. Simultaneously, edge computing platforms place high demands on model complexity and inference efficiency. Therefore, how to form a stable spatial representation under limited visual information and computational resources, and improve the consistency and reliability of 3D target detection results under different observation states, remains a problem that requires further improvement in UAV visual 3D perception. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a UAV three-dimensional target perception method based on dynamic parameter adaptation.

[0005] A UAV 3D target perception method based on dynamic parameter adaptation includes an image feature extraction step: extracting features from the current frame UAV image to obtain the current frame image features; Dynamic altitude layer prediction steps: Generate multiple candidate ground altitude layers based on the current frame flight altitude, and predict the probability of each feature position belonging to the multiple candidate ground altitude layers based on the current frame image features; wherein, the physical altitude center of the multiple candidate ground altitude layers is adaptively updated with the current frame flight altitude. Attitude compensation 3D projection steps: Perform attitude compensation on the pixel back projection rays corresponding to each feature position according to the IMU attitude information, and determine the 3D coordinates of each feature position in the local virtual coordinate system of the current frame based on the attitude-compensated back projection rays, the current frame flight altitude and the candidate ground altitude layer, thereby constructing the local 3D feature point set of the current frame. Bird's-eye view feature detection output steps: Generate a bird's-eye view feature map based on the local 3D feature point set of the current frame, and output a set of local 3D target boxes of the current frame based on the bird's-eye view feature map.

[0006] As one preferred embodiment, the image feature extraction step includes: The current frame UAV image is input into a lightweight backbone network, and a multi-range sparse dynamic attention module is embedded in the later semantic stage of the lightweight backbone network. The multi-range sparse dynamic attention module first divides the main path features into channels, forming retained branches, local square branches, horizontal and vertical strip branches, diagonal sparse branches, and context branches; Then, based on the context anchoring map generated from the main path features, position-related branch selection weights are generated, and the responses of each branch are weighted and fused based on the branch selection weights to obtain structural supplementary features; Finally, the structural supplementary features are written back to the main path features using a gated residual method to obtain the current frame image features.

[0007] As one preferred embodiment, the dynamic height layer prediction step includes: Based on the current frame image features, predict the altitude probability of each feature location on a predetermined number of candidate ground altitude layers; wherein, before inference in each frame, generate the physical altitude center corresponding to the predetermined number of candidate ground altitude layers in the current frame according to the current frame flight altitude and preset altitude distribution parameters.

[0008] As one preferred approach, the attitude compensation 3D projection steps include: Establish a local virtual coordinate system for the current frame. The origin of the local virtual coordinate system is the optical center of the UAV camera in the current frame, the vertical axis points to the ground, and the horizontal axis lies in the horizontal plane of the ground. Based on the downsampling ratio, the feature positions in the current frame image features are mapped to the corresponding image coordinates, and the image coordinates are back-projected based on the camera intrinsic parameter matrix to obtain the corresponding pixel back-projection ray; The direction of the pixel back-projection ray in the local virtual coordinate system of the current frame is corrected based on the IMU attitude information; For any candidate ground altitude layer, the local three-dimensional coordinates of the feature location on the candidate ground altitude layer are determined based on the vertical distance between the current frame flight altitude and the physical altitude center of the candidate ground altitude layer, as well as the vertical component of the attitude-compensated back-projection ray.

[0009] As one preferred embodiment, the dynamic height layer prediction step further includes: The current frame image features are input into the height prediction branch, and the original response scores of each feature position corresponding to a predetermined number of candidate ground height layers are output through the convolutional layer. The scores are then normalized along the height channel dimension to obtain the height probability. The current frame image features are input into the context semantic feature branch to obtain the context semantic features; The height probability is fused with the contextual semantic features position by position to obtain a fused feature that simultaneously contains semantic information and candidate height response confidence.

[0010] As one of the preferred options, it also includes: Within a preset ground altitude coverage area, the physical altitude centers of a predetermined number of candidate ground altitude layers are determined according to a power-law mapping; wherein, the exponent parameter of the power-law mapping is determined based on the flight altitude of the current frame, so that the candidate ground altitude layers are more densely distributed in the low-altitude range when the UAV is flying at low altitude, and the distribution of the candidate ground altitude layers tends to be flat when the UAV is flying at high altitude.

[0011] As one of the preferred solutions, the attitude compensation 3D projection step also includes: A compensation rotation matrix is ​​generated based on the pre-calibrated camera mounting rotation matrix and the current frame attitude increment obtained from the IMU attitude information; The pixel back-projection ray is mapped from the camera coordinate system to the local virtual coordinate system of the current frame using the compensation rotation matrix, so that roll, pitch or yaw changes are reflected in the spatial direction of the pixel back-projection ray.

[0012] As one preferred approach, the bird's-eye view feature detection output step includes: In the local virtual coordinate system of the current frame, the three-dimensional space is divided into multiple voxel lattices along three coordinate directions; The point features of three-dimensional feature points falling within the same voxel grid are averaged to obtain voxel features; if there are no three-dimensional feature points in the voxel grid, the voxel features corresponding to that voxel grid are set to zero. The effective voxel features at the same horizontal position are aggregated along the vertical direction and normalized according to the number of effective voxels to generate the bird's-eye view feature map.

[0013] As one preferred approach, the bird's-eye view feature detection output step also includes: Convolutional feature aggregation is performed on the bird's-eye view feature map to obtain detection features; The confidence branch outputs the probability that a target exists at the location of the bird's-eye view grid. The target center offset, target geometry, and target yaw angle are output through the regression branch. Based on the target center offset and the reference coordinates of the corresponding bird's-eye view grid position or candidate position, the target center coordinates are decoded, and combined with the target geometry and target yaw angle, the local 3D target bounding box of the current frame is obtained.

[0014] The advantages and beneficial effects of this invention are as follows: This invention incorporates the current frame's flight altitude into the construction process of the candidate ground-viewing altitude layer and utilizes IMU attitude information to perform attitude compensation on pixel back-projection rays. This allows the mapping relationship between 2D image features and 3D space to be adaptively updated according to the UAV's current flight altitude and attitude state. Compared to 3D perception methods that use fixed altitude layers or fixed projection relationships, this invention ensures that the candidate altitude representation, pixel ray direction, and local 3D coordinate calculation remain consistent with the actual observation state of the current frame. This reduces the impact of flight altitude changes and roll, pitch, and yaw disturbances on the estimation of the 3D target center position, spatial size, and orientation, improving the accuracy, stability, and intra-frame geometric consistency of 3D target bounding box detection results in UAV top-down or large-angle ground-viewing scenarios. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below with reference to specific embodiments.

[0016] This embodiment provides a UAV 3D target perception method based on dynamic parameter adaptation, applicable to UAV top-down or large-angle ground observation scenarios such as smart city road inspection, traffic target perception, complex intersection observation, and low-altitude patrol. The method takes the current frame UAV image, the flight altitude of the current frame UAV camera optical center relative to the ground, and IMU attitude information synchronized with the current frame UAV image as input, and outputs a set of 3D target bounding boxes in the local virtual coordinate system of the current frame.

[0017] First, image feature extraction is performed on the current frame of the UAV image. Specifically, the current frame of the UAV image... The lightweight backbone network is input, which can adopt an EfficientNet-like network structure or other lightweight convolutional networks with MBConv structures. Considering that UAV platforms are usually deployed on edge computing devices, directly using a large-scale backbone network would increase the computational burden. In top-view road images, structural information such as vehicle top surface boundaries, pedestrian contours, non-motorized vehicle edges, lane lines, and road boundaries have a strong influence on subsequent 3D target box localization. Therefore, this embodiment embeds a multi-scope sparse dynamic attention (MSDA) module in the later semantic stage of the lightweight backbone network.

[0018] Specifically, the lightweight backbone network consists of It consists of several stages, the first... The input features of each stage are ,in For the number of channels, and The spatial dimensions are defined as follows. The front-end stages of the lightweight backbone network primarily handle color, texture, and low-level edges. Structural enhancement at this stage tends to amplify shadows, road textures, and compress noise. The back-end stages already contain semantic information such as vehicles, pedestrians, and road boundaries. Introducing a multi-range sparse dynamic attention module can enhance the target shape and road layout. Specifically, this embodiment retains the standard MBConv in the front-end stage and embeds a multi-range sparse dynamic attention module within the standard MBConv in at least three (preferably three) semantic stages in the back-end stage. Generally, the multi-range sparse dynamic attention module is inserted after depthwise convolution and before dimensionality reduction convolution.

[0019] remember For the current frame time, This is the drone image in the current frame. ,in and These represent the image height and width, respectively. If the enhanced features are output by a lightweight backbone network like EfficientNet, then... In the formula, This indicates a lightweight backbone network that embeds MSDA in the later semantic stage; The number of output feature channels; and This specifies the spatial dimensions of the output feature map. The output retains the original image. The spatial resolution facilitates alignment with subsequent height probability prediction and BEV projection interfaces.

[0020] For the The first core stage to be replaced, let the second stage be... The input features of each stage are First, the main path still uses MBConv's up-dimensional convolution to generate a wider channel representation: in, for Dimensional convolution, For batch normalization, It is a non-linear activation function. These are intermediate features after dimensionality enhancement, used to provide a wider channel representation for subsequent depthwise convolutions and multirange sparse dynamic attention modules (MSDA). Then, local spatial features are extracted further through depthwise convolutions. : In the formula, Represents depthwise convolution. You can use the settings for the corresponding stage of the original EfficientNet. or Depth convolution extracts local spatial features. It retains the local semantic information of the MBConv main path, which is the input feature of MSDA. That is, it uses the up-dimensional convolution and depthwise convolution in the MBConv main path to obtain the main path features.

[0021] Then, MSDA first divides the features along the channel dimension into several sub-features, allowing different structural branches to process only a portion of the channels. This preserves the lightweight nature of MBConv while providing a clear entry point for subsequent heterogeneous structure modeling. in, This indicates the feature division along the channel dimension; To preserve branches, For local square branch input, For horizontal and vertical stripe branch input, For diagonal sparse branch input, This is the input for the context branch. The number of channels for each sub-feature can be set according to the computational budget, but the total number of channels after concatenation is related to... Maintain consistency.

[0022] After channel segmentation, MSDA constructs heterogeneous structural responses on different sub-features. Local square branches preserve compact structures such as vehicle roof areas and small pedestrian targets; horizontal and vertical stripe branches describe lane lines, road edges, and long vehicle sides; diagonal sparse branches supplement diagonal contours appearing at intersections and during tilted-heading shots; contextual branches undergo coarse-scale pooling before restoring to their original spatial dimensions to provide a wider range of road layout information. Specifically, the corresponding calculations are as follows: In the formula, Represents local depthwise convolution. and These represent horizontal and vertical stripe depth convolutions, respectively; and This represents a sparse depthwise convolution that retains effective weights only in the two diagonal directions; Indicates step size or window size. Average pooling, This indicates upsampling back to the original feature size. The above branches cover four types of structures: local, strip, diagonal, and coarse-scale context, but each type of structure only affects a portion of the channels, thus preventing MBConv from becoming a heavy backbone.

[0023] The responses of different branches cannot be simply added together, and the target orientation, road direction, and background interference in UAV images change with flight path and scene variations, making it difficult to adapt fixed fusion weights to all frames. To provide a basis for branch selection, MSDA generates a context anchoring map from the main path features. The context anchoring graph It also includes information on lateral distribution, vertical distribution, and coarse-scale regions: in, This indicates a pooling operation that aggregates along the horizontal direction and broadcasts to the original size. This indicates a pooling operation that aggregates along the vertical direction and broadcasts to the original size. This indicates information used to organize anchoring information. Convolution. Context anchoring graph. It is not used directly as an output feature, but rather to determine which type of structural response is more needed at the current spatial location.

[0024] Obtaining the context anchor graph Subsequently, MSDA generates position-dependent selection weights for the five structural branches. Unlike using only global average pooling to obtain single-channel weights, these weights preserve spatial positional differences, allowing different branch combinations to be used for road edges, target top surfaces, and background regions. In the formula, Used to generate branch selection scores This indicates normalization along the branch dimension. , , , and The proportions of retained branches, local branches, strip branches, diagonal branches, and context branches are controlled separately. Their weights can be broadcast to the channel dimension of the corresponding branch and multiplied element-wise with the branch features.

[0025] After obtaining the branch weights, the responses of each branch are first weighted, and then... Convolution back to the unified channel space forms the structural supplementary features of MSDA. : in, This represents element-wise multiplication. Indicates branch fusion convolution. These are structural supplementary features selected by context-anchored weights. For elongated areas such as lane lines and road edges, strip branches receive higher weights; for sloping vehicle boundaries and turning intersections, diagonal branches provide more supplementary information; for areas with weak texture but clear layout, context branches provide more stable range information.

[0026] To avoid excessive influence of boundary or road textures on category semantics, structural supplementary features are used. Instead of directly replacing the MBConv main path, MSDA further generates a structure injection gate and supplements the structure features using residuals. Write back to the main path: in, Indicates gating mapping, For the Sigmoid function, These are intermediate features after the structural information has been injected. Gating terms are based on the semantics of the main path. Structural supplementary features and context anchoring graph Co-generation allows the module to increase structural response when the target boundary is clear, and retain more main path information in shadow, flat road surface or texture interference areas.

[0027] Finally, MBConv obtains the output features through identity mapping residual bypass. : In the formula, Indicates the original MBConv Dimensionality reduction convolution. This formula represents the output feature. The augmented features are obtained by adding the input features and the dimensionality-reduced augmented features, with the residual bypass employing an identity mapping. This allows the main path semantic information to be preserved while introducing the MSDA structure for supplementation, and the rules for dimensionality increase, depthwise convolution, dimensionality reduction, and residual connection remain unchanged. MSDA is only used as an internal structure selection unit after depthwise convolution. This keeps the number of stages and repetitions of the original EfficientNet unchanged, concentrating the additional computation in the later semantic stages to supplement the boundaries, contours, orientations, and road layout cues of the top-down target. The final augmented features are obtained from the output of the lightweight backbone network. It will be used as input to the flight parameter adaptive 3D projection module.

[0028] Next, a dynamic altitude layer prediction step is performed. For UAV-based top-down or steep-angle ground observation scenarios, the altitude of traffic targets such as vehicles, pedestrians, and non-motorized vehicles is usually within a limited range. Therefore, this embodiment does not directly convert two-dimensional features into arbitrary depth values. Instead, it uses candidate altitude layers as intermediate geometric quantities to upgrade two-dimensional features to three-dimensional space. That is, it does not simply replace depth quantities with altitude quantities, but further incorporates the UAV's current frame flight state into the altitude layer construction and pixel projection process: candidate altitude set. The height of the drone camera's optical center relative to the ground in the current frame and height distribution parameters The pixel ray direction is recalculated and corrected using the current IMU pose increment. The height layer includes frame time indices. The physical meaning of each altitude channel is dynamically updated with the flight status. The flight altitude can be obtained from the UAV flight control system, RTK, barometer, laser rangefinder or a combination thereof; the IMU attitude information is used to reflect the roll, pitch and yaw changes of the UAV in the vicinity of the current frame.

[0029] Specifically, the highly predictive network HeightNet enhances the output features of a lightweight backbone network. The input is a two-branch parallel expansion. The first branch is the height distribution prediction branch, which outputs feature locations through convolutional layers, such as estimating the corresponding target location for each pixel. The response probability at each candidate ground-to-height layer represents the relative confidence that the corresponding pixel location falls into different candidate ground-to-height layers. This branch passes through two layers. Convolutional processing extracts highly correlated local spatial features, followed by batch normalization (BN) and Rectified Linear Activation (ReLU) after each convolutional layer; then passed through a single layer... Convolution maps the number of channels to Output the original response scores for each layer, and finally apply Softmax normalization along the channel dimension. For any pixel in the image... (in For column coordinates, (where the coordinates are row coordinates, and the two together determine the feature position of the pixel on the image plane), and its height probability vector is... , among which, the The component indicates that the target corresponding to this feature location falls into the first... The conditional probabilities of each candidate ground-based height layer are calculated. By organizing the height probability vectors of all feature locations on the enhanced feature map according to a spatial grid, the height probability prediction tensor corresponding to the current frame can be obtained. .

[0030] The second branch is the contextual semantic feature branch, which enhances features through a squeeze-and-excitation (SE) layer. Perform channel adaptive weighting to extract contextual semantic features related to the target attribute and output a contextual feature map. ,in This represents the number of context feature channels. The outputs of the two branches are fused through an outer product operation, which means fusing the outputs pixel-by-pixel in the spatial dimension. Dimensional context features and The dimensional probability vectors are multiplied element-wise and broadcast to generate a fused feature map. : , In the formula, This represents position-wise outer product fusion, resulting in a fused feature map. It also carries the semantic description of the pixels and the response confidence of the candidate ground height layer, providing semantic and geometric weights for subsequent 3D projection.

[0031] Candidate Earth-viewing height layer center height set The adaptive highly discretization strategy is applied before inference in each frame based on... Dynamically generated. The candidate Earth altitude layer coverage area is set to... Typical values ​​are taken above ground level. to The height is measured in meters, covering the ground elevation distribution range of common targets such as vehicles, pedestrians, and non-motorized vehicles in urban traffic scenarios. Within this range, the center height of the candidate ground elevation layer is determined by a power-law mapping: in, For the current frame number The physical height center of the candidate ground-level height layer The number of candidate Earth-viewing height layers. The distribution index is determined based on the UAV's flight altitude in the current frame. The altitude prediction branch... The output channel and the current frame's first... Candidate Earth-viewing height layer One-to-one correspondence.

[0032] Distribution index Determined by the drone's flight altitude in the current frame: In the formula, Basic index, For adjustment coefficients, The maximum operating altitude of the drone set for the system, and the upper limit of the target altitude. The meanings are different. At low altitudes, the target occupies more pixels in the image, and the altitude difference has a more direct impact on the 3D bounding box center and size estimation, making the candidate ground-level altitude layers denser in the low-altitude range. At high altitudes, the target imaging scale decreases, and the distribution of candidate ground-level altitude layers tends to be flatter, reducing redundant use of the altitude channel. Therefore, It remains the same fixed number of channels for network output, while The physical center value of each channel changes with the flight altitude of the current frame.

[0033] Therefore, the highly predictive branch of the first The output channel and the current frame's first... Candidate Earth-viewing height layer One-to-one correspondence. For fixed cameras, the physical center value of the altitude channel can remain constant over a long period; for drone platforms, the physical center value of each frame... All altitudes are redefined based on the current flight altitude. Therefore, the altitude probability no longer corresponds to a fixed discrete channel meaning, but rather to a ground altitude probability that carries the meaning of the current flight state.

[0034] Through the dynamic altitude layer prediction steps described above, each feature location in the current frame image features not only possesses semantic features but also has a candidate altitude probability that matches the current frame's flight altitude. This processing enables altitude representation to adapt to different observation states of the UAV, such as low-altitude and high-altitude, reducing altitude representation mismatches caused by fixed altitude channels at different flight altitudes.

[0035] Subsequently, the attitude-compensated 3D projection step is performed. In this embodiment, the 3D projection is completed in the local virtual coordinate system of the current frame. The origin of the local virtual coordinate system of the current frame is set as the optical center of the UAV camera in the current frame, the vertical axis points to the ground, and the horizontal axis lies in the horizontal plane of the ground. Since the position and attitude of the UAV continuously change during flight, the local virtual coordinate system of the current frame moves with the frame. The 3D coordinates obtained in this embodiment are used to describe the local position of the target in the current frame relative to the camera in the current frame.

[0036] Complete the projection in the local virtual coordinate system of the current frame, index... Used to label this coordinate system. The origin of this coordinate system. Set as the optical center of the drone camera in the current frame. The axis points vertically towards the ground. shaft and The axes lie within the horizontal plane of the ground. Because the drone is in continuous flight, the local virtual coordinate system moves with each frame; therefore, the 3D coordinates obtained in this section only describe the local position of the target relative to the current camera in the current frame.

[0037] Let the intrinsic parameter matrix of the drone camera be... The pre-calibrated camera mounting rotation matrix is The IMU obtains the attitude increment based on the integral of the angular velocity near the current frame. ,in This represents a set of 3D rotation matrices. The pose increment is not output as a separate detection result, but is used to correct the direction of the backprojected ray for each pixel in the current frame. The compensated rotation matrix is ​​defined as follows: in, The compensated rotation matrix is ​​used to map the pixel back-projection rays from the camera coordinate system to the local virtual coordinate system of the current frame. This compensated rotation matrix is ​​directly incorporated into the 3D projection formula, rather than being saved as an independent attitude record. This ensures that roll, pitch, or yaw changes are reflected in the spatial direction of each pixel ray, guaranteeing that subsequent 3D coordinate calculations use the actual observed attitude of the current frame.

[0038] Due to the fusion feature map The spatial dimensions are the same as the original image. fusion feature map location Before projection, the coordinates need to be mapped back to the image coordinates. Let the downsampling factor be... Then the image coordinates corresponding to this feature location are , For this location, the intrinsic parameter matrix is ​​used for backprojection to a reference point at a unit depth, and then the spatial orientation is corrected using a compensation rotation matrix: in, This represents the homogeneous representation of image coordinates. This represents the reference point of the pixel ray at a unit depth. The length of this reference point is not used as the true distance; its direction is used to determine the spatial orientation of the pixel ray in the local virtual coordinate system under the current pose. Denote its... The absolute value of the axis component is ,but in, For a local virtual coordinate system The unit vector along the axis. Taking the absolute value ensures the scaling ratio is positive. For the ... There are 1 candidate ground-level altitude layers, and the vertical distance from the optical center of the drone camera in the current frame to that altitude layer is . Based on the relationship between similar triangles, the position... In the Local three-dimensional coordinates on candidate Earth height layers for, This formula realizes the dynamic projection link of the present invention, molecule As the current frame's flight altitude and candidate ground altitude layers change together, the denominator... The pixel rays change with the pose compensation of the current frame. Compared with the projection method using a fixed height and fixed extrinsic parameters with a fixed camera, it recalculates the geometric relationship from two-dimensional features to three-dimensional space for each frame, so that the generated three-dimensional feature points can correspond to the actual observation state of the UAV in the current frame.

[0039] Each 3D point inherits and merges the feature map. The fusion features corresponding to the pixel-height layer positions: In the formula, This is the feature indexing function, which assigns corresponding semantic features to local 3D points; Indicates taking the fused feature map In frame t, all semantic channels (represented by:), the... Candidate Earth-viewing height layers and spatial locations The eigenvector at that location.

[0040] Repeat the above calculation for all spatial locations and all candidate ground-to-ground height layers to obtain the local three-dimensional feature point set of the current frame. The 3D coordinates of each feature location are no longer determined solely by fixed extrinsic parameters or fixed altitude relationships, but are recalculated based on the current frame's attitude and flight altitude. This reduces pixel ray direction deviations caused by UAV attitude disturbances and improves the correspondence between the local 3D point set and the actual observation state. The local virtual coordinate system of the current frame, the IMU attitude compensation back-projection ray, and the local 3D coordinates calculated based on flight altitude and candidate ground altitude layers are used. This local 3D feature point set of the current frame simultaneously contains 3D coordinates and corresponding features, serving as input for subsequent BEV detection.

[0041] Next, the bird's-eye view feature detection output step is executed. Since the 3D points in the local 3D feature point set of the current frame are usually irregularly distributed in space, making them difficult to directly input into a regular convolutional detection head, this embodiment performs voxelization processing on the 3D space under the local virtual coordinate system of the current frame. Specifically, the 3D space is divided into multiple voxel grids along the three coordinate directions, and the point features of 3D feature points falling within the same voxel grid are averaged and aggregated to obtain the voxel features of the corresponding voxel grid; if there are no 3D feature points in a voxel grid, the voxel features corresponding to that voxel grid are set to zero.

[0042] Input points received in this stage The system already incorporates the drone's current frame's flight altitude and attitude compensation information. The role of the conventional detection head here is to organize the irregular dynamic features generated by the front-end module into a regular BEV plane, thereby decoding the target position. Therefore, this step is responsible for receiving the spatial correction results from the front end and is a necessary step in the processing chain of this solution to convert two-dimensional cues into three-dimensional target bounding boxes.

[0043] Subsequently, the effective voxel features at the same horizontal position are aggregated along the vertical direction to generate a bird's-eye view feature map (BEV feature map). To avoid response amplitude deviations caused by different numbers of effective vertical voxels in different horizontal grids, this embodiment normalizes the aggregation results based on the number of effective voxels. The resulting bird's-eye view feature map retains the semantic information within each horizontal grid while reducing the amplitude bias caused by differences in 3D point density, making it more suitable for subsequent 3D target detection.

[0044] After obtaining the bird's-eye view feature map, it is input into the detection head. The detection head can include a shared convolutional layer, a confidence branch, and a regression branch. The shared convolutional layer is used to perform local feature aggregation on the bird's-eye view feature map; the confidence branch is used to output the probability of a target existing at a location on the bird's-eye view grid; and the regression branch is used to output the target center offset, target geometry, and target yaw angle. During decoding, the target center offset is superimposed onto the reference coordinates of the corresponding bird's-eye view grid location or candidate location to obtain the target center coordinates. Combined with the target geometry and target yaw angle, the local 3D target bounding box of the current frame is obtained.

[0045] The local 3D feature point set of the current frame in the virtual coordinate system has been obtained. These points contain joint features of 3D coordinates and semantic height, but their spatial distribution is irregular and cannot be directly fed into a regular convolutional detection head. In order to output a 3D target bounding box, the irregular point set needs to be transformed into regular BEV features, and the target center, size, and yaw angle need to be regressed on the horizontal plane.

[0046] In the drone's overhead view, the regular BEV feature representation is of direct significance: it expresses the arrangement of targets on the road plane as a regular grid, and the spatial distance and orientation relationship between targets is closer to the real ground coordinates.

[0047] To transform irregular 3D point sets into regular features that can be processed by the detection head, this invention uses a local virtual coordinate system to transform the 3D space along... , , The three directions are divided into , , Individual voxel grids. To clearly distinguish continuous three-dimensional physical coordinates, discrete voxel grid indices are introduced. For index is The voxel lattice is used to calculate the average value of the features of the three-dimensional points falling within it, thus obtaining the voxel features. , in, This represents the set of three-dimensional points that fall within the voxel lattice. Indicates the number of points; if the set is empty, the corresponding voxel feature is set to zero.

[0048] Subsequently, voxel features are aggregated along the vertical direction to generate a bird's-eye view feature map. To avoid response amplitude deviations at different horizontal grid points due to varying effective vertical voxel counts, this invention employs mean pooling with effective voxel count normalization: in, Indicates at horizontal grid points Above, there exists a set of vertical voxel indices for a valid set of points; This represents the number of valid vertical voxels in the set. If... If the set is empty, the corresponding BEV feature is set to zero. This represents the BEV representation of the current frame on the local horizontal plane. This representation preserves the semantic information within each horizontal grid while mitigating the magnitude bias caused by differences in point density, providing rule input for 3D bounding box detection.

[0049] Then, the detection head is in The outputs target confidence and 3D bounding box regression values. Specifically, the shared convolutional layer first locally aggregates the BEV features, then splits into two parallel branches: the confidence branch outputs the probability of a target existing at each bird's-eye view grid location, and the regression branch outputs the target center offset, geometric dimensions, and yaw angle. For the ... The target is represented by a local 3D bounding box in the current frame. in, The coordinates of the target center in the local virtual coordinate system of the current frame. For the target length, width, and height, Let be the target yaw angle. The center offset of the regression branch output can be denoted as . During decoding, this offset is superimposed onto the reference coordinates of the corresponding grid or candidate position to obtain the target center. The horizontal position is primarily determined by the bird's-eye view grid and its offset, while the vertical position is determined by the projected 3D features and the regression branch. Indicates the height of the target itself. Indicates the candidate ground elevation layer. These three terms—indicating the drone camera's height relative to the ground—have different meanings. Regarding the output... The target confidence score and 3D bounding box regression output by the detection head constitute the complete 3D target detection result of the current frame, which can be directly used for single-frame local perception.

[0050] Through the above process, this embodiment forms a continuous processing link between lightweight feature extraction, dynamic altitude layer prediction, attitude-compensated 3D projection, and bird's-eye view feature detection output. Specifically, the image feature extraction step provides suitable boundary, contour, orientation, and road layout clues for the target when viewed from above by the UAV for altitude prediction and projection; the dynamic altitude layer prediction step updates the candidate ground altitude layer based on the current frame's flight altitude, adapting the altitude probability expression to the current flight state; the attitude-compensated 3D projection step writes the IMU attitude change into the pixel back-projection ray direction, enabling the 3D coordinate calculation to correspond to the actual observed attitude in the current frame; and the bird's-eye view feature detection output step regularizes the irregular 3D point set into a bird's-eye view feature map and outputs the 3D target bounding box. These steps are not simply parallel but are passed step-by-step from the current frame image features, altitude probability, attitude compensation ray, and local 3D point set, collectively improving the accuracy and intra-frame consistency of the 3D target bounding box output in UAV-view or high-angle traffic perception scenarios.

[0051] The present invention has been described above by way of example. It should be noted that any simple modifications, alterations or other equivalent substitutions that can be made by those skilled in the art without creative effort without departing from the core of the present invention fall within the protection scope of the present invention.

Claims

1. A UAV 3D target perception method based on dynamic parameter adaptation, characterized in that, include, Image feature extraction steps: Extract features from the current frame of the UAV image to obtain the current frame image features; Dynamic altitude layer prediction steps: Generate multiple candidate ground altitude layers based on the current frame flight altitude, and predict the probability of each feature position belonging to the multiple candidate ground altitude layers based on the current frame image features; wherein, the physical altitude center of the multiple candidate ground altitude layers is adaptively updated with the current frame flight altitude. Attitude compensation 3D projection steps: Perform attitude compensation on the pixel back projection rays corresponding to each feature position according to the IMU attitude information, and determine the 3D coordinates of each feature position in the local virtual coordinate system of the current frame based on the attitude-compensated back projection rays, the current frame flight altitude and the candidate ground altitude layer, thereby constructing the local 3D feature point set of the current frame. Bird's-eye view feature detection output steps: Generate a bird's-eye view feature map based on the local 3D feature point set of the current frame, and output a set of local 3D target boxes of the current frame based on the bird's-eye view feature map.

2. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 1, characterized in that, The image feature extraction steps include: The current frame UAV image is input into a lightweight backbone network, and a multi-range sparse dynamic attention module is embedded in the later semantic stage of the lightweight backbone network. The multi-range sparse dynamic attention module first divides the main path features into channels, forming retained branches, local square branches, horizontal and vertical strip branches, diagonal sparse branches, and context branches; Then, based on the context anchoring map generated from the main path features, position-related branch selection weights are generated, and the responses of each branch are weighted and fused based on the branch selection weights to obtain structural supplementary features; Finally, the structural supplementary features are written back to the main path features using a gated residual method to obtain the current frame image features.

3. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 1, characterized in that, The dynamic height layer prediction step includes: Based on the current frame image features, predict the altitude probability of each feature location on a predetermined number of candidate ground altitude layers; wherein, before inference in each frame, generate the physical altitude center corresponding to the predetermined number of candidate ground altitude layers in the current frame according to the current frame flight altitude and preset altitude distribution parameters.

4. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 1, characterized in that, The attitude compensation 3D projection steps include: Establish a local virtual coordinate system for the current frame. The origin of the local virtual coordinate system is the optical center of the UAV camera in the current frame, the vertical axis points to the ground, and the horizontal axis lies in the horizontal plane of the ground. Based on the downsampling ratio, the feature positions in the current frame image features are mapped to the corresponding image coordinates, and the image coordinates are back-projected based on the camera intrinsic parameter matrix to obtain the corresponding pixel back-projection ray; The direction of the pixel back-projection ray in the local virtual coordinate system of the current frame is corrected based on the IMU attitude information; For any candidate ground altitude layer, the local three-dimensional coordinates of the feature location on the candidate ground altitude layer are determined based on the vertical distance between the current frame flight altitude and the physical altitude center of the candidate ground altitude layer, as well as the vertical component of the attitude-compensated back-projection ray.

5. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 3, characterized in that, The dynamic height layer prediction step further includes: The current frame image features are input into the height prediction branch, and the original response scores of each feature position corresponding to a predetermined number of candidate ground height layers are output through the convolutional layer. The scores are then normalized along the height channel dimension to obtain the height probability. The current frame image features are input into the context semantic feature branch to obtain the context semantic features; The height probability is fused with the contextual semantic features position by position to obtain a fused feature that simultaneously contains semantic information and candidate height response confidence.

6. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 3, characterized in that, Also includes: Within a preset ground altitude coverage area, the physical altitude centers of a predetermined number of candidate ground altitude layers are determined according to a power-law mapping; wherein, the exponent parameter of the power-law mapping is determined based on the flight altitude of the current frame, so that the candidate ground altitude layers are more densely distributed in the low-altitude range when the UAV is flying at low altitude, and the distribution of the candidate ground altitude layers tends to be flat when the UAV is flying at high altitude.

7. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 4, characterized in that, The attitude compensation 3D projection steps also include: A compensation rotation matrix is ​​generated based on the pre-calibrated camera mounting rotation matrix and the current frame attitude increment obtained from the IMU attitude information; The pixel back-projection ray is mapped from the camera coordinate system to the local virtual coordinate system of the current frame using the compensation rotation matrix, so that roll, pitch or yaw changes are reflected in the spatial direction of the pixel back-projection ray.

8. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 1, characterized in that, The steps for outputting the bird's-eye view feature detection include: In the local virtual coordinate system of the current frame, the three-dimensional space is divided into multiple voxel lattices along three coordinate directions; The point features of three-dimensional feature points falling within the same voxel grid are averaged to obtain voxel features; if there are no three-dimensional feature points in the voxel grid, the voxel features corresponding to that voxel grid are set to zero. The effective voxel features at the same horizontal position are aggregated along the vertical direction and normalized according to the number of effective voxels to generate the bird's-eye view feature map.

9. The UAV 3D target perception method based on dynamic parameter adaptation according to claim 1, characterized in that, The bird's-eye view feature detection output step also includes: Convolutional feature aggregation is performed on the bird's-eye view feature map to obtain detection features; The confidence branch outputs the probability that a target exists at the location of the bird's-eye view grid. The target center offset, target geometry, and target yaw angle are output through the regression branch. Based on the target center offset and the reference coordinates of the corresponding bird's-eye view grid position or candidate position, the target center coordinates are decoded, and combined with the target geometry and target yaw angle, the local 3D target bounding box of the current frame is obtained.

Images