An unmanned aerial vehicle aerial pose detection method, device and medium
By training a 2D keypoint prediction model for UAVs, 3D point cloud data is mapped to D-dimensional vectors in 2D images. Combined with target category probability and confidence value adjustment, the pose estimation error problem when network communication is interrupted in the UAV flight control system is solved, achieving pose prediction with high accuracy and low complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG SYNTHESIS ELECTRONICS TECH
- Filing Date
- 2024-11-12
- Publication Date
- 2026-05-15
AI Technical Summary
In UAV flight control systems, when network communication cannot be established, the pilot's judgment is affected by various factors such as their own experience, level of concentration, and visual perception ability, resulting in a large error in the UAV attitude estimation results and a low accuracy rate.
A 2D keypoint prediction model for UAVs is trained using 3D point cloud sample data and 2D image sample data based on the target UAV. By mapping the 3D point cloud data to a D-dimensional vector in the 2D image and adjusting it in combination with the target category probability and confidence value, the accurate prediction of the UAV pose can be achieved.
It improves the accuracy and reliability of UAV pose prediction, reduces computational complexity and cost, reduces misjudgments and omissions, and ensures accurate prediction of target motion state even in the absence of communication or when the target UAV is not part of the system's aircraft.
Smart Images
Figure CN119559254B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of unmanned aerial vehicle (UAV) flight control technology, and in particular to a method, device and medium for detecting the aerial posture of a UAV. Background Technology
[0002] In UAV flight control systems, achieving accurate identification and pose estimation of other UAV objects is a key factor in promoting the development of advanced application scenarios such as collaborative operation and autonomous flight of UAVs.
[0003] However, in practical applications, information exchange between drones often relies on network communication or preset collaborative modes. Through network communication, drones can share their own key information such as position, speed, and attitude in real time, thereby enabling accurate judgment of the position and attitude of other drone targets.
[0004] However, in some special scenarios, such as areas with severe electromagnetic interference, remote areas without network coverage, or sudden malfunctions that cause communication interruptions, the communication links between drones may be affected or even interrupted.
[0005] When network communication cannot be established, the traditional approach often relies on the pilot's experience and judgment to estimate the target drone's attitude. However, this method of relying on the pilot's judgment is affected by various factors such as their own experience, level of concentration, and visual perception ability, resulting in significant errors in the drone's attitude estimation and low accuracy. Summary of the Invention
[0006] This application provides a method, device, and medium for detecting the aerial pose of a drone, which addresses the following technical problem: when network communication cannot be established, the method of judging by the pilot is affected by various factors such as his own experience, level of concentration, and visual perception ability, resulting in a large error in the drone pose estimation result and a low accuracy.
[0007] The embodiments of this application adopt the following technical solutions:
[0008] This application provides a method for detecting the aerial pose of a UAV. The method includes: training a preset neural network model based on 3D point cloud sample data and 2D image sample data of the target UAV to obtain a 2D keypoint prediction model for the UAV; inputting a test image containing the target UAV into the 2D keypoint prediction model to obtain a D-dimensional vector mapping the 3D point cloud data of the target UAV to the 2D image; determining the target category probability corresponding to the D-dimensional vector, and determining the confidence value corresponding to the D-dimensional vector based on a preset confidence function; adjusting the D-dimensional vector based on the target category probability and the confidence value; comparing the offset between the 3D point cloud data of the target UAV and the adjusted D-dimensional vector, and obtaining the aerial pose of the target UAV based on the comparison result.
[0009] This application's embodiments map the 3D point cloud data of the target UAV onto a 2D image using a UAV 2D keypoint prediction model, obtaining a D-dimensional vector. This allows subsequent processing and analysis to be performed on the 2D plane, reducing computational complexity and cost. Secondly, by determining the target category probability and confidence value corresponding to the D-dimensional vector, the D-dimensional vector is further adjusted and optimized to improve the accuracy and reliability of the model's prediction of UAV keypoints, reducing false positives and false negatives. By comparing the offset between the target UAV's 3D point cloud data and the adjusted D-dimensional vector, the motion state of the target can be predicted even when normal communication between UAVs is impossible or the target UAV is not part of the system, while ensuring the accuracy of UAV pose prediction.
[0010] In one implementation of this application, a preset neural network model is trained based on 3D point cloud sample data and 2D image sample data of the target UAV to obtain a 2D keypoint prediction model for the UAV. Specifically, this includes: determining the maximum and minimum values of the target UAV in a three-dimensional coordinate system based on the 3D point cloud sample data and 2D image sample data of the target UAV; determining the 3D bounding box of the target UAV based on the maximum and minimum values; mapping the corner points of the 3D bounding box to coordinates in the 2D image based on rotation matrix and translation, as label data; and training the preset neural network model based on the label data to obtain the 2D keypoint prediction model for the UAV.
[0011] In one implementation of this application, the image to be tested containing the target UAV is input into the UAV 2D keypoint prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image. Specifically, this includes: inputting the image to be tested containing the target UAV into the UAV 2D keypoint prediction model; using the 2D keypoint prediction model, performing downsampling feature extraction on the image to be tested using a fully convolutional architecture; dividing the image to be tested into multiple grids and storing the output of the 2D keypoint prediction model into tensors corresponding to the multiple grids; and determining the D-dimensional vector corresponding to the target UAV based on the positional relationship between the center point coordinates of the target UAV and the preset point coordinates of the grid.
[0012] In one implementation of this application, the D-dimensional vector corresponding to the target drone includes: the target category corresponding to the target drone; the width value of the target drone mapped to the 2D image; the height value of the target drone mapped to the 2D image; the corner coordinates of the corner points of the 3D bounding box corresponding to the target drone mapped to the 2D image; and the center point coordinates of the target drone mapped to the 2D image.
[0013] In one implementation of this application, the target category probability corresponding to the D-dimensional vector is determined, and the confidence value corresponding to the D-dimensional vector is determined based on a preset confidence function. Specifically, this includes: determining the target category probability of each grid based on the target category vector in the D-dimensional vector; and determining the distance difference based on the two-dimensional Euclidean distance between the predicted 2D point and the real point in the D-dimensional vector; if the distance difference is less than a preset distance difference threshold, determining the confidence level corresponding to the target region based on the preset confidence function; and if the distance difference is not less than a preset distance difference, determining the confidence level corresponding to the targetless region based on the preset confidence function.
[0014] In one implementation of this application, after adjusting the D-dimensional vector based on the target category probability and confidence value, the method further includes: performing sigmoid activation mapping on the center point coordinates to limit the offset corresponding to the center point coordinates to between 0 and 1.
[0015] In one implementation of this application, after obtaining the UAV 2D keypoint prediction model, the method further includes: determining the coordinate loss and confidence loss through the mean squared error loss function; determining the classification loss through the cross-entropy loss function; and optimizing the UAV 2D keypoint prediction model based on the coordinate loss, confidence loss, and classification loss.
[0016] In one implementation of this application, the 2D keypoint prediction model for UAVs is optimized based on coordinate loss, confidence loss, and classification loss, specifically including: based on the function:
[0017] L=λ ptL pt +λ conf L conf +λ id L id ;
[0018] The 2D keypoint prediction model for UAVs is optimized; among which, L pt For coordinate loss; L conf Confidence loss; L id For classification loss; λ pt λ represents the weighting coefficients corresponding to the coordinate loss. id λ represents the weighting coefficients corresponding to the classification loss. conf The weighting coefficients for the confidence loss are: 0.1 for the confidence loss without a target grid and 5.0 for the confidence loss with a target grid.
[0019] This application provides an aerial pose detection device for unmanned aerial vehicles (UAVs), comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of a target UAV to obtain a 2D keypoint prediction model for the UAV; input a test image containing the target UAV into the 2D keypoint prediction model for the UAV to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image; determine the target category probability corresponding to the D-dimensional vector, and determine the confidence value corresponding to the D-dimensional vector based on a preset confidence function, thereby adjusting the D-dimensional vector based on the target category probability and the confidence value; compare the offset of the 3D point cloud data of the target UAV with the adjusted D-dimensional vector, and obtain the aerial pose of the target UAV based on the comparison result.
[0020] This application provides a non-volatile computer storage medium storing computer-executable instructions. The computer-executable instructions are configured to: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of a target UAV to obtain a 2D keypoint prediction model for the UAV; input the image to be tested containing the target UAV into the 2D keypoint prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image; determine the target category probability corresponding to the D-dimensional vector and determine the confidence value corresponding to the D-dimensional vector based on a preset confidence function, and adjust the D-dimensional vector based on the target category probability and the confidence value; compare the offset of the 3D point cloud data of the target UAV with the adjusted D-dimensional vector, and obtain the aerial pose of the target UAV based on the comparison result.
[0021] The above-mentioned technical solutions adopted in this application embodiment can achieve the following beneficial effects: This application embodiment maps the 3D point cloud data of the target UAV onto a 2D image using a UAV 2D keypoint prediction model, obtaining a D-dimensional vector. This allows subsequent processing and analysis to be performed on the 2D plane, reducing computational complexity and cost. Secondly, by determining the target category probability and confidence value corresponding to the D-dimensional vector, the D-dimensional vector is further adjusted and optimized, improving the accuracy and reliability of the model's prediction of UAV keypoints and reducing false positives and false negatives. By comparing the offset of the target UAV's 3D point cloud data with the adjusted D-dimensional vector, the motion state of the target can be predicted even when normal communication between UAVs is impossible or the target UAV is not part of the system, while ensuring the accuracy of UAV pose prediction. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0023] Figure 1 A flowchart of an aerial pose detection method for unmanned aerial vehicles provided in this application embodiment;
[0024] Figure 2 This is a structural schematic diagram of an aerial pose detection device for unmanned aerial vehicles (UAVs) provided in an embodiment of this application. Detailed Implementation
[0025] This application provides a method, device, and medium for detecting the aerial posture of an unmanned aerial vehicle (UAV).
[0026] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.
[0027] The technical solutions proposed in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0028] Figure 1 A flowchart of an aerial pose detection method for a drone provided in this application embodiment is shown below. Figure 1As shown, the UAV aerial pose detection method includes the following steps:
[0029] S101. Based on the 3D point cloud sample data and 2D image sample data of the target UAV, a preset neural network model is trained to obtain a 2D key point prediction model for the UAV.
[0030] In one embodiment of this application, the maximum and minimum values of the target UAV in a three-dimensional coordinate system are determined based on 3D point cloud sample data and 2D image sample data. Based on these maximum and minimum values, a 3D bounding box of the target UAV is determined. Based on rotation matrices and translation amounts, the corner points of the 3D bounding box are mapped to coordinates in the 2D image, serving as label data. A preset neural network model is trained based on the label data to obtain a 2D keypoint prediction model for the UAV.
[0031] Specifically, this embodiment requires 3D point cloud data and 2D image data of the object to be detected. The maximum and minimum values of the X, Y, and Z dimensions are calculated from the point cloud data: max_x, min_x, max_y, min_y, max_z, and min_z, respectively, to determine the 3D bounding box corresponding to the target UAV. The corner points of the 3D bounding box are then mapped to the coordinates in the 2D image using the true rotation matrix and translation, serving as the label data for model training. Model training is then performed based on the obtained label data to obtain the 2D keypoint prediction model for the UAV.
[0032] Furthermore, images captured by the drone containing the target drone are used for data augmentation. Specifically, the data input to the model is normalized, and images of different resolutions are used as input at different stages of model training to ensure the algorithm's robustness to objects of different sizes.
[0033] This application's embodiments utilize 3D point cloud data captured by a drone to calculate the maximum and minimum values in the X, Y, and Z dimensions, thus determining the target's 3D bounding box. Combining the true rotation matrix and translation, the corner points of the 3D bounding box are projected onto a 2D image space and used as label data for model training. The input 2D image is normalized during preprocessing, and data augmentation is used to improve the model's robustness to different scenes. Furthermore, the model accepts image inputs of different resolutions at different stages, ensuring the algorithm's efficient adaptability to various target sizes.
[0034] S102. Input the image to be tested containing the target UAV into the UAV 2D key point prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image.
[0035] In one embodiment of this application, a test image containing the target drone is input into a 2D keypoint prediction model for the drone. The 2D keypoint prediction model uses a fully convolutional architecture to extract downsampled features from the test image. The test image is divided into multiple grids, and the output of the 2D keypoint prediction model is stored in tensors corresponding to these grids. Based on the positional relationship between the center point coordinates of the target drone and the preset point coordinates of the grids, the D-dimensional vector corresponding to the target drone is determined.
[0036] Specifically, the UAV 2D keypoint prediction model in this embodiment uses a single full-color image as input and employs a fully convolutional architecture to perform downsampling feature extraction on the input image. During feature extraction, the image is divided into a 13×13 grid. Therefore, the model's output is stored in a 13×13×D tensor. The network determines which grid the target output is located in by judging the positional relationship between the coordinates of the center point of the UAV object and the coordinates of the top-left corner of the grid.
[0037] In one embodiment of this application, the D-dimensional vector corresponding to the target drone includes: the target category corresponding to the target drone, the width value of the target drone mapped to the 2D image, the height value of the target drone mapped to the 2D image, the corner coordinates of the corner points of the 3D bounding box corresponding to the target drone mapped to the 2D image, and the center point coordinates of the target drone mapped to the 2D image.
[0038] Specifically, the D-dimensional vector output by the UAV 2D keypoint prediction model in this embodiment is represented as follows:
[0039] [cls,w,h,x1,y1,x2,y2,…,x8,y8,x0,y0];
[0040] Where cls is the target category, w is the width of the target in the 2D image, h is the height of the target in the 2D image, (x1, y1) to (x8, y8) are the coordinates of the 8 corner points, and (x0, y0) is the coordinate of the center point.
[0041] S103. Determine the target category probability corresponding to the D-dimensional vector, and determine the confidence value corresponding to the D-dimensional vector based on the preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value.
[0042] In one embodiment of this application, the target class probability of each grid is determined based on the target class vector in the D-dimensional vector; and the distance difference is determined based on the two-dimensional Euclidean distance between the predicted 2D point and the ground truth point in the D-dimensional vector. If the distance difference is less than a preset distance difference threshold, the confidence level corresponding to the target region is determined based on a preset confidence function. If the distance difference is not less than the preset distance difference, the confidence level corresponding to the targetless region is determined based on a preset confidence function.
[0043] Specifically, in order to accelerate the training of the UAV 2D keypoint prediction model and ensure that the model can accurately predict the target position, the model needs to determine the high confidence value of the target area and the low confidence value of the targetless area while predicting the corner coordinates. The confidence function c(x) is based on the distance D between the predicted 2D point and the true point. T (x) is used to calculate the confidence value:
[0044]
[0045] Among them, D T (x) represents the two-dimensional Euclidean distance in the image space, d th A threshold is used to determine whether a confidence value is valid. A confidence function is applied to all points, the average is calculated, and this average is assigned as the confidence value. Additionally, the algorithm needs to predict the target class probability for each grid cell, depending on the cell containing the target object.
[0046] This embodiment of the application determines the target's center point's location within a grid, enabling the network to accurately predict the target's region. Simultaneously, the network evaluates confidence using an adaptive confidence function based on the 2D Euclidean distance D. T (x) Calculate the confidence value to ensure a high confidence level in the target location prediction.
[0047] In one embodiment of this application, the center point coordinates are subjected to sigmoid activation mapping to limit the offset corresponding to the center point coordinates to between 0 and 1.
[0048] Specifically, in this embodiment of the application, for the center point coordinates (x0, y0), the algorithm prediction output is subjected to sigmoid activation mapping, and its offset will be limited to between 0 and 1. There is no restriction for corner points, because the UAV target will not be completely located within the grid where the center point is located, so corner points are allowed to fall outside the grid.
[0049] In one embodiment of this application, the coordinate loss and confidence loss are determined using the mean squared error loss function, and the classification loss is determined using the cross-entropy loss function. Based on the coordinate loss, confidence loss, and classification loss, the 2D keypoint prediction model for UAVs is optimized.
[0050] This application embodiment minimizes the point coordinate loss L. pt Confidence loss L conf And classification loss L id To optimize model parameters, based on the function:
[0051] L=λ pt L pt +λ conf L conf +λ id L id ;
[0052] The 2D keypoint prediction model for UAVs is optimized. Among them, L... pt For coordinate loss; L conf Confidence loss; L id For classification loss; λ pt λ represents the weighting coefficients corresponding to the coordinate loss. id λ represents the weighting coefficients corresponding to the classification loss. conf This represents the weighting coefficient corresponding to the confidence loss. In the absence of a target grid, the weighting coefficient for the confidence loss is set to 0.1; in the presence of a target grid, the weighting coefficient is set to 5.0.
[0053] S104. Compare the 3D point cloud data of the target UAV with the adjusted D-dimensional vector by offset, and obtain the aerial pose of the target UAV based on the comparison result.
[0054] In one embodiment of this application, during runtime, the network provides the 2D projection of the UAV's centroid, the angles of the UAV's 3D bounding box, and the UAV category. The 6D UAV pose is estimated from the correspondence between 2D and 3D points using the Perspective-n-Point (PnP) pose estimation method. This embodiment not only improves the real-time performance and accuracy of target detection but also enables robust pose estimation for UAVs in complex environments, making it suitable for various UAV visual inspection tasks.
[0055] In 6D pose detection of UAV targets, the actual data (including but not limited to the target's shape, material, motion state, etc.) is complex and variable, making it difficult to quantify accurately. The technical level and experience of operators also vary. Therefore, it is often difficult to accurately calculate the target pose. Usually, a rough estimate is used or some influencing factors are ignored, resulting in inaccurate detection results, which in turn affects the overall positioning and tracking effect.
[0056] This application's embodiments, through the analysis and comparison of a large amount of visual data and the derivation of relevant mathematical formulas, can accurately calculate the changes in the 6D pose (including position and attitude) of a UAV target. This can significantly improve the accuracy and robustness of target detection, enabling the acquisition of more reliable pose data during UAV localization and tracking, effectively reducing the impact of errors in traditional methods, and improving the overall system performance.
[0057] Figure 2 This is a structural schematic diagram of an aerial pose detection device for a drone provided in an embodiment of this application. Figure 2 As shown, an aerial pose detection device for unmanned aerial vehicles (UAVs) includes: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of the target UAV to obtain a 2D keypoint prediction model for the UAV; input a test image containing the target UAV into the 2D keypoint prediction model to obtain a D-dimensional vector mapping the 3D point cloud data of the target UAV to the 2D image; determine the target category probability corresponding to the D-dimensional vector and determine the confidence value corresponding to the D-dimensional vector based on a preset confidence function, adjusting the D-dimensional vector based on the target category probability and the confidence value; compare the offset of the 3D point cloud data of the target UAV with the adjusted D-dimensional vector, and obtain the aerial pose of the target UAV based on the comparison result.
[0058] This application provides a non-volatile computer storage medium storing computer-executable instructions. The computer-executable instructions are configured to: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of a target UAV to obtain a 2D keypoint prediction model for the UAV; input the image to be tested containing the target UAV into the 2D keypoint prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image; determine the target category probability corresponding to the D-dimensional vector and determine the confidence value corresponding to the D-dimensional vector based on a preset confidence function, and adjust the D-dimensional vector based on the target category probability and the confidence value; compare the offset of the 3D point cloud data of the target UAV with the adjusted D-dimensional vector, and obtain the aerial pose of the target UAV based on the comparison result.
[0059] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0062] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0063] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0064] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0065] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0066] The above descriptions are merely embodiments of this application and are not intended to limit the scope of this application. For those skilled in the art, various modifications and variations can be made to the embodiments of this application. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions in the embodiments of this application.
Claims
1. A method for detecting the aerial pose of an unmanned aerial vehicle (UAV), characterized in that, The method includes: Based on the 3D point cloud sample data and 2D image sample data of the target UAV, a preset neural network model is trained to obtain a 2D key point prediction model for the UAV. The image of the target UAV is input into the 2D key point prediction model of the UAV to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image. The target category probability corresponding to the D-dimensional vector is determined, and the confidence value corresponding to the D-dimensional vector is determined based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value; The 3D point cloud data of the target UAV is compared with the adjusted D-dimensional vector by offset, and the aerial pose of the target UAV is obtained based on the comparison result. The D-dimensional vector corresponding to the target UAV includes: The target category corresponding to the target drone; The width value of the target drone mapped into the 2D image; The altitude value of the target drone mapped onto the 2D image; The corner points of the 3D bounding box corresponding to the target UAV are mapped to the corner point coordinates in the 2D image; The target UAV is mapped to the center point coordinates in the 2D image.
2. The method for detecting the aerial pose of a UAV according to claim 1, characterized in that, The method involves training a pre-defined neural network model using 3D point cloud sample data and 2D image sample data of the target UAV to obtain a 2D keypoint prediction model for the UAV, specifically including: Based on the 3D point cloud sample data and 2D image sample data of the target UAV, the maximum and minimum values of the target UAV in the three-dimensional coordinate system are determined. Based on the maximum and minimum values, the 3D bounding box of the target UAV is determined; Based on the rotation matrix and translation, the corner points of the 3D bounding box are mapped to coordinates in the 2D image as label data; The preset neural network model is trained based on the labeled data to obtain the 2D key point prediction model of the UAV.
3. The method for detecting the aerial pose of a UAV according to claim 1, characterized in that, The step of inputting the image of the target UAV into the 2D key point prediction model of the UAV to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image specifically includes: The image of the target drone is input into the 2D keypoint prediction model of the drone. The 2D keypoint prediction model uses a fully convolutional architecture to extract downsampled features from the image of the target drone. The image to be tested is divided into multiple grids, and the output of the 2D key point prediction model is stored in tensors corresponding to the multiple grids. Based on the positional relationship between the center point coordinates of the target UAV and the grid preset point coordinates, the D-dimensional vector corresponding to the target UAV is determined.
4. The method for detecting the aerial pose of a UAV according to claim 1, characterized in that, The determination of the target category probability corresponding to the D-dimensional vector, and the determination of the confidence value corresponding to the D-dimensional vector based on a preset confidence function, specifically include: Based on the target category vector in the D-dimensional vector, the target category probability of each grid is determined; and Based on the two-dimensional Euclidean distance between the predicted 2D point and the real point in the D-dimensional vector, the distance difference is determined. If the distance difference is less than a preset distance difference threshold, the confidence level corresponding to the target's location is determined based on the preset confidence function. If the distance difference is not less than the preset distance difference, the confidence level corresponding to the targetless region is determined based on the preset confidence function.
5. The method for detecting the aerial pose of a UAV according to claim 1, characterized in that, After adjusting the D-dimensional vector based on the target category probability and the confidence value, the method further includes: The center point coordinates are subjected to sigmoid activation mapping to limit the offset corresponding to the center point coordinates to between 0 and 1.
6. The method for detecting the aerial pose of a UAV according to claim 1, characterized in that, After obtaining the 2D keypoint prediction model of the UAV, the method further includes: The coordinate loss and confidence loss are determined using the mean squared error loss function; and The classification loss is determined using the cross-entropy loss function; The 2D keypoint prediction model for the UAV is optimized based on the coordinate loss, the confidence loss, and the classification loss.
7. The method for detecting the aerial pose of a UAV according to claim 6, characterized in that, The optimization of the UAV 2D keypoint prediction model based on the coordinate loss, the confidence loss, and the classification loss specifically includes: Function-based: ； The 2D key point prediction model of the UAV is optimized; in, L pt The coordinate loss; L conf The confidence loss; L id The classification loss; λ pt The weighting coefficients corresponding to the coordinate loss; λ id The weight coefficients corresponding to the classification loss; λ conf The weighting coefficients corresponding to the confidence loss; In the absence of a target grid, the weight coefficient corresponding to the confidence loss is set to 0.1; In the case of a target grid, the weighting coefficient corresponding to the confidence loss is set to 5.
0.
8. An aerial pose detection device for unmanned aerial vehicles, characterized in that, The device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to perform the method described in any one of claims 1-7.
9. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of performing the method described in any one of claims 1-7.