Unmanned aerial vehicle air pose detection method, equipment and medium

Abstract

The embodiment of the invention discloses an unmanned aerial vehicle air pose detection method and device and a medium, belongs to the technical field of unmanned aerial vehicle flight control, and solves the problems that when network communication cannot be established, an unmanned aerial vehicle pose estimation result is large in error and low in accuracy through a pilot judgment mode. Based on the 3D point cloud sample data and the 2D image sample data of the target unmanned aerial vehicle, training a preset neural network model to obtain an unmanned aerial vehicle 2D key point prediction model; inputting a to-be-detected image with a target unmanned aerial vehicle into the unmanned aerial vehicle 2D key point prediction model to obtain a D-dimensional vector for mapping 3D point cloud data of the target unmanned aerial vehicle into the 2D image; adjusting the D-dimensional vector based on the target category probability and the confidence value; and performing offset comparison on the 3D point cloud data of the target unmanned aerial vehicle and the adjusted D-dimensional vector, and obtaining an air pose corresponding to the target unmanned aerial vehicle based on a comparison result.

Description

Technical Field

[0001] The present application relates to the field of UAV flight control technology, and in particular to a method, device and medium for detecting the aerial posture of a UAV. Background Art

[0002] In the UAV flight control system, achieving accurate recognition and pose estimation of other UAV objects is a key factor in promoting the development of advanced application scenarios such as UAV collaborative operations and autonomous flight.

[0003] However, in practical applications, information exchange between drones often relies on network communication or preset collaborative modes. Through network communication, drones can share their own key information such as position, speed, and attitude in real time, thereby accurately judging the position and attitude of other drone targets.

[0004] However, in some special scenarios, such as areas with severe electromagnetic interference, remote areas without network coverage, or sudden failures that cause communication interruptions, the communication links between drones may be affected or even interrupted.

[0005] When network communication cannot be established, the traditional approach often relies on the pilot's experience and judgment to estimate the target drone's position and posture. However, this method of relying on the pilot's judgment is affected by many factors such as the pilot's own experience, concentration, and visual perception ability, resulting in large errors in the drone's position and posture estimation results and low accuracy. Summary of the invention

[0006] The embodiments of the present application provide a method, device and medium for detecting the aerial posture of a drone, which are used to solve the following technical problems: when network communication cannot be established, the judgment made by the pilot is affected by multiple factors such as his own experience, concentration, visual perception ability, etc., resulting in large errors in the estimation results of the drone posture and low accuracy.

[0007] The present application embodiment adopts the following technical solutions:

[0008] The embodiment of the present application provides a method for detecting the aerial posture of a drone. The method includes: training a preset neural network model based on the 3D point cloud sample data and 2D image sample data of the target drone to obtain a 2D key point prediction model of the drone; inputting the image to be tested with the target drone into the 2D key point prediction model of the drone to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image; determining the target category probability corresponding to the D-dimensional vector, and determining the confidence value corresponding to the D-dimensional vector based on a preset confidence function; adjusting the D-dimensional vector based on the target category probability and the confidence value; performing offset comparison between the 3D point cloud data of the target drone and the adjusted D-dimensional vector, and obtaining the aerial posture corresponding to the target drone based on the comparison result.

[0009] The embodiment of the present application maps the 3D point cloud data of the target drone to a 2D image through a drone 2D key point prediction model to obtain a D-dimensional vector, so that subsequent processing and analysis can be performed on a 2D plane, reducing the complexity and cost of calculations. Secondly, by determining the target category probability and confidence value corresponding to the D-dimensional vector, the D-dimensional vector is further adjusted and optimized to improve the accuracy and reliability of the model's prediction of drone key points and reduce misjudgments and missed judgments. The 3D point cloud data of the target drone is compared with the adjusted D-dimensional vector for offset, which enables the prediction of the target's motion state when normal communication cannot be established between drones or the target drone does not belong to an aircraft in the system, and can ensure the accuracy of the drone posture prediction.

[0010] In one implementation of the present application, a preset neural network model is trained based on the 3D point cloud sample data and 2D image sample data of the target UAV to obtain a 2D key point prediction model of the UAV, specifically including: determining the maximum and minimum values ​​of the target UAV in the three-dimensional coordinate system based on the 3D point cloud sample data and 2D image sample data of the target UAV; determining the 3D bounding box of the target UAV based on the maximum and minimum values; mapping the corner points of the 3D bounding box to coordinates in the 2D image as label data based on the rotation matrix and translation; and training the preset neural network model based on the label data to obtain a 2D key point prediction model of the UAV.

[0011] In one implementation of the present application, an image to be tested with a target drone is input into a drone 2D key point prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image. Specifically, the image to be tested with the target drone is input into the drone 2D key point prediction model, and through the 2D key point prediction model, a fully convolutional architecture is used to downsample the image to be tested and extract features; the image to be tested is divided into multiple grids, and the output of the 2D key point prediction model is stored in a tensor corresponding to the multiple grids; based on the positional relationship between the center point coordinates corresponding to the target drone and the grid preset point coordinates, the D-dimensional vector corresponding to the target drone is determined.

[0012] In one implementation of the present application, the D-dimensional vector corresponding to the target drone includes: the target category corresponding to the target drone; the width value of the target drone mapped to the 2D image; the height value of the target drone mapped to the 2D image; the corner points of the 3D bounding box corresponding to the target drone are mapped to the corner point coordinates in the 2D image; and the center point coordinates of the target drone mapped to the 2D image.

[0013] In one implementation of the present application, the target category probability corresponding to the D-dimensional vector is determined, and the confidence value corresponding to the D-dimensional vector is determined based on a preset confidence function, specifically including: determining the target category probability of each grid based on the target category vector in the D-dimensional vector; and determining the distance difference based on the two-dimensional Euclidean distance corresponding to the predicted 2D point and the real point in the D-dimensional vector; when the distance difference is less than a preset distance difference threshold, determining the confidence corresponding to the target area based on a preset confidence function; when the distance difference is not less than the preset distance difference, determining the confidence corresponding to the target-free area based on a preset confidence function.

[0014] In one implementation of the present application, after adjusting the D-dimensional vector based on the target category probability and the confidence value, the method further includes: performing sigmoid activation mapping processing on the center point coordinates to limit the offset corresponding to the center point coordinates to between 0 and 1.

[0015] In one implementation of the present application, after obtaining the UAV 2D key point prediction model, the method also includes: determining the coordinate loss and confidence loss through the mean square error loss function; and determining the classification loss through the cross entropy loss function; based on the coordinate loss, confidence loss and classification loss, optimizing the UAV 2D key point prediction model.

[0016] In one implementation of the present application, the drone 2D key point prediction model is optimized based on coordinate loss, confidence loss and classification loss, specifically including: based on the function:

[0017] L=λ ptL pt +λ conf L conf +λ id L id ;

[0018] The UAV 2D key point prediction model is optimized; among them, L pt is the coordinate loss; L conf Confidence loss; L id is the classification loss; pt is the weight coefficient corresponding to the coordinate loss; id is the weight coefficient corresponding to the classification loss; conf is the weight coefficient corresponding to the confidence loss; in the grid without target, the weight coefficient corresponding to the confidence loss is set to 0.1; in the grid with target, the weight coefficient corresponding to the confidence loss is set to 5.0.

[0019] An embodiment of the present application provides an aerial posture detection device for a drone, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of a target drone to obtain a 2D key point prediction model for the drone; input a test image with a target drone into the 2D key point prediction model for the drone to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image; determine a target category probability corresponding to the D-dimensional vector, and determine a confidence value corresponding to the D-dimensional vector based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value; perform an offset comparison between the 3D point cloud data of the target drone and the adjusted D-dimensional vector, and obtain the aerial posture corresponding to the target drone based on the comparison result.

[0020] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, and the computer executable instructions are configured to: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of a target drone to obtain a 2D key point prediction model for the drone; input a test image with the target drone into the 2D key point prediction model for the drone to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image; determine a target category probability corresponding to the D-dimensional vector, and determine a confidence value corresponding to the D-dimensional vector based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value; perform an offset comparison between the 3D point cloud data of the target drone and the adjusted D-dimensional vector, and obtain the corresponding aerial posture of the target drone based on the comparison result.

[0021] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: The embodiments of the present application use the UAV 2D key point prediction model to map the 3D point cloud data of the target UAV to a 2D image to obtain a D-dimensional vector, so that subsequent processing and analysis can be performed on a 2D plane, reducing the complexity and cost of calculations. Secondly, by determining the target category probability and confidence value corresponding to the D-dimensional vector, the D-dimensional vector is further adjusted and optimized to improve the accuracy and reliability of the model's prediction of the key points of the UAV, and reduce misjudgments and missed judgments. The 3D point cloud data of the target UAV is compared with the adjusted D-dimensional vector for offset, which enables the prediction of the target's motion state when normal communication cannot be established between UAVs or the target UAV does not belong to the aircraft in the system, and can ensure the accuracy of the UAV posture prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0023] Figure 1 A flowchart of a method for detecting the aerial posture of a drone provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the structure of a drone aerial posture detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application provide a method, device and medium for detecting the aerial posture of a drone.

[0026] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0027] The technical solution proposed in the embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0028] Figure 1 A flowchart of a method for detecting the aerial posture of a drone provided in an embodiment of the present application is shown in FIG. Figure 1As shown, the method for detecting the aerial posture of a UAV includes the following steps:

[0029] S101. Based on the 3D point cloud sample data and 2D image sample data of the target UAV, a preset neural network model is trained to obtain a UAV 2D key point prediction model.

[0030] In one embodiment of the present application, the maximum and minimum values ​​of the target drone in the three-dimensional coordinate system are determined based on the 3D point cloud sample data and the 2D image sample data of the target drone. Based on the maximum and minimum values, the 3D bounding box of the target drone is determined. Based on the rotation matrix and the translation amount, the corner points of the 3D bounding box are mapped to the coordinates in the 2D image as label data. Based on the label data, the preset neural network model is trained to obtain the drone 2D key point prediction model.

[0031] Specifically, the embodiment of the present application requires 3D point cloud data and 2D image data of the object to be detected, and calculates the maximum and minimum values ​​of the three dimensions of X, Y, and Z: max_x, min_x, max_y, min_y, max_z, min_z through the point cloud data, so as to determine the 3D bounding box corresponding to the target drone, and map the corner points of the 3D bounding box to the coordinates in the 2D image in combination with the real rotation matrix and translation as the label data for model training. Model training is performed based on the obtained label data to obtain the 2D key point prediction model of the drone.

[0032] Furthermore, the images captured by the drone with the target drone are input for data augmentation. Specifically, the data input to the model is normalized, and images of different resolutions are used as input at different stages of model training to ensure that the algorithm is robust to objects of different sizes.

[0033] The embodiment of the present application uses the 3D point cloud data captured by the drone to calculate the maximum and minimum values ​​of the three dimensions of X, Y, and Z to determine the 3D bounding box of the target. Combined with the real rotation matrix and translation, the corner points of the 3D bounding box are projected into the 2D image space and used as label data for model training. The input 2D image is normalized in the preprocessing stage, and the robustness of the model to different scenes is improved through data enhancement. In addition, the model accepts image inputs of different resolutions at different stages to ensure that the algorithm has efficient adaptability to various target sizes.

[0034] S102, inputting the image to be tested with the target drone into the drone 2D key point prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image.

[0035] In one embodiment of the present application, the image to be tested with the target drone is input into the drone 2D key point prediction model, and the image to be tested is downsampled and feature extracted using the full convolution architecture through the 2D key point prediction model. The image to be tested is divided into multiple grids, and the output of the 2D key point prediction model is stored in a tensor corresponding to the multiple grids. Based on the positional relationship between the center point coordinates corresponding to the target drone and the grid preset point coordinates, the D-dimensional vector corresponding to the target drone is determined.

[0036] Specifically, the drone 2D key point prediction model in the embodiment of the present application takes a single full-color image as input and uses a fully convolutional architecture to downsample the input image for feature extraction. During the feature extraction process, the image is divided into a 13×13 grid. Therefore, the output of the model is stored in a 13×13×D tensor, and the network determines in which grid the target output is located by judging the positional relationship between the coordinates of the center point of the drone object and the coordinates of the upper left corner of the grid.

[0037] In one embodiment of the present application, the D-dimensional vector corresponding to the target drone includes: the target category corresponding to the target drone, the width value of the target drone mapped to the 2D image, the height value of the target drone mapped to the 2D image, the corner points of the 3D bounding box corresponding to the target drone mapped to the corner point coordinates in the 2D image, and the center point coordinates of the target drone mapped to the 2D image.

[0038] Specifically, the D-dimensional vector output by the drone 2D key point prediction model in the embodiment of the present application is expressed as:

[0039] [cls,w,h,x1,y1,x2,y2,…,x8,y8,x0,y0];

[0040] Among them, cls is the target category, w is the width of the target in the 2D image, h is the height of the target in the 2D image, (x1, y1)~(x8, y8) are the coordinates of the 8 corner points, and (x0, y0) is the coordinate of the center point.

[0041] S103, determining the target category probability corresponding to the D-dimensional vector, and determining the confidence value corresponding to the D-dimensional vector based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value.

[0042] In one embodiment of the present application, the target category probability of each grid is determined based on the target category vector in the D-dimensional vector; and the distance difference is determined based on the two-dimensional Euclidean distance between the predicted 2D point and the real point in the D-dimensional vector. When the distance difference is less than the preset distance difference threshold, the confidence corresponding to the target area is determined based on the preset confidence function. When the distance difference is not less than the preset distance difference, the confidence corresponding to the target-free area is determined based on the preset confidence function.

[0043] Specifically, in order to speed up the training of the drone 2D key point prediction model and ensure that the model can accurately predict the target position, the model needs to determine the high confidence value of the target area and the low confidence value of the target-free area while predicting the corner point coordinates. The confidence function c(x) is calculated based on the distance D between the predicted 2D point and the real point. T (x) to calculate the confidence value:

[0044]

[0045] Among them, D T (x) is the two-dimensional Euclidean distance in the image space, d th A threshold is set to determine whether a confidence value is valid. The confidence function is applied to all points, the average is calculated and assigned to the confidence. In addition, the algorithm needs to predict the probability of the target class for each grid, which depends on the cell containing the target object.

[0046] In the embodiment of the present application, by determining the position of the target center point in the grid, the network can accurately predict the target area. At the same time, the network evaluates the confidence and uses an adaptive confidence function based on the 2D Euclidean distance D T (x) Calculate the confidence value to ensure high confidence target position prediction.

[0047] In one embodiment of the present application, sigmoid activation mapping is performed on the center point coordinates to limit the offset corresponding to the center point coordinates to between 0 and 1.

[0048] Specifically, in the embodiment of the present application, for the center point coordinates (x0, y0), the algorithm predicts the output and performs sigmoid activation mapping, and its offset will be limited to between 0 and 1. There is no restriction on corner points, because the drone target will not be completely located in the grid where the center point is located, so the corner points are allowed to fall outside the grid.

[0049] In one embodiment of the present application, the coordinate loss and the confidence loss are determined by the mean square error loss function, and the classification loss is determined by the cross entropy loss function. Based on the coordinate loss, the confidence loss and the classification loss, the drone 2D key point prediction model is optimized.

[0050] The embodiment of the present application minimizes the point coordinate loss L pt , confidence loss L conf and classification loss L id To optimize the model parameters, based on the function:

[0051] L=λ pt L pt +λ conf L conf +λ id L id ;

[0052] The 2D key point prediction model of the UAV is optimized. pt is the coordinate loss; L conf Confidence loss; L id is the classification loss; pt is the weight coefficient corresponding to the coordinate loss; id is the weight coefficient corresponding to the classification loss; conf is the weight coefficient corresponding to the confidence loss. In the grid without target, the weight coefficient corresponding to the confidence loss is set to 0.1, and in the grid with target, the weight coefficient corresponding to the confidence loss is set to 5.0.

[0053] S104, performing an offset comparison between the 3D point cloud data of the target UAV and the adjusted D-dimensional vector, and obtaining the corresponding aerial posture of the target UAV based on the comparison result.

[0054] In one embodiment of the present application, at runtime, the network gives a 2D projection of the drone's centroid and the corners of the drone's 3D bounding box as well as the drone category. The drone's 6D attitude is estimated from the correspondence between 2D and 3D points using the Perspective-n-Point (PnP) attitude estimation method. The present application embodiment not only improves the real-time and accuracy of target detection, but also enables robust attitude estimation of drones in complex environments, and is applicable to a variety of drone visual detection tasks.

[0055] In the 6D posture detection of UAV targets, the actual data (including but not limited to the target's shape, material, motion state, etc.) is complex and changeable, and difficult to quantify accurately. The technical level and experience of operators also vary. Therefore, accurate calculation of the target posture is often difficult. Usually, rough estimates are used or some influencing factors are directly ignored, resulting in inaccurate detection results, which in turn affects the overall positioning and tracking effects.

[0056] The embodiment of the present application can accurately calculate the changes in the 6D pose (including position and attitude) of the drone target through the analysis and comparison of a large amount of visual data and the derivation of relevant mathematical formulas. This can significantly improve the accuracy and robustness of target detection, so that more reliable pose data can be obtained during the drone positioning and tracking process, effectively reducing the error impact in traditional methods and improving the overall performance of the system.

[0057] Figure 2 This is a schematic diagram of the structure of a drone aerial posture detection device provided in an embodiment of the present application. Figure 2 As shown, a drone aerial posture detection device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of the target drone to obtain a drone 2D key point prediction model; input the image to be tested with the target drone into the drone 2D key point prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image; determine the target category probability corresponding to the D-dimensional vector, and determine the confidence value corresponding to the D-dimensional vector based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value; compare the 3D point cloud data of the target drone with the adjusted D-dimensional vector for offset, and obtain the aerial posture corresponding to the target drone based on the comparison result.

[0058] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, and the computer executable instructions are configured to: train a preset neural network model based on 3D point cloud sample data and 2D image sample data of a target drone to obtain a 2D key point prediction model for the drone; input a test image with the target drone into the 2D key point prediction model for the drone to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image; determine a target category probability corresponding to the D-dimensional vector, and determine a confidence value corresponding to the D-dimensional vector based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value; perform an offset comparison between the 3D point cloud data of the target drone and the adjusted D-dimensional vector, and obtain the corresponding aerial posture of the target drone based on the comparison result.

[0059] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0060] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0061] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0062] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0063] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0064] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0065] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0066] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A method for detecting the aerial posture of a drone, characterized in that: The method comprises: Based on the 3D point cloud sample data and 2D image sample data of the target UAV, the preset neural network model is trained to obtain the UAV 2D key point prediction model; Inputting the image to be tested with the target UAV into the UAV 2D key point prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target UAV to the 2D image; Determine a target category probability corresponding to the D-dimensional vector, and determine a confidence value corresponding to the D-dimensional vector based on a preset confidence function, so as to adjust the D-dimensional vector based on the target category probability and the confidence value; The 3D point cloud data of the target UAV is compared with the adjusted D-dimensional vector by offset, and the corresponding aerial posture of the target UAV is obtained based on the comparison result.

2. The method for detecting the aerial posture of a drone according to claim 1, characterized in that: The preset neural network model is trained based on the 3D point cloud sample data and 2D image sample data of the target UAV to obtain the UAV 2D key point prediction model, which specifically includes: Based on the 3D point cloud sample data and the 2D image sample data of the target UAV, determining the maximum value and the minimum value of the target UAV in the three-dimensional coordinate system; Based on the maximum value and the minimum value, determining a 3D bounding box of the target drone; Based on the rotation matrix and translation, mapping the corner points of the 3D bounding box to coordinates in the 2D image as label data; The preset neural network model is trained based on the label data to obtain the UAV 2D key point prediction model.

3. The method for detecting the aerial posture of a drone according to claim 1, characterized in that: The step of inputting the image to be tested with the target drone into the drone 2D key point prediction model to obtain a D-dimensional vector that maps the 3D point cloud data of the target drone to the 2D image specifically includes: Inputting the image to be tested with the target drone into the drone 2D key point prediction model, and performing down-sampling feature extraction on the image to be tested using a fully convolutional architecture through the 2D key point prediction model; Dividing the image to be tested into a plurality of grids, and storing the output of the 2D key point prediction model into tensors corresponding to the plurality of grids; Based on the positional relationship between the center point coordinates corresponding to the target UAV and the grid preset point coordinates, a D-dimensional vector corresponding to the target UAV is determined.

4. The method for detecting the aerial posture of a drone according to claim 1, characterized in that: The D-dimensional vector corresponding to the target drone includes: The target category corresponding to the target UAV; The target drone is mapped to a width value in the 2D image; The target drone is mapped to a height value in the 2D image; The corner points of the 3D bounding box corresponding to the target drone are mapped to the corner point coordinates in the 2D image; The target drone is mapped to the center point coordinates in the 2D image.

5. The method for detecting the aerial posture of a drone according to claim 1, characterized in that: The determining of the target category probability corresponding to the D-dimensional vector and the determining of the confidence value corresponding to the D-dimensional vector based on a preset confidence function specifically include: Determining the target category probability of each grid based on the target category vector in the D-dimensional vector; and Determining a distance difference based on the corresponding two-dimensional Euclidean distance between the predicted 2D point and the real point in the D-dimensional vector; When the distance difference is less than a preset distance difference threshold, determining the confidence corresponding to the area where the target is located based on the preset confidence function; When the distance difference is not less than the preset distance difference, the confidence corresponding to the target-free area is determined based on the preset confidence function.

6. The method for detecting the aerial posture of a drone according to claim 4, characterized in that: After adjusting the D-dimensional vector based on the target category probability and the confidence value, the method further includes: The center point coordinates are subjected to sigmoid activation mapping processing to limit the offset corresponding to the center point coordinates to between 0 and 1.

7. The method for detecting the aerial posture of a drone according to claim 1, characterized in that: After obtaining the UAV 2D key point prediction model, the method further includes: Determine the coordinate loss and confidence loss through the mean square error loss function; and The classification loss is determined by the cross entropy loss function; Based on the coordinate loss, the confidence loss and the classification loss, the UAV 2D key point prediction model is optimized.

8. The method for detecting the aerial posture of a UAV according to claim 7, characterized in that: The optimizing the drone 2D key point prediction model based on the coordinate loss, the confidence loss and the classification loss specifically includes: Function-based: L=λ pt L pt +λ conf L conf +λ id L id ; Optimizing the UAV 2D key point prediction model; Among them, L pt is the coordinate loss; L conf The confidence loss; L id is the classification loss; pt is the weight coefficient corresponding to the coordinate loss; id is the weight coefficient corresponding to the classification loss; conf is the weight coefficient corresponding to the confidence loss; In the target-free grid, the weight coefficient corresponding to the confidence loss is set to 0.1; In the target grid, the weight coefficient corresponding to the confidence loss is set to 5.

0.

9. A drone aerial posture detection device, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions can execute the method according to any one of claims 1 to 8.

Images