A method and system for identifying a UAV based on a deep recognition algorithm

Abstract

The application discloses a kind of unmanned aerial vehicle identification method and system based on deep identification algorithm.Therein, the method includes: constructing calibration training set, inputting calibration training set into original segmentation model and original distance model, and combining "unmanned aerial vehicle" semantic description vector to carry out cross attention training, the segmentation text feature map and distance text feature map obtained respectively are fused to the feature map of the other collaborative training, to obtain target segmentation model and target distance model;The picture to be identified is inferred through target segmentation model and target distance model, and the double determination of pixel proportion and length proportion, to obtain unmanned aerial vehicle target.The application introduces semantic feature guided segmentation model and pixel-level distance prediction model, replaces real-time dependence on laser radar, significantly reduces the hardware cost of system, improves the identification precision and accuracy of long-distance and small target unmanned aerial vehicle, solves the problems of high cost, low precision and high false detection and missed detection rate of traditional method.

Description

Technical Field

[0001] This invention relates to the field of drone identification technology, and more specifically, to a drone identification method and system based on a depth recognition algorithm. Background Technology

[0002] With the popularization of drone technology, its applications in aerial photography, logistics, and other fields are becoming increasingly widespread, but this also brings privacy leaks and security risks. Existing drone identification technologies are mainly divided into two categories: 1) vision-based target detection or segmentation methods, which suffer from high false negative rates (such as missing distant targets) and high false positive rates (such as misidentification of birds); 2) 3D point cloud-based identification methods, which rely on expensive LiDAR equipment and require secondary visual verification, resulting in high costs. Therefore, a high-precision, low-cost drone identification solution is urgently needed. Summary of the Invention

[0003] This invention provides a drone identification method and system based on a deep recognition algorithm to solve the problems of high false negative rate, high false positive rate, and high cost in existing drone identification methods.

[0004] To achieve the above objectives, this invention provides a drone identification method based on a deep learning algorithm. The method includes: S1, acquiring a training set via a camera, manually calibrating all pixels in each image of the training set, the target category corresponding to each pixel, and the target distance corresponding to each pixel to obtain a calibration training set; S2, inputting the calibration training set into the backbone network of the original segmentation model to obtain a segmentation feature map; inputting the calibration training set into the backbone network of the original distance model to obtain a distance feature map; inputting the "drone" font into a language encoder to obtain a descriptive feature vector; inputting the descriptive feature vector into the original segmentation model and jointly training it with the segmentation feature map to obtain a segmented text feature map; inputting the descriptive feature vector into the original distance model and jointly training it with the distance feature map to obtain a distance text feature map; and integrating the distance text feature map into the segmented text feature map. The original segmentation model is trained multiple times to obtain a target segmentation model; the segmentation text feature map is incorporated into the distance text feature map, and the original distance model is trained multiple times to obtain a target distance model; S3, the image to be identified is input into the target segmentation model and the target distance model for identification to obtain a target category map and a first distance map; the target category map and the first distance map are matched to obtain a target category distance map; the pixel ratio of each target is calculated based on the target category distance map; it is sequentially determined whether the pixel ratio of each target is within a first preset range. If not, the current target is determined not to be a drone; if so, the length ratio of the current target is calculated based on the target category map and the target category distance map, and it is determined whether the length ratio of the current target is within a second preset range. If not, the current target is determined not to be a drone; if so, the current target is determined to be a drone.

[0005] Optionally, the target distance corresponding to each pixel is obtained by mapping the two-dimensional coordinates of each pixel in the image to the three-dimensional LiDAR coordinate system to obtain the three-dimensional coordinates of each pixel; the height of the three-dimensional coordinates of each pixel is taken as the target distance of each pixel.

[0006] Optionally, the step of integrating a distance text feature map into the segmented text feature map and training the original segmentation model for multiple rounds to obtain the target segmentation model; integrating a segmented text feature map into the distance text feature map and training the original distance model for multiple rounds to obtain the target distance model includes: integrating a distance text feature map into the segmented text feature map and performing cross-attention to obtain a joint segmentation feature map; performing a self-attention mechanism on the joint segmentation feature map to obtain a self-attention segmentation feature map; upsampling the self-attention segmentation feature map to obtain the predicted category and predicted category probability of each pixel in each image in the calibration training set; integrating a segmented text feature map into the distance text feature map and performing cross-attention to obtain a joint distance feature map; performing a self-attention mechanism on the joint distance feature map to obtain a self-attention distance feature map; performing a convolution operation on the self-attention distance feature map to obtain the predicted camera focal length; upsampling the self-attention distance feature map to obtain a second distance map; multiplying the second distance map and the predicted camera focal length to obtain the predicted distance of each pixel in each image in the calibration training set; and based on the distance in each image in the calibration training set... The predicted category and predicted category probability of each pixel are used to calculate the total segmentation loss value of the calibration training set, calculated from the manually calibrated category of each pixel in each image of the calibration training set. The original segmentation model is then updated in reverse based on the total segmentation loss value. A focal length loss value is calculated based on the predicted and actual camera focal lengths. A total distance loss value is calculated based on the predicted and manually calibrated distances of each pixel in each image of the calibration training set. The original distance model is then updated in reverse based on the focal length loss value and the total distance loss value. Step S2 is repeated to input the calibration training set into the original segmentation model for multiple rounds of model training until the total segmentation loss value fluctuates within a third preset range. Model training is then stopped, and the original segmentation model updated in the last round of training is used as the target segmentation model. Step S2 is repeated to input the calibration training set into the original distance model for multiple rounds of model training until the focal length loss value fluctuates within a fourth preset range and the total distance loss value fluctuates within a fifth preset range. Model training is then stopped, and the original distance model updated in the last round of training is used as the target distance model.

[0007] Optionally, calculating the pixel ratio of the target based on the target category distance map includes: counting the total number of pixels of the current target in the target category distance map to obtain a first pixel count; calculating the difference between the distance of each pixel of the current target in the target category distance map and the average distance of all pixels of the current target in the target category distance map; counting the total number of pixels of the current target in the target category distance map corresponding to the difference within a preset difference range to obtain a second pixel count; and obtaining the pixel ratio of the current target in the target category distance map based on the ratio of the second pixel count to the first pixel count.

[0008] Optionally, the step of calculating the length ratio of the current target based on the target category map and the target category distance map includes: obtaining the first predicted length of the current target image by passing the coordinates of the current target in the target category map through connected components; obtaining the actual distance of the current target based on the average distance of all pixels of the current target in the target category distance map; obtaining the second predicted length of the current target image based on the actual distance of the current target and the pre-statistical correspondence between the actual distance and the image length; and calculating the length ratio of the current target based on the first predicted length and the second predicted length of the current target image.

[0009] On the other hand, this invention provides a drone recognition system based on a deep recognition algorithm. The system includes: a calibration unit, used to acquire a training set via a camera, manually calibrating all pixels of each image in the training set, the target category corresponding to each pixel, and the target distance corresponding to each pixel, to obtain a calibration training set; a model training unit, used to input the calibration training set into the backbone network of the original segmentation model to obtain a segmentation feature map; input the calibration training set into the backbone network of the original distance model to obtain a distance feature map; input the "drone" font into a language encoder to obtain a descriptive feature vector; input the descriptive feature vector into the original segmentation model and perform joint training with the segmentation feature map to obtain a segmented text feature map; input the descriptive feature vector into the original distance model and perform joint training with the distance feature map to obtain a distance text feature map; and integrate the distance text feature map into the segmented text feature map to obtain a segmented text feature map. The original segmentation model is trained multiple times to obtain a target segmentation model; the segmentation text feature map is incorporated into the distance text feature map, and the original distance model is trained multiple times to obtain a target distance model; a recognition unit is used to input the image to be recognized into the target segmentation model and the target distance model for recognition, respectively, to obtain a target category map and a first distance map; the target category map and the first distance map are matched to obtain a target category distance map; the pixel ratio of each target is calculated according to the target category distance map; it is sequentially determined whether the pixel ratio of each target is within a first preset range. If not, the current target is determined not to be a drone; if so, the length ratio of the current target is calculated according to the target category map and the target category distance map, and it is determined whether the length ratio of the current target is within a second preset range. If not, the current target is determined not to be a drone; if so, the current target is determined to be a drone.

[0010] Optionally, the target distance corresponding to each pixel is obtained by mapping the two-dimensional coordinates of each pixel in the image to the three-dimensional LiDAR coordinate system to obtain the three-dimensional coordinates of each pixel; the height of the three-dimensional coordinates of each pixel is taken as the target distance of each pixel.

[0011] Optionally, the step of integrating a distance text feature map into the segmented text feature map and training the original segmentation model for multiple rounds to obtain the target segmentation model; integrating a segmented text feature map into the distance text feature map and training the original distance model for multiple rounds to obtain the target distance model includes: integrating a distance text feature map into the segmented text feature map and performing cross-attention to obtain a joint segmentation feature map; performing a self-attention mechanism on the joint segmentation feature map to obtain a self-attention segmentation feature map; upsampling the self-attention segmentation feature map to obtain the predicted category and predicted category probability of each pixel in each image in the calibration training set; integrating a segmented text feature map into the distance text feature map and performing cross-attention to obtain a joint distance feature map; performing a self-attention mechanism on the joint distance feature map to obtain a self-attention distance feature map; performing a convolution operation on the self-attention distance feature map to obtain the predicted camera focal length; upsampling the self-attention distance feature map to obtain a second distance map; multiplying the second distance map and the predicted camera focal length to obtain the predicted distance of each pixel in each image in the calibration training set; and multiplying the predicted distance of each pixel in each image in the calibration training set by the predicted distance of each pixel in the calibration training set. The total segmentation loss value of the calibration training set is calculated by taking the predicted category and predicted category probability of each pixel and the manually calibrated category of each pixel in each image of the calibration training set. The original segmentation model is then updated in reverse based on the total segmentation loss value. A focal length loss value is calculated based on the predicted and actual camera focal lengths. A total distance loss value of the calibration training set is calculated based on the predicted distance and manually calibrated distance of each pixel in each image of the calibration training set. The original distance model is then updated in reverse based on the focal length loss value and the total distance loss value. The model training unit repeatedly inputs the calibration training set into the original segmentation model for multiple rounds of model training until the total segmentation loss value fluctuates within a third preset range. Model training is then stopped, and the original segmentation model updated in the last round of training is used as the target segmentation model. The same model training unit repeatedly inputs the calibration training set into the original distance model for multiple rounds of model training until the focal length loss value fluctuates within a fourth preset range and the total distance loss value fluctuates within a fifth preset range. Model training is then stopped, and the original distance model updated in the last round of training is used as the target distance model.

[0012] Optionally, calculating the pixel ratio of the target based on the target category distance map includes: counting the total number of pixels of the current target in the target category distance map to obtain a first pixel count; calculating the difference between the distance of each pixel of the current target in the target category distance map and the average distance of all pixels of the current target in the target category distance map; counting the total number of pixels of the current target in the target category distance map corresponding to the difference within a preset difference range to obtain a second pixel count; and obtaining the pixel ratio of the current target in the target category distance map based on the ratio of the second pixel count to the first pixel count.

[0013] Optionally, the step of calculating the length ratio of the current target based on the target category map and the target category distance map includes: obtaining the first predicted length of the current target image by passing the coordinates of the current target in the target category map through connected components; obtaining the actual distance of the current target based on the average distance of all pixels of the current target in the target category distance map; obtaining the second predicted length of the current target image based on the actual distance of the current target and the pre-statistical correspondence between the actual distance and the image length; and calculating the length ratio of the current target based on the first predicted length and the second predicted length of the current target image.

[0014] The beneficial effects of this invention are:

[0015] This invention provides a drone identification method and system based on a depth recognition algorithm. The method adds semantic features to conventional segmentation algorithms, specifically by incorporating descriptive feature vectors for enhanced training, thereby increasing the model's ability to extract drone features. It also adds a distance calculation branch to the conventional segmentation algorithm, significantly reducing false positive and false negative rates. Furthermore, the distance calculation branch incorporates focal length supervision, enabling accurate depth calculation with only a standard camera. During drone identification, both pixel ratio and length ratio are used to determine whether a device is a drone, improving the accuracy of drone identification and reducing the false positive rate. Attached Figure Description

[0016] Figure 1 This is a flowchart of a drone identification method based on a depth recognition algorithm provided in an embodiment of the present invention;

[0017] Figure 2 This is a schematic diagram of the structure of a drone identification system based on a depth recognition algorithm provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] Figure 1 This is a flowchart of a drone identification method based on a depth recognition algorithm provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0020] S1. Obtain the training set through the camera, and manually label all pixels of each image in the training set, the target category corresponding to each pixel, and the target distance corresponding to each pixel to obtain the labeled training set;

[0021] The target distance for each pixel is obtained using the following method:

[0022] The three-dimensional coordinates of each pixel are obtained by mapping the two-dimensional coordinates of each pixel in the image to the three-dimensional laser coordinate system.

[0023] The height of the three-dimensional coordinates of each pixel is taken as the target distance for each pixel.

[0024] In one optional implementation, image data required for training, i.e., the training set, is acquired via a camera. The image acquisition process should ensure target diversity, including drone targets at different distances and in different poses. Each image should contain potential drone targets and may also include non-target objects (such as birds, kites, etc.) to improve the model's discrimination ability.

[0025] The collected images are labeled pixel by pixel, and the target category corresponding to each pixel in the image is manually identified. For example: Category 0: background (non-target area); Category 1: first type of drone; Category 2: second type of drone; Category 3: birds; Category 4: obstructions (such as power lines, tree branches, etc.).

[0026] Each pixel is labeled with its corresponding target distance. This distance information is not directly measured, but obtained through spatial registration of the image and LiDAR data (point cloud). The specific steps are as follows:

[0027] (1) 3D laser point cloud acquisition

[0028] Simultaneously, 3D LiDAR point cloud data corresponding to the images is acquired. The point cloud data uses the LiDAR as the origin and records the 3D spatial coordinates (X, Y, Z) of each object in the environment.

[0029] (2) External parameter calibration and coordinate mapping

[0030] Transform the point cloud data in the LiDAR coordinate system to the camera coordinate system using the known camera extrinsic parameters (R, T):

[0031]

[0032] in, = : Coordinates transformed to the camera coordinate system : represents the coordinates of a point in the lidar coordinate system, R is a 3×3 rotation matrix, and T is a 3×1 translation vector.

[0033] (3) Project the camera coordinate system coordinates onto the image plane

[0034] The coordinates transformed to the camera coordinate system are projected onto the image plane using the camera intrinsic parameter matrix K to obtain the two-dimensional pixel coordinates. :

[0035]

[0036] in, Two-dimensional pixel coordinates, = : These are the coordinates transformed to the camera coordinate system, and K represents the internal geometric properties of the camera.

[0037] (4) Establish the mapping relationship between two-dimensional pixels and three-dimensional space

[0038] By combining steps (2) and (3), a one-to-one correspondence is established between each 3D point in the point cloud data and a pixel in the image, thus obtaining a mapping table. → .

[0039] (5) Obtain the spatial distance of pixels

[0040] Finally, with the camera as the origin, the Z-axis coordinate (i.e., height direction) of each pixel in three-dimensional space is determined. Extract the target distance for that pixel.

[0041] After completing the above steps, the calibration training set is obtained, which includes three types of information: all pixels of each image, the category label of each pixel, and the distance label of each pixel.

[0042] S2. Input the calibration training set into the backbone network of the original segmentation model to obtain the segmentation feature map; input the calibration training set into the backbone network of the original distance model to obtain the distance feature map; input the "drone" font into the language encoder to obtain the descriptive feature vector; input the descriptive feature vector into the original segmentation model and perform joint training with the segmentation feature map to obtain the segmented text feature map; input the descriptive feature vector into the original distance model and perform joint training with the distance feature map to obtain the distance text feature map; integrate the distance text feature map into the segmented text feature map and perform multiple rounds of model training on the original segmentation model to obtain the target segmentation model; integrate the segmented text feature map into the distance text feature map and perform multiple rounds of model training on the original distance model to obtain the target distance model;

[0043] Specifically, after completing the training set calibration in step S1, the deep model is then trained. The core of this application is to construct two complementary models: one for pixel-level category prediction (i.e., object segmentation model) and the other for pixel-level spatial distance prediction (i.e., object distance model). These two models are based on the Vision Transformer encoder backbone network and incorporate semantic features and a mutual fusion mechanism to achieve multi-round collaborative optimization training.

[0044] First, two original models are initialized: the original segmentation model and the original distance model. Both are based on the Vision Transformer (ViT) architecture and include modules such as encoder, bottleneck layer, self-attention mechanism and upsampling decoder.

[0045] The calibration training set is input into the backbone network of the original segmentation model, and after being encoded by the segmentation ViT encoder, a segmentation feature map is obtained. The same image (i.e., the calibration training set) is input into the backbone network of the original distance model, and after being encoded by the distance ViT encoder, a distance feature map is obtained.

[0046] The keyword "drone" is input into a language encoder (such as BERT) to obtain a one-dimensional descriptive feature vector, which serves as a semantic prior to guide the semantic attention mechanism.

[0047] The descriptive feature vector and the segmentation feature map are fused at the bottleneck layer using a cross-attention mechanism to obtain the segmented text cross-attention feature map; the segmented text cross-attention feature map is then subjected to a self-attention mechanism to obtain the segmented text feature map (i.e., the segmented text self-attention feature map); the descriptive feature vector and the distance feature map are fused at the bottleneck layer using a cross-attention mechanism to obtain the distance text cross-attention feature map; the distance text cross-attention feature map is then subjected to a self-attention mechanism to obtain the distance text feature map (i.e., the distance text self-attention feature map).

[0048] The original segmentation model is trained through multiple rounds by incorporating a distance text feature map into the segmented text feature map to obtain a target segmentation model; the original distance model is trained through multiple rounds by incorporating a segmented text feature map into the distance text feature map to obtain a target distance model; this includes:

[0049] The segmentation joint feature map is obtained by incorporating the distance text feature map into the segmentation text feature map and performing cross-attention; the segmentation joint feature map is then subjected to a self-attention mechanism to obtain a segmentation self-attention feature map (i.e., forming a segmentation self-attention feature map with more global perception capability); the segmentation self-attention feature map is then upsampled to obtain the predicted category and predicted category probability of each pixel in each image in the calibration training set.

[0050] The distance joint feature map is obtained by incorporating the segmented text feature map into the distance text feature map and performing cross-attention; the distance joint feature map is then subjected to a self-attention mechanism to obtain a distance self-attention feature map; the distance self-attention feature map is then convolved (i.e., through three 3×3 convolutional layers and one fully connected layer) to obtain the predicted camera focal length; the distance self-attention feature map is then upsampled to obtain a second distance map; the second distance map and the predicted camera focal length are then multiplied by a dot to obtain the predicted distance of each pixel in each image in the calibration training set;

[0051] The total segmentation loss value for the calibration training set is calculated based on the predicted category and probability of each pixel in each image of the calibration training set, and the manually calibrated category of each pixel in each image of the calibration training set. The original segmentation model is then updated in reverse based on the total segmentation loss value. A focal length loss value is calculated based on the predicted camera focal length and the actual camera focal length. The total distance loss value for the calibration training set is calculated based on the predicted distance and the manually calibrated distance of each pixel in each image of the calibration training set. The original distance model is then updated in reverse based on the focal length loss value and the total distance loss value.

[0052] In one optional implementation, the total segmentation loss of the calibration training set is the sum of the segmentation loss values ​​of all images in the calibration training set; the segmentation loss value of each image in the calibration training set is calculated according to the following formula:

[0053]

[0054] in, To calibrate the segmentation loss value of the current image in the training set, X and Y represent the width and height of the current image in the training set, respectively. This indicates whether the predicted category of the pixel in the i-th row and j-th column of the current image in the calibration training set is consistent with the manually labeled category of the pixel in the i-th row and j-th column of the current image in the calibration training set. If they are consistent, then... =1, otherwise, =0, To calibrate the predicted class probability of the pixel in the i-th row and j-th column of the current image in the training set;

[0055] The focal length loss value is calculated according to the following formula:

[0056]

[0057] in, This is the focal length loss value. For the actual camera focal length, The predicted camera focal length;

[0058] The total distance loss of the calibration training set is the sum of the distance loss values ​​of all images in the calibration training set; the distance loss value of each image in the calibration training set is calculated according to the following formula:

[0059]

[0060] in, To calibrate the distance loss value of the current image in the training set, let X and Y represent the width and height of the current image in the training set, respectively. To calibrate the manually calibrated distance of the pixel in the i-th row and j-th column of the current image in the training set, To calibrate the predicted distance of the pixel in the i-th row and j-th column of the current image in the training set.

[0061] Repeat step S2 to input the calibrated training set into the original segmentation model for multiple rounds of model training until the total segmentation loss value is within the third preset range (i.e., If the focal length loss value fluctuates within a certain range, stop model training and use the original segmentation model obtained from the last round of training as the target segmentation model; repeat step S2 to input the calibrated training set into the original distance model for multiple rounds of model training until the focal length loss value is within the fourth preset range (i.e., ...). Fluctuations within the range and the total distance loss value within the fifth preset range (i.e.) If the distance fluctuates within a certain range, stop model training and use the original distance model obtained from the last round of training as the target distance model.

[0062] Specifically, the original segmentation model and the original distance model are trained simultaneously, with the total segmentation loss value as the monitoring metric. The calibrated training set is repeatedly input into the original segmentation model for training. When the total segmentation loss value tends to stabilize, i.e., within the third preset range (i.e., ... When the original segmentation model fluctuates within a certain range, it is considered to have converged. Training of the original segmentation model is stopped, and its parameters are frozen. The original segmentation model updated in the last round of training is used as the target segmentation model. Using both focal length loss and total distance loss as dual monitoring indicators, the calibrated training set is repeatedly input into the original distance model for training. When both focal length loss and total distance loss tend to stabilize, i.e., the focal length loss falls within the fourth preset range (i.e.,...), the model is considered to have converged. Fluctuations within ) and the total distance loss value within the fifth preset range (i.e. When the original distance model is considered to have converged during the fluctuation within the range, the training of the original distance model is stopped, the parameters of the original distance model are frozen, and the original distance model obtained from the last round of training is used as the target distance model.

[0063] S3. Input the image to be identified into the target segmentation model and the target distance model respectively for identification to obtain the target category map and the first distance map; match the target category map and the first distance map to obtain the target category distance map; calculate the pixel ratio of each target based on the target category distance map; sequentially determine whether the pixel ratio of each target is within the first preset range. If not, determine that the current target is not a drone; if so, calculate the length ratio of the current target based on the target category map and the target category distance map, and determine whether the length ratio of the current target is within the second preset range. If not, determine that the current target is not a drone; if so, determine that the current target is a drone.

[0064] After completing the model training in step S2 and obtaining a stable target segmentation model and target distance model, this step performs actual inference and judgment on the image to be identified. It comprehensively uses pixel category and spatial distance information to identify and discriminate the target appearing in the image.

[0065] The image to be identified is simultaneously input into two trained models: the target segmentation model and the target distance model. The target segmentation model outputs a target category map, which represents all pixels in the image to be identified that are predicted to be classified as drones. The target distance model outputs a first distance map, which represents the predicted distance of each pixel in the image to be identified.

[0066] The target category map and the first distance map are fused at the pixel level to obtain the target category distance map; that is, the category of all pixels in the target category map that are predicted to be UAV is mapped to the predicted distance of all pixels in the first distance map, and the target category distance map is obtained; the target category distance map is represented as all pixels that are predicted to be UAV (showing both the predicted category and the predicted distance).

[0067] The pixel proportion of each target is calculated based on the target category distance map; this includes:

[0068] The first pixel count is obtained by counting the number of all pixels of the current target in the target category distance map;

[0069] The difference between the distance of the target category to each pixel of the current target in the map and the mean of the distances of the target category to all pixels of the current target in the map is calculated.

[0070] The number of all pixels of the current target in the target category distance map corresponding to the difference within the preset difference range is counted to obtain the number of the second pixel;

[0071] In a preferred embodiment of this application, the preset difference value is 0.5. It should be noted that this application does not limit the preset difference value. The number of the second pixel is calculated according to the following formula:

[0072]

[0073] in, Here, X represents the number of second-order pixels, and X and Y represent the width and height of the current target in the target category distance map, respectively. This represents the distance of the target category from the pixel in the i-th row and j-th column of the current target in the map. This represents the average distance between the target category and all pixels of the current target in the graph.

[0074] The pixel proportion of the current target in the target category distance map is obtained by calculating the ratio of the number of second pixels to the number of first pixels. The formula is as follows:

[0075]

[0076] in, This represents the pixel proportion of the current target in the target category distance map. The number of the second pixel. This represents the number of the first pixel.

[0077] The pixel ratio of each target in the target category distance map can be obtained sequentially using the above method. Then, it is determined whether the pixel ratio of each target is within the first preset range. If not, it means that the pixel distance difference within the current target is large and lacks spatial consistency, and the current target is determined not to be a drone. If it is, the length ratio of the current target is calculated based on the target category map and the target category distance map.

[0078] The calculation of the length ratio of the current target based on the target category map and the target category distance map includes:

[0079] The first predicted length of the current target image is obtained by passing the coordinates of the current target in the target category map through connected components;

[0080] By analyzing the connected components of the current target in the target category map, the length of its maximum connected component boundary is extracted, and the first predicted length of the current target image is obtained.

[0081] The actual distance to the current target is obtained based on the average distance of all pixels of the current target in the target category distance map; the second predicted length of the current target image is obtained based on the actual distance of the current target and the pre-statistical correspondence between the actual distance and the image length.

[0082] The length ratio of the current target is calculated based on the first predicted length and the second predicted length of the current target image. The formula is as follows:

[0083]

[0084] in, The length ratio of the current target. The first predicted length of the current target image. This is the second predicted length of the current target image.

[0085] Determine if the length ratio of the current target is within the second preset range. If not, determine if the current target is not a drone; if so, determine if the current target is a drone.

[0086] The above judgment process is executed sequentially for all targets in the target category distance map, and finally all targets that meet the following two conditions are output as "drones": the pixel ratio is within the first preset range and the length ratio is within the second preset range.

[0087] The UAV recognition method based on deep recognition algorithms proposed in this application overcomes the problems of high false negative and high false positive rates in existing technologies by constructing a dual deep network structure of target segmentation model and target distance model, and introducing semantic prior fusion, pixel-level deep modeling, and pixel ratio and length ratio discrimination mechanism. By using point cloud data to perform spatial distance calibration on images during the training phase, a training set with real distance labels is constructed, enabling the model to learn the mapping relationship between pixels and spatial distance in the image. In the actual use (inference) phase, only a two-dimensional image needs to be input, and the system can automatically predict the spatial distance of each pixel through the trained deep model, thereby realizing distance perception function without relying on LiDAR, effectively getting rid of dependence on high-cost hardware such as LiDAR, and significantly reducing the cost of use.

[0088] Figure 2 This is a schematic diagram of the structure of a drone identification system based on a depth recognition algorithm provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the system includes:

[0089] The calibration unit 201 is used to acquire the training set through the camera, manually calibrate all pixels of each image in the training set, the target category corresponding to each pixel, and the target distance corresponding to each pixel, so as to obtain the calibration training set.

[0090] The model training unit 202 is used to input the calibration training set into the backbone network of the original segmentation model to obtain a segmentation feature map; input the calibration training set into the backbone network of the original distance model to obtain a distance feature map; input the "drone" font into the language encoder to obtain a descriptive feature vector; input the descriptive feature vector into the original segmentation model and perform joint training with the segmentation feature map to obtain a segmented text feature map; input the descriptive feature vector into the original distance model and perform joint training with the distance feature map to obtain a distance text feature map; integrate the distance text feature map into the segmented text feature map and perform multiple rounds of model training on the original segmentation model to obtain a target segmentation model; integrate the segmented text feature map into the distance text feature map and perform multiple rounds of model training on the original distance model to obtain a target distance model.

[0091] The recognition unit 203 is used to input the image to be recognized into the target segmentation model and the target distance model respectively for recognition, to obtain a target category map and a first distance map; to match the target category map and the first distance map to obtain a target category distance map; to calculate the pixel ratio of each target based on the target category distance map; to determine whether the pixel ratio of each target is within a first preset range. If not, the current target is determined not to be a drone; if so, the length ratio of the current target is calculated based on the target category map and the target category distance map, and the length ratio of the current target is determined to be within a second preset range. If not, the current target is determined not to be a drone; if so, the current target is determined to be a drone.

[0092] In an optional implementation, the target distance corresponding to each pixel is obtained according to the following method:

[0093] The three-dimensional coordinates of each pixel are obtained by mapping the two-dimensional coordinates of each pixel in the image to the three-dimensional LiDAR coordinate system.

[0094] The height of the three-dimensional coordinates of each pixel is taken as the target distance for each pixel.

[0095] In an optional implementation, the step of incorporating a distance text feature map into the segmented text feature map and training the original segmentation model for multiple rounds to obtain the target segmentation model; and the step of incorporating a segmented text feature map into the distance text feature map and training the original distance model for multiple rounds to obtain the target distance model, includes:

[0096] The segmentation joint feature map is obtained by incorporating the distance text feature map into the segmentation text feature map and performing cross-attention; the segmentation joint feature map is then subjected to a self-attention mechanism to obtain the segmentation self-attention feature map; the segmentation self-attention feature map is then upsampled to obtain the predicted category and predicted category probability of each pixel in each image in the labeled training set;

[0097] The distance text feature map is integrated into the distance text feature map and cross-attention is applied to obtain the distance joint feature map; the distance joint feature map is then subjected to a self-attention mechanism to obtain the distance self-attention feature map; the distance self-attention feature map is then convolved to obtain the predicted camera focal length; the distance self-attention feature map is then upsampled to obtain the second distance map; the second distance map and the predicted camera focal length are then multiplied by a dot to obtain the predicted distance of each pixel in each image in the calibration training set;

[0098] The total segmentation loss value for the calibration training set is calculated based on the predicted category and probability of each pixel in each image of the calibration training set, and the manually calibrated category of each pixel in each image of the calibration training set. The original segmentation model is then updated in reverse based on the total segmentation loss value. A focal length loss value is calculated based on the predicted camera focal length and the actual camera focal length. The total distance loss value for the calibration training set is calculated based on the predicted distance and the manually calibrated distance of each pixel in each image of the calibration training set. The original distance model is then updated in reverse based on the focal length loss value and the total distance loss value.

[0099] The repeated model training unit inputs the calibrated training set into the original segmentation model for multiple rounds of model training until the total segmentation loss value fluctuates within a third preset range. Then, the model training stops, and the original segmentation model updated in the last round of training is used as the target segmentation model. The repeated model training unit inputs the calibrated training set into the original distance model for multiple rounds of model training until the focal length loss value fluctuates within a fourth preset range and the total distance loss value fluctuates within a fifth preset range. Then, the model training stops, and the original distance model updated in the last round of training is used as the target distance model.

[0100] In an optional implementation, calculating the pixel ratio of the target based on the target category distance map includes:

[0101] The first pixel count is obtained by counting the number of all pixels of the current target in the target category distance map;

[0102] The difference between the distance of the target category to each pixel of the current target in the map and the mean of the distances of the target category to all pixels of the current target in the map is calculated.

[0103] The number of all pixels of the current target in the target category distance map corresponding to the difference within the preset difference range is counted to obtain the number of the second pixel;

[0104] The pixel proportion of the current target in the target category distance map is obtained by calculating the ratio of the number of second pixels to the number of first pixels.

[0105] In an optional implementation, calculating the length ratio of the current target based on the target category map and the target category distance map includes:

[0106] The first predicted length of the current target image is obtained by passing the coordinates of the current target in the target category map through connected components;

[0107] The actual distance to the current target is obtained based on the average distance of all pixels of the current target in the target category distance map; the second predicted length of the current target image is obtained based on the actual distance of the current target and the pre-statistical correspondence between the actual distance and the image length.

[0108] The length ratio of the current target is calculated based on the first predicted length and the second predicted length of the current target image.

[0109] The beneficial effects of this invention are:

[0110] This invention introduces a semantic description feature vector for "drone" on top of the traditional visual segmentation model. It enhances the model's ability to recognize target semantic features through a cross-attention mechanism, achieving effective detection of long-range, small-sized drones and significantly reducing the false negative rate. This invention integrates pixel category and spatial distance dimensions for fusion judgment. By constructing a target category distance map and introducing a pixel ratio and length ratio judgment mechanism, it significantly reduces false recognition of interfering objects such as birds and kites, effectively improving the model's robustness in complex scenes. This invention employs a joint training mechanism, simultaneously optimizing segmentation loss, distance loss, and focal length loss to improve the synergistic effect between the two models, avoiding the problems of independent training and unstable fusion effects in traditional models. This invention eliminates the real-time dependence on LiDAR equipment. During the training phase, it uses point cloud to establish 3D distance annotations for each image pixel. During the inference phase, it only uses ordinary cameras to acquire images, and the model automatically predicts the distance map, eliminating the need for real-time point cloud input and LiDAR dependence, greatly reducing deployment costs.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying a UAV based on a deep recognition algorithm, characterized in that, include: S1. Obtain the training set through the camera, and manually label all pixels of each image in the training set, the target category corresponding to each pixel, and the target distance corresponding to each pixel to obtain the labeled training set; S2. Input the calibration training set into the backbone network of the original segmentation model to obtain the segmentation feature map; input the calibration training set into the backbone network of the original distance model to obtain the distance feature map; input the "drone" font into the language encoder to obtain the descriptive feature vector; input the descriptive feature vector into the original segmentation model and perform joint training with the segmentation feature map to obtain the segmented text feature map; input the descriptive feature vector into the original distance model and perform joint training with the distance feature map to obtain the distance text feature map; integrate the distance text feature map into the segmented text feature map and perform multiple rounds of model training on the original segmentation model to obtain the target segmentation model; integrate the segmented text feature map into the distance text feature map and perform multiple rounds of model training on the original distance model to obtain the target distance model; S3. Input the image to be identified into the target segmentation model and the target distance model respectively for identification to obtain the target category map and the first distance map; match the target category map and the first distance map to obtain the target category distance map; calculate the pixel ratio of each target based on the target category distance map; The pixel ratio of each target is sequentially determined to be within the first preset range. If not, the current target is determined not to be a drone. If so, calculate the length ratio of the current target based on the target category map and the target category distance map, and determine whether the length ratio of the current target is within the second preset range. If not, determine that the current target is not a drone. If so, then the current target is determined to be a drone.

2. The method of claim 1, wherein, The target distance for each pixel is obtained using the following method: The three-dimensional coordinates of each pixel are obtained by mapping the two-dimensional coordinates of each pixel in the image to the three-dimensional LiDAR coordinate system. The height of the three-dimensional coordinates of each pixel is taken as the target distance for each pixel.

3. The method of claim 1, wherein, The process of integrating a distance text feature map into the segmented text feature map and training the original segmentation model through multiple rounds to obtain the target segmentation model includes: The segmentation joint feature map is obtained by incorporating the distance text feature map into the segmentation text feature map and performing cross-attention; the segmentation joint feature map is then subjected to a self-attention mechanism to obtain the segmentation self-attention feature map; the segmentation self-attention feature map is then upsampled to obtain the predicted category and predicted category probability of each pixel in each image in the labeled training set; The distance text feature map is integrated into the distance text feature map and cross-attention is applied to obtain the distance joint feature map; the distance joint feature map is then subjected to a self-attention mechanism to obtain the distance self-attention feature map; the distance self-attention feature map is then convolved to obtain the predicted camera focal length; the distance self-attention feature map is then upsampled to obtain the second distance map; the second distance map and the predicted camera focal length are then multiplied by a dot to obtain the predicted distance of each pixel in each image in the calibration training set; The total segmentation loss value for the calibration training set is calculated based on the predicted category and probability of each pixel in each image of the calibration training set, and the manually calibrated category of each pixel in each image of the calibration training set. The original segmentation model is then updated in reverse based on the total segmentation loss value. A focal length loss value is calculated based on the predicted camera focal length and the actual camera focal length. The total distance loss value for the calibration training set is calculated based on the predicted distance and the manually calibrated distance of each pixel in each image of the calibration training set. The original distance model is then updated in reverse based on the focal length loss value and the total distance loss value. Repeat step S2 to input the calibrated training set into the original segmentation model for multiple rounds of model training until the total segmentation loss value fluctuates within the third preset range. Stop model training and use the original segmentation model updated in the last round of training as the target segmentation model. Repeat step S2 to input the calibrated training set into the original distance model for multiple rounds of model training until the focal length loss value fluctuates within the fourth preset range and the total distance loss value fluctuates within the fifth preset range. Stop model training and use the original distance model updated in the last round of training as the target distance model.

4. The method of claim 1, wherein, The calculation of the pixel ratio of the target based on the target category distance map includes: The first pixel count is obtained by counting the number of all pixels of the current target in the target category distance map; The difference between the distance of the target category to each pixel of the current target in the map and the mean of the distances of the target category to all pixels of the current target in the map is calculated. The number of all pixels of the current target in the target category distance map corresponding to the difference within the preset difference range is counted to obtain the number of the second pixel; The pixel proportion of the current target in the target category distance map is obtained by calculating the ratio of the number of second pixels to the number of first pixels.

5. The method of claim 4, wherein, The calculation of the length ratio of the current target based on the target category map and the target category distance map includes: The first predicted length of the current target image is obtained by passing the coordinates of the current target in the target category map through connected components; The actual distance to the current target is obtained based on the average distance of all pixels of the current target in the target category distance map; the second predicted length of the current target image is obtained based on the actual distance of the current target and the pre-statistical correspondence between the actual distance and the image length. The length ratio of the current target is calculated based on the first predicted length and the second predicted length of the current target image. 6.A drone identification system based on deep recognition algorithm, characterized in that, include: The calibration unit is used to acquire the training set through the camera, and manually calibrate all pixels of each image in the training set, the target category corresponding to each pixel, and the target distance corresponding to each pixel to obtain the calibration training set; The model training unit is used to input the calibration training set into the backbone network of the original segmentation model to obtain a segmentation feature map; input the calibration training set into the backbone network of the original distance model to obtain a distance feature map; input the "drone" font into the language encoder to obtain a descriptive feature vector; input the descriptive feature vector into the original segmentation model and perform joint training with the segmentation feature map to obtain a segmented text feature map; input the descriptive feature vector into the original distance model and perform joint training with the distance feature map to obtain a distance text feature map; integrate the distance text feature map into the segmented text feature map and perform multiple rounds of model training on the original segmentation model to obtain a target segmentation model; integrate the segmented text feature map into the distance text feature map and perform multiple rounds of model training on the original distance model to obtain a target distance model. The recognition unit is used to input the image to be recognized into the target segmentation model and the target distance model for recognition, and to obtain the target category map and the first distance map. The target category map is obtained by matching the target category map with the first distance map; the pixel ratio of each target is calculated based on the target category distance map. The pixel ratio of each target is sequentially determined to be within the first preset range. If not, the current target is determined not to be a drone. If so, calculate the length ratio of the current target based on the target category map and the target category distance map, and determine whether the length ratio of the current target is within the second preset range. If not, determine that the current target is not a drone. If so, then the current target is determined to be a drone.

7. The system of claim 6, wherein, The target distance for each pixel is obtained using the following method: The three-dimensional coordinates of each pixel are obtained by mapping the two-dimensional coordinates of each pixel in the image to the three-dimensional LiDAR coordinate system. The height of the three-dimensional coordinates of each pixel is taken as the target distance for each pixel.

8. The system of claim 6, wherein, The process of integrating a distance text feature map into the segmented text feature map and training the original segmentation model through multiple rounds to obtain the target segmentation model includes: The segmentation joint feature map is obtained by incorporating the distance text feature map into the segmentation text feature map and performing cross-attention; the segmentation joint feature map is then subjected to a self-attention mechanism to obtain the segmentation self-attention feature map; the segmentation self-attention feature map is then upsampled to obtain the predicted category and predicted category probability of each pixel in each image in the labeled training set; The distance text feature map is integrated into the distance text feature map and cross-attention is applied to obtain the distance joint feature map; the distance joint feature map is then subjected to a self-attention mechanism to obtain the distance self-attention feature map; the distance self-attention feature map is then convolved to obtain the predicted camera focal length; the distance self-attention feature map is then upsampled to obtain the second distance map; the second distance map and the predicted camera focal length are then multiplied by a dot to obtain the predicted distance of each pixel in each image in the calibration training set; The total segmentation loss value for the calibration training set is calculated based on the predicted category and probability of each pixel in each image of the calibration training set, and the manually calibrated category of each pixel in each image of the calibration training set. The original segmentation model is then updated in reverse based on the total segmentation loss value. A focal length loss value is calculated based on the predicted camera focal length and the actual camera focal length. The total distance loss value for the calibration training set is calculated based on the predicted distance and the manually calibrated distance of each pixel in each image of the calibration training set. The original distance model is then updated in reverse based on the focal length loss value and the total distance loss value. The repeated model training unit inputs the calibrated training set into the original segmentation model for multiple rounds of model training until the total segmentation loss value fluctuates within a third preset range. Then, the model training stops, and the original segmentation model updated in the last round of training is used as the target segmentation model. The repeated model training unit inputs the calibrated training set into the original distance model for multiple rounds of model training until the focal length loss value fluctuates within a fourth preset range and the total distance loss value fluctuates within a fifth preset range. Then, the model training stops, and the original distance model updated in the last round of training is used as the target distance model.

9. The system of claim 6, wherein, The calculation of the pixel ratio of the target based on the target category distance map includes: The first pixel count is obtained by counting the number of all pixels of the current target in the target category distance map; The difference between the distance of the target category to each pixel of the current target in the map and the mean of the distances of the target category to all pixels of the current target in the map is calculated. The number of all pixels of the current target in the target category distance map corresponding to the difference within the preset difference range is counted to obtain the number of the second pixel; The pixel proportion of the current target in the target category distance map is obtained by calculating the ratio of the number of second pixels to the number of first pixels.

10. The system of claim 9, wherein, The calculation of the length ratio of the current target based on the target category map and the target category distance map includes: The first predicted length of the current target image is obtained by passing the coordinates of the current target in the target category map through connected components; The actual distance to the current target is obtained based on the average distance of all pixels of the current target in the target category distance map; the second predicted length of the current target image is obtained based on the actual distance of the current target and the pre-statistical correspondence between the actual distance and the image length. The length ratio of the current target is calculated based on the first predicted length and the second predicted length of the current target image.

Images