A high-dynamic multi-target unmanned aerial vehicle detection method based on feature fusion

Abstract

The application provides a high dynamic multi-target unmanned aerial vehicle detection method based on feature fusion, which comprises the following steps: real-time detection, image acquisition; feature extraction and feature fusion; morphological operation; target contour searching; target searching; error compensation for multiple detection and missed detection existing in the detection; and target output according to the detection and compensation judgment. The application mainly aims at the problem that multiple moving targets are easily lost or misdetected when they are staggered, has certain robustness to light, extracts a target original binary image through color space conversion, judges whether to perform morphological dilation on the filtered binary image through brightness detection after median filtering and Gaussian filtering, then performs edge detection on the relatively accurate target binary image to obtain a minimum enclosing rectangle frame, draws the corresponding motion trajectory of the target corresponding to the color after frame-by-frame detection, and completes multi-target video recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-target unmanned aerial vehicle dynamic monitoring, and particularly relates to a high-dynamic multi-target unmanned aerial vehicle detection method based on feature fusion. BACKGROUND

[0002] The detection and identification of unmanned aerial vehicles are important challenges in security and defense, and there is still a lack of defense means for unmanned aerial vehicles in the military field. In practical applications, the ground contains a large amount of clutter in the down-looking scene in ground and low-altitude radar target identification. Some scholars have also adopted deep learning-based algorithms in their research. A large number of standard samples are required in the target detection and identification process. At present, there is no such public unmanned aerial vehicle dataset as MNIST, COCO, and PASCAL VOC. Moreover, unmanned aerial vehicles are of various types and sizes, and it is obviously very difficult to realize the collection of all models and sizes of unmanned aerial vehicles as training data sets for training. The traditional unmanned aerial vehicle low-altitude defense system mainly detects static targets, and the accuracy of dynamic detection is not high. Therefore, a high-dynamic multi-target unmanned aerial vehicle detection method based on unmanned aerial vehicle formation and unmanned aerial vehicle feature fusion extraction is proposed to realize the positioning detection of multiple unmanned aerial vehicles. SUMMARY

[0003] Therefore, the present application provides a high-dynamic multi-target unmanned aerial vehicle detection method based on feature fusion, which mainly aims at the problem of easy loss of target or false detection when multiple moving targets are tracked, has a certain robustness to light, extracts a target original binary image through color space conversion, performs morphological dilation on the filtered binary image through brightness detection after median filtering and Gaussian filtering, then performs edge detection on the relatively accurate target binary image to obtain a minimum enclosing rectangle frame, and draws the motion trajectory corresponding to the target corresponding to the color after frame-by-frame detection, thereby completing the video recognition of multiple targets.

[0004] To solve the above technical problems, the present application provides a high-dynamic multi-target unmanned aerial vehicle detection method based on feature fusion, which comprises the following steps:

[0005] Step 1: real-time detection to obtain images;

[0006] Step 2: feature extraction and feature fusion;

[0007] Step 3: morphological operation;

[0008] Step 4: finding the target contour;

[0009] Step 5: target search;

[0010] Step 6: error compensation for multiple detections and missed detections in the detection;

[0011] Step 7: According to the judgment of detection and compensation, output the target.

[0012] Further, in step 1, the image information is collected in real time by the low-altitude security system, and the image size is 1920*1080.

[0013] Further, in step 2, the feature extraction includes color feature extraction and texture feature extraction.

[0014] Further, the color feature extraction includes the following steps:

[0015] The image is subjected to median filtering processing; a one-dimensional sequence f1, f2, …, f n , take the window length m, m is an odd number, the expression of median filtering is as follows:

[0016]

[0017] Color space conversion, convert the input RGB image to YCrCb;

[0018] Split the channel that stores the color, extract the red component about the unmanned aerial vehicle, and save it separately, and set the parameter threshold.

[0019] Further, the texture feature extraction adopts the LBP method, and the specific steps are as follows:

[0020] First, divide the window into 16*16 cells, for each pixel in each cell, compare the gray values of the adjacent 8 pixels with it, if the surrounding pixel value is greater than the center pixel value, mark it as 1, otherwise as 0, 8 points in 3*3 neighborhood can generate 8-bit binary number, that is, the LBP value of the center pixel of the window can be obtained;

[0021] Calculate the histogram of each cell, that is, the frequency of each number, and normalize the histogram;

[0022] Finally, the statistical histogram of each cell is connected to form a feature vector, which is the texture feature vector of the whole image.

[0023] Further, in step 2, the feature fusion based on the unmanned aerial vehicle and the algorithm optimization based on the formation of the unmanned aerial vehicle, the specific steps are as follows:

[0024] Gaussian normalization method is used for normalization between features;

[0025] Based on the formation of multiple unmanned aerial vehicles flying in the air, as a way of error compensation in the process of unmanned aerial vehicle detection.

[0026] Further, the morphological operation of step 3 includes the following steps:

[0027] First, set the dilated erosion convolution kernel B, based on the effect of image processing parameter value adjustment, convolution kernel B and image convolution, and calculate the local maximum minimum value of the coverage area;

[0028] The extracted red region is first dilated and then eroded;

[0029] Output image.

[0030] Further, the step 4 of finding the target contour includes the following steps:

[0031] Set the contour number space;

[0032] Set the contour index space;

[0033] Use the findContours function in opencv to find all contours;

[0034] Draw the boundary contour.

[0035] Further, the target search in step 5 includes the following steps:

[0036] Input the total number of aircrafts to be detected num;

[0037] Select the detection function according to the number;

[0038] Find the center point centerPoint[i] of each rectangular frame of the above contour, where i is the current contour, and set the vector space to store centerPoint;

[0039] Find the center point coordinates point(x[i], y[i]) and store them;

[0040] Record the average height of each contour center and calculate the average height avH of all target frames;

[0041]

[0042] Find the maximum and minimum target frame y coordinates:

[0043] max.y = max{centerPoint[i].y}

[0044] min.y = min{centerPoint[i].y};

[0045] Determine whether the center point is located above or below the overall image, that is, determine the size of avH and image height rows / 2, greater than below, otherwise above;

[0046] In step 6, according to the center point position and target frame data, it is judged whether error compensation is needed.

[0047] Further, in step 6, firstly, the number of target frames is compared with the number of aircrafts,

[0048] When the number of target frames is equal to the number of unmanned aerial vehicles:

[0049] It is judged whether the target frame positions centerPoint[i].y are close, i.e. the difference is less than a certain threshold value;

[0050] It is judged whether the formation meets the input, i.e. three aircrafts or two aircrafts exist with the same slope, which is determined in turn according to the total number of aircrafts;

[0051] When the above two requirements are met, the target is outputted completely;

[0052] When the number of target frames is less than the number of unmanned aerial vehicles, i.e. there is suspected missing detection:

[0053] The number of target frames is counted;

[0054] When the target frame coordinates point(x[i],y[i]) are compared with the image boundary (a,0)(0,b)(1980,b)(a,1080), a∈(0,1980) b∈(0,1080), when less than a certain threshold value, it is judged that there is part of the aircrafts not in the field of view due to the boundary, and the existing target is outputted, and the boundary value can be changed due to different image sizes;

[0055] When greater than the threshold value, each target frame is counted, and it is judged whether the surrounding target frame meets the requirement, i.e. according to the unmanned aerial vehicle formation, the number of aircrafts in the same slope in three directions is the same, a missing target in a position is judged, and a new target frame is supplemented in the missing position;

[0056] When the number of target frames is greater than the number of unmanned aerial vehicles, i.e. there is multiple detection:

[0057] The target frame centerPoint[i].y is less than a certain threshold value, the target frames next to each other are merged as the same target, the situation of the target frame point(x[i],y[i]) around each target frame is counted, the target frames with close coordinates that meet the requirement are retained, and the suspected target frames far away from the target group are removed.

[0058] The beneficial effects of the above technical solutions of the present application are as follows:

[0059] 1. The key technical point of the present application lies in the detection of dynamic targets;

[0060] 2. The detection of high dynamic background;

[0061] 3. The detection of multiple unmanned aerial vehicle targets;

[0062] 4. The beneficial effect of the present invention is that the present invention designs a high dynamic multi-target UAV detection method based on feature fusion. The designed method can meet the detection of different numbers of UAVs in a high dynamic background and can also distinguish them from the surrounding environment.

[0063] 5. The algorithm designed in this invention achieves low-altitude security, improves identification accuracy, and facilitates simultaneous detection of multiple targets. This invention is not limited to any type of drone and has strong versatility. It can be applied to different types of drones, including rotary-wing, fixed-wing, and helicopter drones, significantly improving the intelligence level of anti-drone systems. Attached Figure Description

[0064] Fig. 1 This is a flowchart of the high dynamic multi-target UAV detection method based on feature fusion according to the present invention;

[0065] Fig. 2 This is a flowchart of the color feature extraction process of the present invention;

[0066] Fig. 3 This is a flowchart of the morphological operations of the present invention;

[0067] Fig. 4 This is a flowchart illustrating the process of finding the target contour in this invention.

[0068] Fig. 5 This is a flowchart of the target search process of the present invention;

[0069] Fig. 6 This is a flowchart of the detection error compensation process of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will be described in conjunction with the accompanying drawings of the embodiments of the present invention. Figs. 1-6 The technical solutions of the embodiments of the present invention will be clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0071] like Figs. 1-6 As shown: This embodiment provides a high-dynamic multi-target UAV detection method based on feature fusion, including the following steps:

[0072] Step 1: Real-time detection and image acquisition;

[0073] Step 2: Feature extraction and feature fusion;

[0074] Step 3: Morphological operations;

[0075] Step 4: Locate the target outline;

[0076] Step 5: target search;

[0077] Step 6: error compensation for multiple detection and missed detection in detection;

[0078] Step 7: output the target according to the detection and compensation.

[0079] Further, in step 1, the image information is collected in real time by the low-altitude security system, and the image size is 1920*1080.

[0080] Further, in step 2, the feature extraction includes color feature extraction and texture feature extraction.

[0081] Further, the color feature extraction includes the following steps:

[0082] The image is subjected to median filtering processing; a one-dimensional sequence f1, f2, …, f n , take the window length m, m is an odd number, and the expression of median filtering is as follows:

[0083]

[0084] Color space conversion, convert the input RGB image to YCrCb;

[0085] Split the channel storing the color, extract the red component about the unmanned aerial vehicle, and save it separately, and set the parameter threshold.

[0086] Further, the texture feature extraction adopts the LBP method, and the specific steps are as follows:

[0087] First, divide the window into 16*16 cells, compare the gray values of the 8 adjacent pixels in each cell with the center pixel, if the surrounding pixel value is greater than the center pixel value, mark it as 1, otherwise as 0, 8 points in a 3*3 neighborhood can generate 8-bit binary number, that is, the LBP value of the center pixel of the window can be obtained;

[0088] Calculate the histogram of each cell, that is, the frequency of each number, and normalize the histogram;

[0089] Finally, the statistical histogram of each cell is connected to form a feature vector, which is the texture feature vector of the whole image.

[0090] Further, the feature fusion based on the unmanned aerial vehicle and the algorithm optimization based on the unmanned aerial vehicle formation in step 2 are as follows:

[0091] Gaussian normalization method for feature normalization;

[0092] As an error compensation approach in the process of detecting unmanned aerial vehicles, the formation of multiple unmanned aerial vehicles flying in the air is used.

[0093] Further, the morphological operation of step 3 includes the following steps:

[0094] First, set the dilation erosion convolution kernel B, adjust the parameter value based on the effect of image processing, and perform convolution on the image with the convolution kernel B, and calculate the local maximum and minimum value of the covered area;

[0095] Perform the operation of first dilation and then erosion on the extracted red region;

[0096] Output the image.

[0097] Further, the target contour searching of step 4 includes the following steps:

[0098] Set the contour number space;

[0099] Set the contour index space;

[0100] Use the findContours function in opencv to find all contours;

[0101] Draw the boundary contour.

[0102] Further, the target searching in step 5 includes the following steps:

[0103] Input the total number of aircrafts to be detected num;

[0104] Select the detection function according to the number;

[0105] Find the center point centerPoint[i] of each rectangular frame of the above contour, where i is the current contour, and set a vector space to store centerPoint;

[0106] Find the center point coordinates point(x[i], y[i]) and store them;

[0107] Record the average height of each contour center, and calculate the average height avH of all target frames;

[0108]

[0109] Find the maximum and minimum target frame y coordinates:

[0110] max.y = max{centerPoint[i].y}

[0111] min.y = min{centerPoint[i].y};

[0112] Determine whether the center point is above or below the overall image, i.e., determine the size of avH and the image height rows / 2, greater than, i.e., below, otherwise above;

[0113] In step 6, determine whether error compensation is needed according to the center point position and target frame data.

[0114] Further, in step 6, first determine the comparison between the number of target frames and the number of aircrafts,

[0115] When the number of target frames is equal to the number of unmanned aerial vehicles:

[0116] Determine whether the target frame positions centerPoint[i].y are close, i.e., the difference is less than a certain threshold;

[0117] Determine whether the formation meets the input, i.e., three aircrafts or two aircrafts exist with the same slope, depending on the total number of aircrafts in turn;

[0118] When the above two requirements are met, output all targets;

[0119] When the number of target frames is less than the number of unmanned aerial vehicles, i.e., there is suspected missing detection:

[0120] Count the number of target frames;

[0121] When the target frame coordinates point(x[i],y[i]) and the image boundary(a,0)(0,b)(1980,b)(a,1080), a∈(0,1980) b∈(0,1080) are compared, when less than a certain threshold, it is determined that there is a part of the aircraft not in the field of view due to the boundary, output the existing target, the boundary value can vary due to different image sizes;

[0122] When greater than the threshold, count each target frame and determine whether there is a surrounding target frame that meets the requirements, i.e., according to the unmanned aerial vehicle formation, there are the same number of aircrafts in three directions with the same slope, determine that a position is missing a target, i.e., supplement a new target frame at the missing position;

[0123] When the number of target frames is greater than the number of unmanned aerial vehicles, i.e., there is multiple detection:

[0124] When the target frame centerPoint[i].y is less than a certain threshold, merge the target frames next to each other as the same target, count the target frame point(x[i],y[i]) around each target frame, keep the target frames that meet the requirements with close coordinates, and eliminate the suspected target frames that are far away from the target group.

[0125] In the present application, unless otherwise explicitly specified and limited, for example, can be fixedly connected, or can be detachably connected, or integrated; can be mechanically connected, or electrically connected; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction relationship of two elements, unless otherwise explicitly limited, the specific meaning of the above-mentioned terms in the present application can be understood according to the specific circumstances by those skilled in the art.

[0126] The above is the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can also be made, which should be considered as the protection scope of the present application.

Claims

1. A high dynamic multi-target UAV detection method based on feature fusion, characterized in that: Comprising the following steps: Step 1: Real-time detection, image acquisition; Step 2: Feature extraction and feature fusion; Step 3: Morphological operation; Step 4: Finding the target contour; Step 5: Target search; Step 6: Error compensation for multiple detection and missed detection in detection; Step 7: Output the target according to the judgment of detection and compensation; The target search in step 5 comprises the following steps: Input the total number of aircrafts to be detected num; Select the detection function according to the number; Find the center point centerPoint[i] of each rectangular frame of the above contour, where i is the current contour, and set up a vector space to store centerPoint; Find the center point coordinates point(x[i], y[i]) and store them; Record the average height of each contour center and calculate the average height avH of all target frames; Find the maximum and minimum y coordinates of the target frame: max.y = max{centerPoint[i].y} min.y = min{centerPoint[i].y}; Determine whether the center point is located above or below the overall image, that is, determine the size of avH and the image height rows / 2, greater than the lower, otherwise the upper; In step 6, determine whether error compensation is needed according to the center point position and target frame data; In step 6, first determine the comparison between the number of target frames and the number of aircrafts, When the number of target frames is equal to the number of unmanned aerial vehicles: Determine whether the target frame position centerPoint[i].y is close, that is, the difference is less than a certain threshold; Determine whether the formation meets the input, that is, three aircrafts or two aircrafts exist with the same slope, depending on the total number of aircrafts in turn; When the above two requirements are met, output all targets; When the number of target frames is less than the number of unmanned aerial vehicles, that is, there is suspected missed detection: Count the number of target frames; When the center point coordinates point(x[i], y[i]) in the target frame and the image boundary (a, 0) (0, b) (1980, b) (a, 1080), a∈(0, 1980) b∈(0, 1080) are compared, when less than a certain threshold, it is judged that part of the aircraft is not in the field of view due to the boundary, and the existing target is output, the boundary value can vary with the image size; When greater than the threshold, count each target frame and determine whether the surrounding target frame meets the requirements, that is, according to the formation of unmanned aerial vehicles, the number of aircrafts in the same slope in three directions is the same, and it is judged that a position is missing a target, that is, a new target frame is supplemented in the missing position; When the number of target frames is greater than the number of unmanned aerial vehicles, that is., there is multiple detection: The target frame position centerPoint[i].y is less than a certain threshold, the target frames next to each other are merged as the same target, the center point coordinates point(x[i], y[i]) around each target frame are counted, the target frames with close coordinates that meet the requirements are retained, and the suspected target frames far from the target group are removed.

2. The feature fusion-based high dynamic multi-target UAV detection method of claim 1, wherein: In step 1, the image information is collected in real time by the low-altitude security system, and the image size is 1920*1080.

3. The feature fusion-based high dynamic multi-target UAV detection method of claim 1, wherein: The feature extraction in Step 2 includes color feature extraction and texture feature extraction.

4. The feature fusion-based high dynamic multi-target UAV detection method of claim 3, wherein: The color feature extraction includes the following steps: The image is median filtered; a one-dimensional sequence f1, f2, …, f n is taken with window length m, m being odd, and the expression for median filtering is as follows: Color space conversion, converting the input RGB image to YCrCb; Splitting the channel storing the color, extracting the red component about the UAV and saving it separately, and setting the parameter threshold.

5. The feature fusion-based high dynamic multi-target UAV detection method of claim 4, wherein: The texture feature extraction adopts the LBP method, and the specific steps are as follows: First, divide the window into 16*16 cells, for each pixel in each cell, compare the gray values of the adjacent 8 pixels with it, if the surrounding pixel value is greater than the center pixel value, mark it as 1, otherwise as 0, 8 points in 3*3 neighborhood can generate 8-bit binary number, that is, the LBP value of the center pixel point of the window can be obtained; Calculate the histogram of each cell, that is, the frequency of each number, and normalize the histogram; Finally, connect the statistical histogram of each cell to become a feature vector, which is the texture feature vector of the whole image.

6. The feature fusion-based high dynamic multi-target UAV detection method of claim 5, wherein: The feature fusion in Step 2 includes the following steps: Gaussian normalization method for feature normalization; Based on the formation of multiple UAVs flying in the air, as a way to compensate for errors in the UAV detection process.

7. The feature fusion-based high dynamic multi-target UAV detection method of claim 6, wherein: The morphological operation in Step 3 includes the following steps: First, set the convolution kernel B of dilation and erosion, adjust the parameter value based on the effect of image processing, convolve the kernel B with the image, and calculate the local maximum and minimum value of the covered area; First dilate and then erode the extracted red region; Output image.

8. The feature fusion-based high dynamic multi-target UAV detection method of claim 7, wherein: The target contour finding in Step 4 includes the following steps: Set the contour number space; Set the contour index space; Use the findContours function in opencv to find all contours; Draw the boundary contour.

Images