Anti-interference water surface small target detection method and system based on suppression convolution kernel filtering

Abstract

The invention discloses an anti-interference water surface small target detection method and system based on suppression of convolution kernel filtering. The method comprises the following steps: 1) obtaining a to-be-detected water surface image shot by an unmanned aerial vehicle; 2) image preprocessing; 3) performing edge extraction by using an edge detection method; 4) estimating a target imaging pixel upper limit S * S required to be detected according to the physical size of the small target, the working height of the unmanned aerial vehicle and the internal reference of the camera; 5) performing single filtering on the image after edge extraction by using a suppression convolution kernel, and performing filtering suppression on an interference part; the suppression convolution kernel comprises four layers: a central point, an enhancement layer, a hollow layer and a suppression layer; and 6) carrying out contour extraction on the suppressed image to obtain a required small target detection target frame. According to the method, data set learning is not needed, and the real-time performance of water surface small target detection can be improved while the detection accuracy is considered.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to an anti-interference water surface small target detection method and system based on suppressed convolution kernel filtering. Background Art

[0002] In order to improve retrieval efficiency when UAVs are flying over water, they often need to perform large-scale scanning at high altitudes (greater than 300 meters). At this time, the pixels of targets on the water surface will become very small (less than 20*20 pixels), making them difficult to detect.

[0003] Small targets are difficult to classify because of the small number of pixels. At the same time, in scanning tasks, detection can be performed first and then the drone can further focus and enlarge the image or lower the altitude for further category confirmation (i.e., a two-stage method). Therefore, the detection method can perform detection only without target recognition.

[0004] The difficulty of detecting small targets on the water surface is not only that the targets are small and difficult to detect, but also that there is a lack of data sets and they are easily interfered by water waves and land background. Because of the lack of data on small targets on the water surface, and the inability of data sets to cover all categories of surface targets in a simple detection task, and the small number of target pixels makes it difficult to extract high-level features for learning, methods based on deep learning are no longer applicable, and an anti-interference method for detecting small targets on the water surface is needed. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an anti-interference surface small target detection method based on suppressed convolution kernel filtering in view of the defects in the prior art.

[0006] The technical solution adopted by the present invention to solve the technical problem is: an anti-interference water surface small target detection method based on suppressed convolution kernel filtering, comprising the following steps:

[0007] 1) Obtain the water surface image to be detected taken by the drone;

[0008] 2) Image preprocessing; the preprocessing includes grayscale conversion and mean filtering;

[0009] 3) Use edge detection method to extract edges;

[0010] 4) Estimate the upper limit S*S of the target imaging pixel to be detected based on the physical size of the small target, the operating altitude of the drone, and the camera internal parameters;

[0011] 5) The image after edge extraction is filtered once with a suppression convolution kernel to suppress the interference part;

[0012] The suppression convolution kernel includes four layers: center point, enhancement layer, hollow layer and suppression layer;

[0013] 6) Perform contour extraction on the suppressed image to obtain the required small target detection target frame.

[0014] According to the above scheme, in step 3), after edge extraction using the edge detection method, the following processing is also included:

[0015] The edge extraction result is mean filtered, and then the inside of the edge is filled using a morphological closing operation.

[0016] According to the above scheme, in step 3), the morphological closing operation selects a rectangular structural element, and the number of iterations is 2, that is, two expansions and two erosions are performed.

[0017] According to the above scheme, in step 3), the size of the closing operation structure element is selected to be an odd number greater than or equal to S-1 or S-2.

[0018] According to the above scheme, in step 4), the convolution kernel structure is designed as follows:

[0019] The suppression convolution kernel consists of four layers: center point, enhancement layer, hollow layer and suppression layer;

[0020] The center point and enhancement layer are 3*3 matrices, where the center point is the center of the matrix and the enhancement layer is the 8-neighborhood of the center point;

[0021] The number of hollow layers m=S / 2 is set according to the upper limit S*S of the target imaging pixel. If S / 2 is not an integer, the value of m is rounded up to an integer.

[0022] The number of suppression layers is set according to the number of hollow layers, 3*m.

[0023] According to the above scheme, in step 5), the center point parameter is set to 2.5, the enhancement layer parameter is set to 0.8, the hollow layer parameter is set to 0, and the suppression layer initial parameter is set to -0.025.

[0024] According to the above scheme, in step 6), the contour of the suppressed image is extracted to obtain the required small target detection target frame, which is as follows:

[0025] The result after the suppression convolution kernel filtering is mean filtered and binarized, and the binarized result is sent to the contour detector, and then the rectangular outer frame of the detected contour is extracted, which is the candidate target box.

[0026] The present invention also relates to an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method described in any one of the above-mentioned schemes.

[0027] The present invention also relates to a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the methods described above when executed by a processor.

[0028] An anti-interference water surface small target detection system based on suppressed convolution kernel filtering, comprising:

[0029] An image acquisition module, used to obtain the water surface image to be detected taken by the drone;

[0030] An image preprocessing module is used to preprocess the collected image, including grayscale conversion and mean filtering;

[0031] An edge extraction module is used to extract edges from the preprocessed image using an edge detection method;

[0032] The suppression convolution kernel filter module is used to perform a single filter on the image after edge extraction using the convolution kernel to filter and suppress the interference part; the suppression convolution kernel includes four layers: a center point, an enhancement layer, a hollow layer and a suppression layer;

[0033] The contour extraction module is used to extract the contour of the suppressed image to obtain the required small target detection target frame.

[0034] The beneficial effects produced by the present invention are:

[0035] 1. The method of the present invention does not require a data set for learning and can perform real-time detection;

[0036] 2. The present invention adopts a suppressive convolution kernel for filtering, and the convolution kernel parameters can be adjusted according to the upper threshold limit of the small target pixels to be detected to improve the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0038] Figure 1 is a method principle diagram of an embodiment of the present invention;

[0039] Figure 2 is a structural diagram of a suppression convolution kernel according to an embodiment of the present invention;

[0040] Figure 3 It is a three-dimensional display diagram of the suppressed convolution kernel according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] like Figure 1 As shown, a method for detecting small surface targets with anti-interference based on suppression convolution kernel filtering includes the following steps:

[0043] 1) Obtain the water surface image to be detected taken by the drone;

[0044] By taking a bird's-eye view of a small target on the water surface, an image of the water surface to be detected taken by the drone is obtained. This method has no strict requirements on the frame size of the image to be detected and the actual physical size of the target;

[0045] 2) Image preprocessing: Preprocessing, including grayscale and mean filtering;

[0046] To meet the real-time computing requirements, the image extracted from the original video stream of the drone is downsampled to 768*432 pixels as the input of the detection module. Then, the downsampled image is preprocessed, including grayscale conversion and using 3*3 mean filtering to remove some too fine edges.

[0047] 3) Use edge detection method to extract edges;

[0048] The classic Sobel operator edge detection method is used for edge extraction.

[0049] When using the Sobel operator for edge extraction, the gradient edges in the x and y directions are calculated for the preprocessed image, and the absolute values ​​are taken first, then normalized, and finally the normalized results are added to obtain the Sobel edge extraction result. Since the Sobel edge extraction result is obtained by adding the normalized results in the x and y directions, there are a lot of parts that are higher than 1. At this time, no normalization is performed again, but the parts greater than 1 are set to 1, so that more edges can be retained.

[0050] The edge extraction result is subjected to mean filtering. The convolution kernel size of the mean filter is usually selected as 3*3 or 5*5, which can appropriately reduce the edge strength and facilitate subsequent suppression operations. Then, the morphological closing operation is used, and the rectangular structural element is selected. The number of iterations is 2, that is, two dilations and two erosions are performed.

[0051] By performing mean filtering and morphological closing operations on the edge extraction results, the water surface targets can be retained and filled, and the layered river banks and shimmering highlight areas can also be filled, which is convenient for subsequent filtering and suppression using the suppression convolution kernel.

[0052] 4) The upper limit of the target imaging pixel to be detected is estimated based on the physical size of the small target, the operating altitude of the UAV, and the camera internal parameters;

[0053] Under normal circumstances, the default target upper limit parameter of 20*20 pixels (i.e. the common definition of the absolute scale of small targets in authoritative datasets such as COCO) can be used;

[0054] In this embodiment, the upper limit of the target imaging pixel to be detected is calculated according to the physical size of the target in actual use, the operating altitude of the drone, and the camera internal parameters, and the default parameters are adjusted;

[0055] 5) Design a suppression convolution kernel, use the designed convolution kernel to perform a single filter on the image after edge extraction, and suppress the interference part of the filter;

[0056] The convolution kernel structure is as follows:

[0057] like Figure 2 ,The suppression convolution kernel contains four layers: the center point, the enhancement layer, the hollow layer and the suppression layer;

[0058] The center point and enhancement layer are 3*3 matrices, where the center point is the center of the matrix and the enhancement layer is the 8-neighborhood of the center point;

[0059] The number of hollow layers m=S / 2 is set according to the upper limit S*S of the target imaging pixel. If S / 2 is not an integer, the value of m is rounded up to an integer.

[0060] In this embodiment, the number of hollow layers is set to 10. After including the hollow layer, the size of the entire convolution kernel becomes 23*23 (slightly larger than the target to be detected). Because the hollow layer does not suppress or suppresses very little, designing it as the target size threshold can prevent the target itself from suppressing itself.

[0061] Set the number of suppression layers 3*m according to the number of hollow layers;

[0062] The number of suppression layers is three times that of hollow layers, which is neither too large to cause excessive suppression nor too small to cause insufficient suppression;

[0063] Finally, the convolution kernel is designed as a 30-10-1-1-1-10-30 structure with a size of 83*83.

[0064] The detailed design of parameters in this embodiment is as follows:

[0065] Assume that the target size threshold to be detected is S*S, and the parameters of the suppression convolution kernel are designed as follows:

[0066] Center point layer number: 1;

[0067] Number of enhancement layers: 1;

[0068] Number of hollow layers: S / 2;

[0069] Number of suppression layers: 3*S / 2;

[0070] Mean filter size: 3*3 or 5*5;

[0071] Center point parameters: 2.5*S / 20;

[0072] Enhancement layer parameters: 0.8*S / 20;

[0073] Hollow layer parameters: 0 or -0.025 / S*20 / 3;

[0074] Initial parameters of the suppression layer: -0.025 / S*20;

[0075] Figure 3 This is a three-dimensional display of the suppressed convolution kernel. It shows the parameter details of different layers. The number of layers and values ​​of each layer of the convolution kernel can be adjusted. The core follows the four-layer design principle: the center point and enhancement layer are used to enhance the point, while the hollow and suppression layers use the surrounding pixels to suppress the point.

[0076] 6) Contour extraction to obtain the frame: Contour extraction is performed on the suppressed image to obtain the required target frame;

[0077] The method provided in this embodiment is as follows: the result after the suppression convolution kernel filtering is subjected to 3*3 mean filtering, and binarized with a threshold of 255*0.3=76.5. Among them, the mean filtering is to prevent too many interruptions during binarization; the result after binarization is usually a scattered highlight group, each highlight group represents that there may be a target here, that is, a candidate target. The binarized result is sent to the opencv contour detector (other contour detectors can also be selected), and then the rectangular outer frame of the detected contour is extracted, which is the candidate target frame.

[0078] Verification experiment:

[0079] On a 768*432 image, the default suppression convolution kernel with a target size threshold of 20*20 and the corresponding parameters were used to detect real-life videos, including both high-altitude scanning perspectives and perspectives when the drone zoomed in. Each video included digital buoys that needed to be identified, and some videos also included ships. The detection effect was good.

[0080] The detection results show that under different viewing angles, different heights and different interference environments, the present invention can detect small targets relatively stably, and achieve the purpose of detecting small targets on the water surface while resisting interference.

[0081] At the same time, we use real-life video to conduct quantitative experiments. The video mainly includes three types of surface targets: buoys, stationary ships, and moving ships. Due to the distance relationship, ship type, and the influence of wake waves, the size and shape of moving ships and stationary ships are very different. Therefore, we consider two situations: one is to use buoys and stationary ships as the targets to be detected, and the other is to use all three types of targets as targets to be detected.

[0082] When only two types of targets are required to be detected, the moving ships are regarded as interference. After observing and calculating the sizes of various targets, the suppression convolution kernel of 14*14 target threshold is selected for experiment. The experimental results show that in a total of 772 frames, 770 frames detected two types of targets with an accuracy of 1 and a recall of 1; 2 frames detected one type of target with an accuracy of 1 and a recall of 50%; the statistics of all frame detection results show that the average accuracy is 1 and the average recall is 99.87%.

[0083] When all three types of targets are required to be detected, considering the large scale of the moving ship, a suppression convolution kernel with a target threshold of 32*32 is used for the experiment. The experimental statistical results are shown in Table 1. The average accuracy is 95.59% and the average recall is 1 by counting the detection results of all frames.

[0084] Table 1 Statistics of detection results of three types of targets in real-life videos

[0085] Test results Frame rate Accuracy Recall Three goals 650 1 1 Four goals 99 0.75 1 Five goals 22 0.6 1 Six goals 1 0.5 1

[0086] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A method for detecting small targets on the water surface with anti-interference based on suppression convolution kernel filtering, characterized in that: The following steps are involved: 1) Obtain the water surface image to be detected taken by the drone; 2) Image preprocessing; 3) Use edge detection method to extract edges; 4) Estimate the upper limit S*S of the target imaging pixel to be detected based on the physical size of the small target, the operating altitude of the drone, and the camera internal parameters; 5) The image after edge extraction is filtered once with a suppression convolution kernel to suppress the interference part; The suppression convolution kernel includes four layers: center point, enhancement layer, hollow layer and suppression layer; 6) Perform contour extraction on the suppressed image to obtain the required small target detection target frame.

2. The anti-interference water surface small target detection method based on suppressing convolution kernel filtering according to claim 1 is characterized in that: In the step 3), after edge extraction is performed using an edge detection method, the following processing is also included: The edge extraction result is mean filtered, and then the inside of the edge is filled using a morphological closing operation.

3. The anti-interference water surface small target detection method based on suppressing convolution kernel filtering according to claim 2 is characterized in that: In the step 3), the morphological closing operation selects a rectangular structure element, and the number of iterations is 2, that is, two dilations and two erosions are performed.

4. The anti-interference water surface small target detection method based on suppressing convolution kernel filtering according to claim 3 is characterized in that: In the step 3), the size of the closing operation structure element is selected to be an odd number greater than or equal to S-1 or S-2.

5. According to the anti-interference water surface small target detection method based on suppression convolution kernel filtering according to claim 1, in the step 4), the convolution kernel structure is designed as follows: The suppression convolution kernel consists of four layers: center point, enhancement layer, hollow layer and suppression layer; The center point and enhancement layer are 3*3 matrices, where the center point is the center of the matrix and the enhancement layer is the 8-neighborhood of the center point; The number of hollow layers m=S / 2 is set according to the upper limit S*S of the target imaging pixel. If S / 2 is not an integer, the value of m is rounded up to an integer. The number of suppression layers is set according to the number of hollow layers, 3*m.

6. According to the anti-interference surface small target detection method based on suppressed convolution kernel filtering according to claim 1, in the step 5), the center point parameter is set to 2.5, the enhancement layer parameter is set to 0.8, the hollow layer parameter is set to 0, and the suppression layer initial parameter is set to -0.

025.

7. According to the anti-interference water surface small target detection method based on suppression convolution kernel filtering according to claim 1, in the step 6), the contour of the suppressed image is extracted to obtain the required small target detection target frame, which is specifically as follows: The result after the suppression convolution kernel filtering is mean filtered and binarized, and the binarized result is sent to the contour detector, and then the rectangular outer frame of the detected contour is extracted, which is the candidate target box.

8. An anti-interference water surface small target detection system based on suppressed convolution kernel filtering, characterized in that: include: An image acquisition module, used to obtain the water surface image to be detected taken by the drone; An image preprocessing module is used to preprocess the collected image, including grayscale conversion and mean filtering; An edge extraction module is used to extract edges from the preprocessed image using an edge detection method; The suppression convolution kernel filter module is used to perform a single filter on the image after edge extraction using the convolution kernel to filter and suppress the interference part; the suppression convolution kernel includes four layers: a center point, an enhancement layer, a hollow layer and a suppression layer; The contour extraction module is used to extract the contour of the suppressed image to obtain the required small target detection target frame.

9. An electronic device, characterized in that: include: one or more processors; as well as a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images