Small target detection method based on sea-sky-line

Abstract

The invention discloses a sea-sky-line-based small target detection method, which comprises the following steps of: firstly, establishing a region complexity model based on single-pixel complexity, region gray scale and image signal-to-noise ratio, then segmenting an image according to scene complexity, then sequentially carrying out region complexity statistics and space suppression, and according to the region complexity and a space suppression result, carrying out small target detection on the sea-sky-line-based small target. And carrying out region division on the image according to a set threshold value, obtaining a binary image, carrying out feature extraction, and finally carrying out Topat-based small target extraction. According to the method, the image is segmented according to scene complexity, and no target is extracted in a complex area, so that the purpose of removing false alarms is achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and infrared image processing, and in particular relates to a small target detection method based on sea-sky line. Background Art

[0002] In the field of image processing and computer vision, the generalization of algorithms is constrained by the scene. In different scenes, various image algorithms often have difficulty in achieving good results. For example, detection tasks will produce a large number of false alarm targets. These false alarm targets not only increase the difficulty of subsequent processing, but may also interfere with the final target recognition results. Therefore, scene recognition has become one of the important preprocessing links for subsequent image tasks.

[0003] In order to solve the above problems, there have been a lot of research in recent years. According to the output results, the methods can be divided into two categories. The first category is the method that directly outputs the scene category, such as the scene classification method based on the image features of traditional image signals and support vector machines, and the image classification model based on deep learning PLACECNN. The second category is the segmentation method that divides the input scene into regional categories, such as K-means clustering based on clustering methods, and semantic segmentation methods based on deep learning. Both methods have their shortcomings: the method that directly outputs the image type allows the entire algorithm framework to add more corresponding strategies to adapt to the actual scene, but it is difficult to add effective hard constraints to the output results of the target detection and target segmentation methods; the method that outputs the scene segmentation area can express the category of the pixel in more detail, which is convenient for direct screening of subsequent visual tasks such as detection tasks, but it brings a larger amount of calculation and requires stronger and more robust feature expression.

[0004] Whether it is a scene segmentation method based on support vector machines or deep learning, it requires data for training, which is called a data-driven method. Such methods will bring huge amounts of computation and are difficult to deploy on edge devices such as embedded systems and FPGAs. Traditional image feature statistics rely on prior knowledge and require more strategies to adapt them to the scene. Summary of the invention

[0005] Aiming at the problem of high false alarm rate of small target detection and segmentation on the sea surface, the present invention proposes a small target detection method based on sea-sky line, which assists small target detection with the help of scene segmentation of sea-sky background.

[0006] In order to achieve the above object, the technical solution adopted by the present invention to solve the technical problem is: a small target detection method based on sea-sky line, comprising the following steps: S1, regional complexity modeling: Establish a regional complexity model RC=F(C,RG,SNR) based on single pixel complexity C, regional grayscale RG and image signal-to-noise ratio SNR, where single pixel complexity C is used to measure the difference between pixels and their neighbors, regional grayscale RG is used to reflect the brightness characteristics of the region, and image signal-to-noise ratio SNR is used to reflect the ratio of noise to signal in the image; S2, introducing sea-sky-line attitude coordinates into the image, and segmenting the image into a complex area and a flat area according to the complexity of the scene, wherein the complex area includes a complex cloud area and a complex sea area, and the flat area includes a flat sky background area and a flat sea area, so as to achieve refined segmentation of the image scene; S3, use the overlapping sliding window strategy to traverse the image and calculate the regional complexity in each window; eliminate the brightness effect of the target on the image through the opening operation of the large window, and only retain the grayscale mean of the background. Then use the image pyramid idea to perform morphological filtering on the downsampled image to obtain the background mean and perform normalization processing. Take the sea-sky line attitude coordinates as the dividing line, and calculate the standard deviation of the sea surface and the sky as the stretching strength to complete the spatial suppression; according to the regional complexity and spatial suppression results, divide the image into regions according to the set threshold to obtain a binary image; S4, performing regional connection and isolated point elimination processing on the divided binary image to obtain the final scene segmentation result; S5, small object detection based on Tophat S51, performing sea-sky-line calibration on the input image, and then performing TopHat transformation preprocessing and image scaling respectively; S52, performing threshold segmentation and overlapping windowing to extract regions on the image preprocessed by TopHat in step S51; S53, performing regional statistics on the image obtained by overlapping the windowing regions in step S52; S54, performing an opening operation and normalization on the image scaled in step S51 in sequence, and then performing an Ä operation (spatial dot multiplication) with the image after regional statistics, and then performing threshold segmentation and morphological filtering in sequence, and then performing regional suppression with the image after threshold segmentation in step S52; S55, find the minimum bounding box and output the target information.

[0007] Furthermore, the complex area and the flat area are respectively regarded as the highlight area and the low-brightness area. According to the significant difference in grayscale between the highlight area and the low-brightness area, the brightness of the downsampled image area is used as the spatial suppression item to give the highlight area a higher complexity.

[0008] Furthermore, the image signal-to-noise ratio (SNR) is used to process the influence of the signal-to-noise ratio through prior knowledge.

[0009] Furthermore, in step S3, the average of multiple calculations is used to complete regional complexity statistics for the overlapping area.

[0010] Furthermore, two types of operations can be performed on the region suppression part in step S54. One is to set the region to zero to eliminate targets in complex areas, and the other is to suppress target weights to suppress targets with lower importance, thereby achieving re-screening of target detection.

[0011] The beneficial effects of the present invention are as follows: the present invention introduces sea-sky-line attitude coordinates into an image, divides the image into a complex area and a flat area according to the complexity of the scene, and realizes fine segmentation of the image scene; adopts different processing strategies according to the characteristics of each sub-area to reduce false alarm targets in the target detection process; divides the image according to the complexity of the scene to achieve the purpose of removing false alarms, and assists in small target detection through scene segmentation; and is applied to a small target detection method based on Tophat to achieve feature reuse, thereby effectively reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a scene segmentation framework diagram of the present invention; Figure 2 A comparison diagram of two window drawing methods of the present invention; Figure 3 It is a flow chart of the small target detection method based on Tophat of the present invention. DETAILED DESCRIPTION

[0013] The present invention is further described below with reference to the accompanying drawings and embodiments.

[0014] The method of the present invention is to segment the image according to the scene complexity and not extract the target in the complex area to achieve the purpose of eliminating false alarms. To this end, it is first necessary to clarify the definition and characteristics of various scenes in the image, such as the high brightness of complex clouds, the texture complexity of complex sea surfaces, etc. At the same time, according to the task requirements, the image segmentation task is defined as a semantic segmentation task, and the scene segmentation process of region division-feature extraction-post-processing is determined, such as Figure 1 shown.

[0015] The invention discloses a small target detection method based on sea-sky line, which comprises the following steps.

[0016] S1, establish regional complexity model.

[0017] In order to quantify the regional complexity of an image, the present invention introduces three factors: single pixel complexity C, regional grayscale RG, and image signal-to-noise ratio SNR, and establishes a regional complexity model RC=F(C, RG, SNR). The single pixel complexity C is used to measure the difference between a pixel and its neighborhood; the regional grayscale RG reflects the brightness characteristics of the region; and the image signal-to-noise ratio SNR reflects the ratio of noise to signal in the image.

[0018] As for the image signal-to-noise ratio (SNR), since it is difficult to directly obtain the image signal-to-noise ratio, the present invention uses prior knowledge to deal with the influence of the signal-to-noise ratio. The role of the signal-to-noise ratio is mainly reflected in the threshold setting of scene segmentation and target extraction. Therefore, the present invention connects the signal-to-noise ratio interface to adjust the value.

[0019] S2, segments the image according to scene complexity.

[0020] The sea-sky-line attitude coordinates are introduced into the image, and the image is divided into complex areas and flat areas according to the scene complexity, and further subdivided into sub-areas such as complex cloud area, complex sea area, flat sky background area, and flat sea area, so as to achieve fine segmentation of the image scene. At the same time, different processing strategies are adopted according to the characteristics of each sub-area to reduce false alarm targets in the target detection process.

[0021] For the regional grayscale RG, considering the significant difference in grayscale between the highlight area (such as complex cloud area, bright band area on the sea surface) and the low-brightness area (such as gentle sky background area, gentle sea surface area), the present invention uses the downsampled image area brightness as the spatial suppression item to give the highlight area a higher complexity.

[0022] For the single pixel complexity C, the present invention uses the result of the Tophat algorithm to measure the difference between the pixel and the neighborhood. The Tophat algorithm can highlight the highlight area and details in the image, so that the scene segmentation can be coupled with the original target extraction, reducing repeated calculations.

[0023] According to the regional complexity model, a method combining regional complexity statistics and spatial suppression is used to segment the scene. The specific steps are as follows.

[0024] S3, regional complexity statistics: Use overlapping sliding window strategy to traverse the image, calculate the regional complexity in each window, and use the average of multiple calculations to complete the regional complexity statistics for overlapping areas. Figure 2 As shown in the figure, the problem of non-smooth segmentation results caused by hard segmentation can be alleviated and the hole phenomenon in the segmented area can be reduced. For overlapping areas, the average of multiple calculations is used to obtain the final regional complexity. Figure 2 The figure shows a comparison of two windowing methods, the left side is the hard windowing method, and the right side is the overlapping windowing method.

[0025] Spatial suppression: Since there are deficiencies in using only complexity features for region division, such as the possibility that a highlight area and a low-noise flat area may be mistakenly classified as the same situation, the present invention introduces a spatial suppression branch to solve this problem. The spatial suppression branch eliminates the brightness influence of the target on the image through a large window opening operation, retaining only the grayscale mean of the background, and then uses the image pyramid idea to perform morphological filtering on the downsampled image to obtain the background mean and perform normalization processing. During the normalization process, the present invention uses the sea-sky line attitude coordinates as the dividing line, and calculates the standard deviation of the sea surface and the sky as the stretching strength to establish a normalization calculation process.

[0026] According to the regional complexity and spatial suppression results, the image is divided into regions according to the set threshold to obtain a binary image.

[0027] S4, feature extraction. The divided binary image needs to be processed by region connection and isolated point elimination to obtain the final scene segmentation result.

[0028] S5, post-processing.

[0029] Small target detection based on Tophat: Since it is necessary to extract regional complexity, the present invention adopts the Tophat small target detection method, which can achieve good feature reuse and effectively reduce the amount of calculation.

[0030] This paper proposes a small target detection framework based on Tophat, such as Figure 3 As shown, since the present invention needs to extract regional complexity, it can achieve feature reuse well with the Tophat small target detection method, which can effectively reduce the amount of calculation. The specific steps are as follows.

[0031] S51, perform sea-sky-line calibration on the input image, and then perform TopHat transformation preprocessing and image scaling respectively.

[0032] S52, performing threshold segmentation and overlapping windowing to extract regions on the image pre-processed by TopHat in step S51.

[0033] S53, performing regional statistics on the image after the overlapping windowing and region extraction in step S52.

[0034] S54, the image scaled in step S51 is opened and normalized in sequence, and then spatial vector dot multiplication is performed with the image after regional statistics, followed by threshold segmentation and morphological filtering, and then regional suppression is performed with the image after threshold segmentation in step S52.

[0035] Two types of operations can be performed in the region suppression part. One is to set the region to zero to remove targets in complex areas, and the other is to suppress target weights to suppress targets with lower importance, thereby achieving re-screening of target detection.

[0036] S55, find the minimum bounding box and output the target information.

[0037] The weight of the small target detection part, the weight of the scene segmentation, and the calibration of the sea-sky line in the image coordinates can be manually intervened in the system through the host computer, or the relative threshold can be set to automatically close the loop.

[0038] At the same time, in order to better deploy on edge devices, except for the morphological filtering of the scene segmentation part before regional suppression, the other parts of the present invention uniformly use 7×9 size morphology to implement this process, which makes the operator optimization and implementation of edge devices have obvious advantages. After the threshold segmentation of the scene segmentation part, the morphological filtering needs to connect the segmentation point set and then delete the isolated noise points, so it is necessary to first morphologically expand and then cooperate with a larger size morphological corrosion operator. Here, the corrosion operator uses 7×9 and the expansion operator uses 5×5 size, which effectively connects the regions and removes isolated noise points.

[0039] The above embodiments are only illustrative of the principles and effects of the present invention, as well as some embodiments of its application. For those skilled in the art, several modifications and improvements may be made without departing from the creative concept of the present invention, and all of these belong to the protection scope of the present invention.

Claims

1. A small target detection method based on sea-sky line, characterized by: The following steps are included S1, establish a regional complexity model RC=F(C,RG,SNR) based on single pixel complexity C, regional grayscale RG and image signal-to-noise ratio SNR, where single pixel complexity C is used to measure the difference between pixels and their neighbors, regional grayscale RG is used to reflect the brightness characteristics of the region, and image signal-to-noise ratio SNR is used to reflect the ratio of noise to signal in the image; S2, introducing sea-sky-line attitude coordinates into the image, and dividing the image into a complex area and a flat area according to the complexity of the scene, wherein the complex area includes a complex cloud area and a complex sea area, and the flat area includes a flat sky background area and a flat sea area; S3, use the overlapping sliding window strategy to traverse the image and calculate the regional complexity in each window; eliminate the brightness effect of the target on the image through the opening operation of the large window, and only retain the grayscale mean of the background. Then use the image pyramid idea to perform morphological filtering on the downsampled image to obtain the background mean and perform normalization. Use the sea-sky line attitude coordinates as the dividing line, and calculate the standard deviation of the sea surface and the sky as the stretching strength to complete spatial suppression; divide the image into regions according to the regional complexity and spatial suppression results to obtain a binary image; S4, performing regional connection and isolated point elimination processing on the divided binary image to obtain the final scene segmentation result; S5, small object detection based on Tophat S51, performing sea-sky-line calibration on the input image, and then performing TopHat transformation and image scaling respectively; S52, performing threshold segmentation and overlapping windowing to extract regions on the image preprocessed by TopHat; S53, performing regional statistics on the image after overlapping windowing and taking regions; S54, performing an opening operation and normalization on the scaled image in sequence, and then performing a spatial vector dot multiplication with the image after regional statistics, and then performing threshold segmentation and morphological filtering in sequence, and then performing regional suppression with the image after threshold segmentation in step S52; S55, find the minimum bounding box and output the target information.

2. The small target detection method based on sea-sky line according to claim 1, characterized in that: The complex area and the flat area are regarded as the highlight area and the low-brightness area respectively. According to the difference in grayscale between the highlight area and the low-brightness area, the brightness of the downsampled image area is used as the spatial suppression item to give complexity to the highlight area.

3. The small target detection method based on sea-sky line according to claim 1, characterized in that: The image signal-to-noise ratio SNR processes the influence of the signal-to-noise ratio through prior knowledge.

4. A small target detection method based on sea-sky line according to claim 1, 2 or 3, characterized in that: In step S3, the average of multiple calculations is used to complete the regional complexity statistics for the overlapping area.

5. The small target detection method based on sea-sky line according to claim 4, characterized in that: In the step S54, the target in the complex area is eliminated by setting the area to zero for regional suppression, or the target with lower importance is suppressed by suppressing the target weight, so as to achieve the re-screening of target detection.

Images