Weak small ship target fusion detection method and device based on multi-visual salient features

Abstract

The present invention provides a method and device for fusion detection of weak and small ship targets based on multi-visual salient features, comprising the following steps: performing land and sea segmentation, cloud removal and morphological filtering on the sea area sequence images acquired by the optical remote sensing staring satellite to obtain the sea area near-infrared image; based on the sea area near-infrared image, obtaining the target grayscale feature saliency map through the mean shift algorithm; based on the sea area near-infrared image, obtaining the target point distribution saliency map through the human visual system model; using the target point distribution saliency map as a guide, performing image segmentation based on region growth on the grayscale feature saliency map, and extracting the ship targets in the sea area near-infrared image in combination with the ship target size features. The method solves the problem that the weak and small target detection method in the prior art has a high missed alarm rate and false alarm rate, low robustness and accuracy, and is not suitable for weak and small multi-target detection under low signal-to-noise ratio and large sea clutter conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and a device for fusion detection of weak and small ship targets based on multi-visual significant features. Background Art

[0002] Marine monitoring requires that the monitoring means can meet the monitoring capabilities of the overall distribution and motion state perception of marine targets. Optical remote sensing satellites have broken through the limitations of traditional marine, land, air and ocean monitoring means, and have achieved continuous and real-time monitoring of targets in a large range of oceans and land, which is of great significance to traffic control and safety assurance. my country's geostationary orbit optical remote sensing satellite Gaofen-4 (GF-4) can obtain sea images with a resolution of 50m and a width of 500km through staring imaging, which can meet the monitoring of large marine ship targets. According to the definition of the Society of Photo-optical Instrumentation Engineers (SPIE), in an image of 256×256 pixels, a low-contrast target with an imaging size of 0.1% of the total pixels (i.e., less than 81 pixels) is called a weak target. In GF-4 satellite images, most ship targets only occupy a few pixels, and are greatly disturbed by sea clutter and clouds, and have a low signal-to-noise ratio, so they are weak targets. In addition, ships are widely and unevenly distributed at sea, especially in the waters near the coast, where there are ship targets of various types and sizes. These factors make it very difficult to detect such weak targets. Weak target detection methods are mainly divided into detection algorithms based on filtering, detection algorithms based on image data structures, intelligent detection algorithms based on deep learning, and detection algorithms based on the human visual system (HVS). Among them:

[0003] Filtering algorithms are divided into two categories: spatial domain filtering algorithms and transform domain filtering algorithms. However, since small targets have less feature information such as texture and size, the difference between the target and the background is the main feature used in target detection. The role of filtering is to highlight small targets and suppress background noise. Fig.10 (b) Yes Fig.10 (a) A magnified image of the near-infrared band of the selected area. The white bright spots are some of the ship targets in the ocean image taken by the GF-4 optical remote sensing satellite. It can be seen from the figure that the ship targets are greatly affected by the land and shallow sea areas. Therefore, for the image processed by the traditional spatial domain filtering algorithm, there is still a lot of background noise and it cannot be directly segmented.

[0004] The detection algorithm based on image structure uses the sparsity of the target and the low rank of the background to achieve target detection. However, this type of method requires many iterations of matrix decomposition. Fig.10For the image shown, the efficiency of high-dimensional singular value decomposition is low, and the iteration is not easy to converge, so it will result in a long calculation time.

[0005] When deep learning methods are used for target detection, the weights must first be trained through a training set. In order to enhance the robustness of target detection, a sufficiently large training set is required. Since weak targets have less feature information, the data set is complex to create, which also affects the convergence time of deep learning methods.

[0006] The target detection method based on the HVS model is a new target detection method that has emerged in recent years. It uses the visual attention mechanism to generate saliency maps of features such as size, shape, grayscale and contrast, and realizes weak target detection through feature comparison and fusion methods; however, the existing algorithm has limited ability to suppress background noise and is not robust, so it is not suitable for detecting multiple weak targets in complex backgrounds.

[0007] In addition, for the detection of multiple weak and small ship targets, the background is often complex and the clutter interference is large; the existing detection methods cannot meet the detection accuracy and robustness requirements. Summary of the invention

[0008] In view of the above analysis, the present invention aims to provide a method and device for fusion detection of weak and small ship targets based on multi-visual significant features, which is used to solve the problems that the weak and small ship target detection method in the prior art has a high missed alarm rate and false alarm rate, and low robustness and accuracy, and is not suitable for weak and small multi-target detection under low signal-to-noise ratio and large sea clutter conditions.

[0009] The purpose of the present invention is mainly achieved through the following technical solutions:

[0010] On the one hand, the present invention provides a method and device for fusion detection of weak and small ship targets based on multi-visual salient features, comprising the following steps:

[0011] The sea area sequence images acquired by the optical remote sensing staring satellite are subjected to sea-land segmentation, cloud removal and morphological filtering to obtain the sea area near-infrared image;

[0012] Based on the near-infrared image of the sea area, a target grayscale feature saliency map is obtained by using a mean shift algorithm;

[0013] Based on the near-infrared image of the sea area, a saliency map of target point distribution is obtained through a human visual system model method;

[0014] Using the target point distribution saliency map as a guide, the grayscale feature saliency map is subjected to image segmentation based on region growing, and the ship targets existing in the near-infrared image of the sea area are extracted in combination with the ship target size features.

[0015] Furthermore, the target point distribution saliency map is obtained through the human visual system model, including: obtaining the contrast feature map of the sea area near-infrared image through a lateral inhibition network algorithm, and thresholding the contrast feature map to obtain the target point distribution saliency map.

[0016] The method of obtaining the contrast feature map of the near-infrared image of the sea area through the side inhibition network algorithm includes: performing a convolution operation on the near-infrared image of the sea area through the coefficient matrix of the side inhibition network, taking the absolute value of the grayscale of the pixel after the convolution process, and obtaining the contrast feature map.

[0017] Furthermore, thresholding the contrast feature map to obtain the target point distribution saliency map includes: setting a threshold based on the grayscale value distribution of the ship target in the contrast feature map, binarizing the contrast feature map with the grayscale threshold, and obtaining the target point distribution saliency map.

[0018] The lateral inhibition coefficient is in the following matrix form:

[0019]

[0020] Furthermore, the sea-land segmentation, cloud removal and morphological filtering of the sea area sequence images acquired by the optical remote sensing staring satellite include:

[0021] The NDWI method is used to perform threshold segmentation on the sea area sequence images to obtain a rough ocean and cloud-land segmentation binary map;

[0022] According to the size characteristics of the ship target, cloud layer, and land, the sliding window size is determined, and the rough ocean and cloud-land segmentation binary map is traversed using the sliding window, and the area where the proportion of zero values ​​in the sliding window is less than the threshold is determined as a water area, so as to obtain an image of the sea area after the ocean and cloud-land segmentation;

[0023] A near-infrared band map of the sea area sequence image is multiplied by the sea surface area image to obtain a sea area near-infrared image.

[0024] Furthermore, the mean shift algorithm includes: selecting a pixel point to be processed in the near-infrared image of the sea area in turn, setting a sample area with the pixel point to be processed as the center; setting a joint domain weight kernel function of space and pixel gray value, performing Mean Shift iteration in the sample area using the joint domain weight kernel function of space and pixel gray value, and assigning the gray value of the sample center point after the iteration is terminated to the pixel point to be processed;

[0025] Based on the mean shift algorithm, all pixel points of the near-infrared image of the sea area are traversed to obtain a target grayscale feature saliency map.

[0026] Furthermore, the extraction of the ship target in the near-infrared image of the sea area includes: setting a gray threshold T cluster , taking the target point in the target point distribution saliency map as the guide point, and taking the guide point in the four neighborhoods with grayscale differences less than the threshold T cluster The points are classified into one category, and the grayscale values ​​of all pixels of this category are assigned to the mean grayscale values ​​of all pixels of this category; different target points are divided into different categories A i Based on the size parameters of the ship target in the near infrared image of the sea area, from A i The target points that meet the ship size characteristics are screened out, which are the ship targets existing in the near-infrared image of the sea area.

[0027] When performing the image segmentation based on region growing, the threshold of the grayscale difference between the four neighborhoods of the guide point and the guide point is set to T cluster =5.

[0028] On the other hand, a weak small ship target detection device is also disclosed, characterized in that it includes at least one processor and at least one memory communicatively connected to the processor;

[0029] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the aforementioned weak and small ship target fusion detection method based on multiple visual significant features.

[0030] Beneficial effects of this technical solution:

[0031] 1. After performing sea-land segmentation, removing cloud-land interference, and morphological filtering, the ship target detection method of the present invention uses the mean shift method to filter out the sea surface background clutter and obtain the target grayscale saliency map, so that the method can still accurately identify weak ship targets when the sea clutter is large, solving the problem that the existing algorithm has limited ability to suppress background noise.

[0032] 2. The present invention obtains a target contrast feature map based on a human visual system model method, and obtains a target point distribution saliency map by thresholding; thus, the purpose of detecting multiple weak targets is well achieved.

[0033] 3. The present invention reduces the false alarm rate and missed alarm rate by fusing the grayscale features, contrast features and size features of small targets, preserves accurate target position information, and has high robustness and detection accuracy.

[0034] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0036] Figure 1 The flowchart of the detection method based on mean shift and visual saliency according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the small and weak ship target fusion detection method based on multi-visual salient features.

[0038] Figure 3 is a lateral inhibition network model; where Figure 3 (a) is the non-circulatory side inhibition network model, and (b) is the circulatory side inhibition network model.

[0039] Figure 4 is a graph in the image preprocessing process of an embodiment of the present invention; wherein, Figure 4 (a) is the image after NWDI processing. Figure 4 (b) is the image after land and sea segmentation. Figure 4 (c) is a binary image after filtering out ships and broken cloud patches. Figure 4 (d) is the water part after removing land and clouds.

[0040] Figure 5 is an extracted image of the target grayscale saliency map of an embodiment of the present invention; wherein, Figure 5 (a) is the water area image in the near-infrared band. Figure 5 (b) is the grayscale feature saliency map after MS processing.

[0041] Figure 6 is an image in the process of extracting the target point distribution saliency map of the embodiment of the present invention; wherein, Figure 6 (a) is the water area image in the near-infrared band. Figure 6 (b) is the contrast feature map, Figure 6 (c) OLI The three-dimensional grayscale histogram of Figure 6 (d) is the target point distribution saliency map B.

[0042] Figure 7 is the final target detection binary image of the embodiment of the present invention; wherein, Figure 7(a) is the water area image I1′ in the near-infrared band. Figure 7 (b) is the final target detection result O.

[0043] Figure 8 is a comparison diagram of the results of processing the image area I1′ by the detection algorithm of the embodiment of the present invention and five weak and small target detection algorithms; wherein, Figure 8 (a) is the water area image I1′ in the near-infrared band. Figure 8 (b) is the processing result of the adaptive threshold edge detection algorithm. Figure 8 (c) is the result of LCM algorithm processing. Figure 8 (d) is the result of CFAR algorithm processing. Figure 8 (e) is the result of GCF-SB algorithm processing. Figure 8 (f) is the processing result diagram of the algorithm in this paper.

[0044] Fig. 9 : is a comparison diagram of the results of the detection algorithm of the embodiment of the present invention and five small and weak target detection algorithms for processing the image area I1″; wherein, Fig. 9 (a) is the water area image in the near-infrared band Fig. 9 (b) is the processing result of the adaptive threshold edge detection algorithm. Fig. 9 (c) is the result of LCM algorithm processing. Fig. 9 (d) is the result of CFAR algorithm processing. Fig. 9 (e) is the result of GCF-SB algorithm processing. Fig. 9 (f) is the processing result diagram of the algorithm in this paper.

[0045] Fig.10 It is an ocean image taken by the GF-4 optical remote sensing satellite; among them, Fig.10 (a) is a multi-band image of the ocean. Fig.10 (b) Fig.10 Near-infrared band map of the selected area in (a). DETAILED DESCRIPTION

[0046] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the implementation cases of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0047] The present invention provides a method for fusion detection of weak small ship targets based on multi-visual salient features, such as Figure 1 As shown, the following steps are included:

[0048] The ocean area sequence images acquired by the optical remote sensing staring satellite are processed by sea-land segmentation, cloud removal and morphological filtering to obtain the ocean area near-infrared images.

[0049] Based on the near-infrared image of the sea area, a target grayscale feature saliency map is obtained by using a mean shift algorithm;

[0050] Based on the near-infrared image of the sea area, a saliency map of target point distribution is obtained through a human visual system model method;

[0051] Using the target point distribution saliency map as a guide, the grayscale feature saliency map is subjected to image segmentation based on region growing, and the ship targets existing in the near-infrared image of the sea area are extracted in combination with the target size features.

[0052] Since ships are small and weak targets in the images taken by GF-4 and are greatly disturbed by sea clutter and clouds, the existing target detection methods cannot meet the detection requirements for accuracy and robustness. Although the research on small and weak target detection has become more and more in-depth in recent years, there are still few methods for detecting small and weak targets such as ships. Most methods are only applicable to single ship target detection with weak sea clutter intensity. In order to improve the detection accuracy of multiple ship targets under complex sea clutter noise interference, the present invention proposes an optical image small and weak ship target detection method based on mean shift and visual saliency, such as Figure 2 As shown in the figure, it is mainly divided into four steps: image preprocessing, extraction of grayscale feature saliency map, extraction of target point distribution saliency map, and target detection by fusing grayscale, contrast, and size feature saliency maps. First, the Normalized Difference Water Index (NDWI) is used to segment land and sea to eliminate land and cloud interference. Secondly, the mean shift algorithm is used to obtain the target grayscale feature saliency map. Then, the image contrast feature map is obtained through the classic algorithm Lateral Inhibitory network (LI) based on the principle of the human visual system model (HVS). The contrast feature map is then thresholded to obtain the point distribution saliency map that filters out ship tracks and sea clutter. Finally, the target point distribution saliency map is used to guide the image segmentation of the grayscale feature saliency map for regional growth, and the ship target size characteristics are combined for screening and judgment, thereby realizing the detection of weak and small ship multi-targets in optical images.

[0053] A specific embodiment of the present invention:

[0054] Step S1: performing sea-land segmentation, cloud removal and morphological filtering on the sea area sequence images acquired by the optical remote sensing staring satellite to obtain a near-infrared image of the sea area;

[0055] Specifically, the NDWI method can be used to perform threshold segmentation on the sea area sequence image to obtain a rough ocean and cloud-land segmentation binary map;

[0056] According to the size characteristics of the ship target, cloud layer, and land, the sliding window size is determined, and the rough ocean and cloud-land segmentation binary map is traversed using the sliding window, and the area where the proportion of zero values ​​in the sliding window is less than the threshold is determined as a water area, so as to obtain an image of the sea area after the ocean and cloud-land segmentation;

[0057] A near-infrared band map of the sea area sequence image is multiplied by the sea surface area image to obtain a sea area near-infrared image.

[0058] As a specific example, in the images of the GF-4 remote sensing satellite, interference from land and clouds will reduce the accuracy of ship target detection, so land and cloud removal is required. This paper uses the NDWI method to perform land and sea segmentation and remove large areas of clouds to extract ocean areas. For multi-band remote sensing images, the characteristics of water bodies with weak reflectivity in the near-infrared band and strong reflectivity in the green band are used to enhance the difference between water bodies and land and clouds, which is convenient for threshold segmentation. The method is as follows:

[0059]

[0060] Among them, ρ g is the green band reflectivity, ρ NIR is the near-infrared reflectivity; set a fixed threshold T w , when NDWI ≥ T w When NDWI<T w When , the value of this point is set to 0, and the entire image is traversed for binary segmentation to obtain a rough binary image of ocean and cloud land segmentation. Since the spectral characteristics of ships may be similar to those of land, the non-water area obtained in the image at this time may contain ship targets, so the location of the ship needs to be regarded as a water area. This embodiment determines the size and step length of the sliding window by comparing the size of the ship target with the size characteristics of the cloud layer and the land, that is, by calculating the size of the sliding window with a side length of l H , the step length is l step Sliding window W l Traverse the entire image and calculate the proportion of 0 values ​​in the sliding window; set the threshold T for the proportion of 0 values ratio , T ratio is the proportion threshold; when the proportion of 0 values ​​is less than the threshold T ratioWhen it is determined to be a ship target, all values ​​in the sliding window are set to 1; otherwise, it is land or cloud, and all values ​​in the sliding window are set to 0. The image obtained by sliding window traversal is F, which is the image of the sea area after land and sea segmentation and cloud removal. Since the ocean has strong absorption and weak reflection in the near-infrared band, the near-infrared band map of the sea area sequence image acquired by the optical remote sensing staring satellite is multiplied with the sea area image after the ocean and land and cloud segmentation, and the image obtained is the near-infrared image of the sea area; specifically, the near-infrared band map I0 of the sea area sequence image acquired by the optical remote sensing staring satellite is multiplied with the sea area image F after the ocean and land and cloud segmentation, and the pre-processed image I1 is the near-infrared image of the sea area with land and clouds removed, that is:

[0061] I1(x,y)=I0(x,y)·F(x,y) (2)

[0062] Step S2: Based on the near-infrared image of the sea area, a target grayscale feature saliency map is obtained by using a mean shift algorithm.

[0063] Preferably, the mean shift is performed by the following method: sequentially selecting a pixel point to be processed in the near-infrared image of the sea area, setting a sample area with the pixel point to be processed as the center; setting a joint domain weight kernel function K of space and pixel gray value s,g Perform Mean Shift iteration in the sample area using a joint domain weight kernel function of space and pixel gray value, and assign the gray value of the sample center point after the iteration is terminated to the pixel point to be processed;

[0064] Based on the mean shift algorithm, all pixel points of the near infrared image of the sea area are traversed to obtain a target grayscale feature saliency map;

[0065] As a specific embodiment, according to the characteristics of image noise, the weight kernel function is selected as a Gaussian kernel function. Since the image has properties such as resolution and grayscale, using one kernel function cannot achieve the ideal processing result. Therefore, this embodiment sets a kernel function of the joint space-pixel grayscale value domain to perform mean shift iteration, so as to filter out the interference and noise in the image, so that the texture of the target or background area becomes uniform, consistent and smooth, which is convenient for subsequent target detection and segmentation.

[0066] Starting from the pixel point in the upper left corner of the near-infrared image of the sea area, select the pixels to be processed row by row from left to right, and set the side length to 4H with the pixel to be processed as the center. spatial The square area is used as the sample area, and Mean Shift iteration is performed until the iteration termination condition is met. Specifically, when the center point of the two adjacent iteration results satisfies the gray value difference less than the threshold T gray , and the distance between the positions is less than the threshold T spatial When , the iteration terminates. That is:

[0067] and

[0068] in, Respectively represent the grayscale value of the sample center point after the kth and k+1th iterations, Respectively represent the positions of the sample center points after the kth and k+1th iterations. After the iteration is terminated, the sample center point (x c ,y c ) gray value C g (x c ,y c ) is assigned to the pixel point to be processed (x, y), and the image is traversed in this way to obtain the processed grayscale image O MS , then:

[0069] O MS (x,y)=C g (x c ,y c ) (4)

[0070] Preferably, starting from the pixel at the upper left corner of the sea area near-infrared image, pixel points to be processed are selected row by row in order from left to right, and the sea area near-infrared image is traversed to obtain a processed target grayscale feature saliency map.

[0071] The present invention performs Mean Shift iteration in the sample point set by combining the spatial and pixel grayscale value domain weight kernel function, fully combining the grayscale and size characteristics of the target point. The obtained target grayscale feature saliency map can more accurately identify the ship target while filtering out clutter, and well retains the position information of the target point.

[0072] Specifically, the Mean Shift algorithm is a non-parametric density estimation method. Assuming that the grayscale value of the pixel corresponding to any pixel point (x, y) in the image is r, the three-dimensional vector corresponding to the point is expressed as (x, y, r). Let point z = (x, y, r) be a sample point in the joint space-pixel grayscale value domain, introduce the kernel function K(*), and the probability density function of the sample point z = (x, y, r) is:

[0073]

[0074] Where N is the normalization factor of the kernel function K(*), and h is the kernel radius. The derivative is:

[0075]

[0076]

[0077] In which, define g(*) = -K(*), let M(z) be the mean shift vector, and satisfy:

[0078]

[0079] when When , the probability density function reaches its maximum value, and at this time the mean shift vector M(z) = 0, and the new center point z′ is obtained as:

[0080]

[0081] The new center point z′ is used as the sample point, and the above process is repeated to calculate its mean shift vector until the condition for stopping the iteration is met.

[0082] In particular, when the joint domain weight kernel function of space and pixel gray value is introduced, equation (5) becomes:

[0083]

[0084] Among them, z s Indicates the position (x, y) of the pixel z to be processed, z g represents the grayscale r of the pixel z to be processed. Then, the center point z′ obtained by equation (9) is:

[0085]

[0086] The steps and iteration process of the mean shift algorithm are as follows:

[0087] (1) Let {z i |i=1,...,N} is a d-dimensional independent and identically distributed sequence, {θ j |j=1,2,...} is the contact position of the mean shift vector point. Given any pixel point z=(x,y,r) to be processed, and let z=θ j , through formula (11) we can get θ j+1 :

[0088]

[0089] (2) Using formulas (8), (11) and (12), we can get the j-th mean shift vector as:

[0090] M j =θ j+1 -θ j (13)

[0091] (3) When the mean shift vector satisfies the relation: |M j When |<ε, the iteration ends, otherwise it continues to execute;

[0092] (4) Let z = θ j+1 , continue to execute step (1) until the end.

[0093] Through the above iterative search process, the extreme point of probability density is obtained, which is also the maximum point.

[0094] Step S3: Based on the pre-processed near-infrared image of the sea area, a saliency map of target point distribution is obtained through a human visual system model method.

[0095] The human visual system has a lateral inhibition phenomenon, which is called the Mach band effect. Using the lateral inhibition network, the image edges can be highlighted and the contrast can be enhanced. The lateral inhibition network model is divided into two types: non-circular and cyclic. The network structure is as follows: Figure 3 (a) and Figure 3 (b) as shown.

[0096] The non-cyclic side inhibition network model has a clear input-output correspondence and high stability, so it is widely used. In addition, the non-cyclic side inhibition network model is divided into two models: subtractive non-cyclic side inhibition network and diversion non-cyclic side inhibition network. Compared with the two, the subtractive non-cyclic side inhibition network has a better effect and a lower algorithm complexity, and is more suitable for optical imaging. The expression of the subtractive non-cyclic side inhibition network is:

[0097]

[0098] Among them, I(x,y) is the input neuron (x,y), O(x,y) is the output neuron (x,y), L i,j (x,y) is the neuron inhibition coefficient, l is the width of the neuron receptive range, e i,j To indicate that the element in row i and column j is 1.

[0099] Specifically, this embodiment performs a convolution operation on the near-infrared image of the sea area through the coefficient matrix of the side inhibition network, takes the absolute value of the grayscale of the pixel after the convolution process, and obtains a contrast feature map. The grayscale value distribution of the ship target is statistically analyzed based on the grayscale histogram of the contrast feature map, a threshold is set, and the contrast feature map is thresholded to obtain a target point distribution saliency map.

[0100] The lateral suppression coefficient selected in this embodiment adopts the following matrix form:

[0101]

[0102] By performing convolution operation on the sea area near-infrared image I1 with L, we can obtain:

[0103] O LI =L*I1 (16)

[0104] To obtain a contrast feature saliency map, the following operations need to be performed:

[0105] O L ' I (x,y)=|O LI (x,y)|; (17)

[0106] Among them, O LI (x,y) is the grayscale of point (x,y).

[0107] Since the detected target is a small ship target, according to the characteristics of GF-4 satellite images and the ship target size parameters, the ship target only contains a few pixels, and the edge information can be regarded as the ship target point information. LI , and perform gray threshold T on its contrast feature saliency map LI The binary segmentation result is the target point distribution saliency map B.

[0108] Step S4: using the target point distribution saliency map as a guide, performing region growing-based image segmentation on the grayscale feature saliency map, and extracting the ship targets existing in the near-infrared image of the sea area in combination with the target size features.

[0109] As a specific embodiment, the aforementioned target point distribution saliency map B is used as a guide to perform image segmentation based on region growing in the image processed by the MeanShift algorithm, and the grayscale threshold T is set. cluster , the adjacent pixels in the four directions of up, down, left, and right of each guiding point, that is, the pixels in the four neighborhoods whose difference with the guiding point is less than the threshold T cluster The points are classified into one category, and the grayscale values ​​of all elements of this category are assigned to the mean grayscale values ​​of all pixels in this category, and different targets are divided into different categories A i , where the gray value corresponding to the nth element of the i-th category is Based on the size parameters of the ship target in the GF-4 satellite image, i The point targets that meet the ship size characteristics are selected from the image, which are the ship targets in the image and are recorded as S i The output image O obtained after binarization is the final target detection result.

[0110]

[0111] The saliency map fusion done in this paper is based on the fusion of target grayscale feature saliency map, target point distribution feature saliency map and target size feature. That is, the algorithm integrates three saliency features: target grayscale feature, contrast feature and size feature, making target detection more robust and accurate.

[0112] Another embodiment of the present invention selects a frame of image of the East my country Sea taken by GF-4 for processing. The image has 5 bands, an image size of 10240×10240 pixels, and a resolution of 50m.

[0113] In order to facilitate the display of results, this embodiment captures a representative square area I1′ as the image to be processed. Figure 7 (a) shown.

[0114] First, we select the green and near-infrared bands and use the NDWI method to remove the land and clouds. Figure 4 (a) shows the result, and then set a fixed threshold T w = 0.1, and a rough binary image of ocean and cloud land segmentation is obtained, such as Figure 4 (b). Figure 4 As can be seen in (b), the black area contains not only clouds and land, but also some ship targets that are segmented out, so the location of the ship needs to be set as water. Based on the prior knowledge of the size characteristics of ship targets, clouds, and land, the sliding window W selected in this embodiment is l The side length is l H =4, step length is l step =4, set the threshold value T for the proportion of 0 values ​​in the sliding window ratio = 0.5, and the binary image after the segmentation of ocean and cloud land is obtained, and the influence of ships is removed, such as Figure 4 However, by comparing the images of the near-infrared, green, and blue bands, it is found that the contrast between the ship and the background is higher in the near-infrared band, and the sea clutter is smaller, so it is considered Figure 4 The binary image shown in (c) is fused with the near-infrared image to obtain the preprocessed image I1, which is the water area image without land and clouds. Figure 4 (d) as shown.

[0115] Based on the water area image with land and clouds removed, a representative offshore area I′1 with multiple ship targets is taken as the test area image for target detection, as shown in Figure 5 According to the prior knowledge of ship targets, it is known that the ship target does not exceed 3×5 pixels. In this embodiment, the airspace weight kernel radius H is set spatial =2, set the gray value weight kernel radius H gray =7, when the iteration stops, we get Figure 5 (b) is the processed target grayscale feature saliency map O MS .

[0116] On the other hand, for the near-infrared sea image I1′, Figure 6As shown in (a), point distribution saliency map extraction is performed. First, the coefficient matrix L of the side inhibition network is used to perform a convolution operation on I1′, and the absolute value of the grayscale of all pixels after the convolution operation is taken to obtain Figure 6 (b) shows the contrast feature saliency map. LI .pass Figure 6 (c) shows the gray value distribution of the three-dimensional gray histogram. After thresholding, the saliency map B of the target area is obtained. Figure 6 (d)).

[0117] Finally, with the target point distribution saliency map B as a guide, in the image O processed by the MS algorithm MS Perform image segmentation based on region growing in the image segmentation, and set the threshold T of the grayscale difference between the four neighborhoods of the pixel and its grayscale cluster =5, divide different targets into different categories A i Based on the size characteristics of the target, in A i , and then the output image O is obtained as the final target detection result, such as Figure 7 (b) as shown.

[0118] The third embodiment of the present invention:

[0119] In order to verify the performance of the algorithm of the present invention for small target detection, a comparative experiment was conducted with the adaptive threshold edge detection algorithm based on Otsu, the constant false alarm rate (CFAR) algorithm, the small target detection algorithm based on the GCF-SB visual attention model, and the algorithm based on the local contrast (LCM); the near-infrared band image I1′ processed by the present invention was taken as the image to be detected. The processing results are shown in Figure 2. Figure 8 shown.

[0120] Combination Figure 8 From the data in Table 1, we can see that:

[0121] Figure 8 (b) Otsu-based adaptive threshold edge detection algorithm and Figure 8 (c) The result obtained by the LCM algorithm shows an erroneous detection area in the lower left corner. The reason is that the principle of the adaptive threshold edge detection algorithm is to perform binary segmentation by iteratively selecting the optimal global threshold. The image I1′ to be detected in this embodiment is a shallow sea area close to the coast, and its sea clutter reflection intensity is equivalent to that of some ship targets, so there will be a false detection background area. The number of false alarms of this method is 1, and the number of missed alarms is 16.

[0122] Figure 8The detection principle of the LCM algorithm in (c) is to cut the image into regions of the same size, roughly locate each target region, and then divide the target region into several blocks according to the target size, calculate the grayscale mean of each region, and obtain the region with the largest grayscale mean as the target region, and then use the appropriate threshold segmentation to achieve target detection. Its segmentation principle is similar to the adaptive threshold edge detection algorithm, so it still leads to missed alarms and false detections. The number of false alarms of this method is 0 and the number of missed alarms is 21.

[0123] Figure 8 The CFAR algorithm in (d) uses the sliding window traversal method to calculate the target discrimination condition. When the sliding window edge is within the target, the target will be mistakenly detected as two targets; for the case of sudden changes in background clutter, the false alarm rate will increase. The number of false alarms and missed alarms of this method is 7 and 3, respectively.

[0124] Figure 8 The GCF-SB algorithm in (e) uses the mean shift algorithm for smoothing. However, smoothing will cause the edge contour to become larger and some targets to be lost. The extracted edge features are interfered by the ship's trajectory and the waves it stirs up. Therefore, after the grayscale contrast features are fused, the detected targets will become larger and distorted. Smoothing will also cause some missed alarms. The number of false alarms for this method is 4 and the number of missed alarms is 5.

[0125] The detection method used in the present invention uses a small number of smoothing iterations to retain all target information as much as possible, further filters out background clutter interference through a clustering method, extracts edge features through a suitable threshold, retains the true size of the ship as much as possible, and achieves a better target detection effect through the final matching fusion method. Figure 8 (f) as shown.

[0126] Table 1 Data analysis of the results of five small target detection algorithms for image I1′ processing

[0127]

[0128]

[0129] In order to further verify the universality and reliability of the algorithm of the present invention, another area I″1 in the image is intercepted as the image to be detected, and the experimental results of the five algorithms are compared. Fig. 9 As shown, Fig. 9 (a) is the water area image in the near-infrared band. Fig. 9 (b) is the processing result of the adaptive threshold edge detection algorithm. Fig. 9 (c) is the result of LCM algorithm processing. Fig. 9 (d) is the result of CFAR algorithm processing. Fig. 9 (e) is the result of GCF-SB algorithm processing. Fig. 9 (f) is the processing result diagram of the algorithm in this paper.

[0130] Combination Fig. 9 As can be seen from the data in Table 2, there is a small piece of complex background on the left side of the selected image to be detected. The detection results of the five algorithms are analyzed. Since the image sea clutter interference is small, the detection effect of the four comparison algorithms is improved, but there are still false alarms and missed alarms, and the algorithm of the present invention still has a good target detection effect. The comparison data is summarized in the following Table 2:

[0131] Table 2 Data analysis of the results of five small target detection algorithms for image I″1 processing

[0132]

[0133] In summary, the present invention proposes a visually significant target segmentation detection method based on mean shift and the principle of human visual system for the detection of multiple weak and small ships in optical remote sensing images. The method is divided into four parts: using the MS algorithm to smooth and remove background point noise, and obtaining the grayscale saliency map of the target; then using the side inhibition network algorithm based on the HVS principle to obtain the target contrast feature saliency map, and thresholding to obtain the target point distribution saliency map; then using the target point distribution saliency map as a guide to segment and extract the target grayscale saliency map, and combining the target size features for screening and judgment, finally achieving a target detection effect with better performance. Experimental results show that the algorithm in this paper has a very significant advantage in the detection of multiple weak and small targets, and is suitable for the detection of multiple weak and small targets in optical images with complex backgrounds and low signal-to-noise ratios. The present invention takes the detection of small ship targets on the sea surface in optical remote sensing images as an application case. The overall solution solves the problems that the small target detection method in the prior art has a high false alarm rate, low robustness and accuracy, and is not suitable for small target detection under low signal-to-noise ratio and large sea clutter. At the same time, the method can also be used for the detection of small targets in air, sky, land, sea and other backgrounds, and has high robustness, accuracy and universality.

[0134] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0135] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A weak small ship target fusion detection method based on multi-visual salient features, characterized in that: The steps include: The sea area sequence images acquired by the optical remote sensing staring satellite are subjected to sea-land segmentation, cloud removal and morphological filtering to obtain the sea area near-infrared image; Based on the near-infrared image of the sea area, a target grayscale feature saliency map is obtained by a mean shift algorithm, wherein the mean shift algorithm comprises: selecting a pixel point to be processed of the near-infrared image of the sea area in turn, and setting a sample area with the pixel point to be processed as the center; setting a space and pixel grayscale value joint domain weight kernel function, performing Mean Shift iteration in the sample area by using the space and pixel grayscale value joint domain weight kernel function, and assigning the grayscale value of the sample center point after the iteration is terminated to the pixel point to be processed; traversing all pixel points of the near-infrared image of the sea area based on the mean shift algorithm to obtain a target grayscale feature saliency map; Based on the near-infrared image of the sea area, a saliency map of target point distribution is obtained through a human visual system model; Using the target point distribution saliency map as a guide, the grayscale feature saliency map is subjected to image segmentation based on region growing, and the ship targets existing in the near-infrared image of the sea area are extracted in combination with the ship target size features.

2. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 1 is characterized in that: The method of obtaining a target point distribution saliency map through a human visual system model includes: obtaining a contrast feature map of the sea area near-infrared image through a lateral inhibition network algorithm, and thresholding the contrast feature map to obtain the target point distribution saliency map.

3. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 2 is characterized in that: The method of obtaining the contrast feature map of the near-infrared image of the sea area through the side inhibition network algorithm includes: performing a convolution operation on the near-infrared image of the sea area through the coefficient matrix of the side inhibition network, taking the absolute value of the grayscale of the pixel after the convolution process, and obtaining the contrast feature map.

4. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 2 is characterized in that: The step of thresholding the contrast feature map to obtain the target point distribution saliency map includes: setting a threshold based on the gray value distribution of the ship target in the contrast feature map, binarizing the contrast feature map with the gray value threshold, and obtaining the target point distribution saliency map.

5. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 3 is characterized in that: The coefficient matrix of the lateral inhibition network is expressed as:

6. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 1 is characterized in that: The sea-land segmentation, cloud removal and morphological filtering of the sea area sequence images acquired by the optical remote sensing staring satellite include: The NDWI method is used to perform threshold segmentation on the sea area sequence images to obtain a rough ocean and cloud-land segmentation binary map; According to the size characteristics of the ship target, cloud layer, and land, the sliding window size is determined, and the rough ocean and cloud-land segmentation binary map is traversed using the sliding window, and the area where the proportion of zero values ​​in the sliding window is less than the threshold is determined as a water area, so as to obtain an image of the sea area after the ocean and cloud-land segmentation; A near-infrared band map of the sea area sequence image is multiplied by the sea surface area image to obtain a sea area near-infrared image.

7. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 1 is characterized in that: The step of extracting the ship target in the near infrared image of the sea area includes: setting a gray threshold T cluster , taking the target point in the target point distribution saliency map as the guide point, and taking the guide point in the four neighborhoods with grayscale differences less than the threshold T cluster The points are classified into one category, and the grayscale values ​​of all pixels of this category are assigned to the mean grayscale values ​​of all pixels of this category; different target points are divided into different categories A i Based on the size parameters of the ship target in the near infrared image of the sea area, from A i The target points that meet the ship size characteristics are screened out, which are the ship targets existing in the near-infrared image of the sea area.

8. The method for detecting small and weak ship targets based on multi-visual salient features according to claim 7 is characterized in that: When performing the image segmentation based on region growing, the threshold of the grayscale difference between the four neighborhoods of the guide point and the guide point is set to T cluster =5.

9. A weak small ship target fusion detection device based on multi-visual salient features, characterized in that: comprising at least one processor, and at least one memory communicatively connected to the processor; The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the weak and small ship target fusion detection method based on multi-visual significant features as described in any one of claims 1-8.

Images