A ship detection method in SAR images based on human visual attention mechanism

Abstract

The present invention belongs to the field of target detection technology, and specifically provides a SAR image ship detection method based on the human eye visual attention mechanism. The human eye visual attention mechanism is introduced into SAR image ship target detection, and by modifying the Itti model, brightness, local contrast features, and global contrast features are used to find significant areas, thereby overcoming the problem of uneven display of significant areas caused by only considering local features, which in turn leads to low subsequent target detection accuracy. Then, HOG features are extracted, an SVM classifier is trained, and a sliding window is used to detect ship targets. Finally, the test results of the HRSID dataset show that the present invention can solve the problem of false detection of SAR image ship detection under complex backgrounds, and improve the accuracy of SAR image ship target detection to 95.56%.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular to a method for detecting ships in SAR images based on the visual attention mechanism of the human eye. Background Art

[0002] Synthetic Aperture Radar (SAR) is unrestricted by weather and time, and can be installed on a variety of flight platforms, including drones, aircraft, and satellites. These characteristics have led to its widespread application in both civilian and military fields. As SAR image resolution increases, texture and edge information within the images become increasingly rich, making traditional methods inadequate for processing this rich information. Furthermore, in complex scenes, target detection is significantly more challenging due to the relatively high intensity and uneven distribution of clutter scattering and the relatively weak and densely distributed target scattering. Therefore, processing high-resolution images and accurately locating and finding areas of interest within complex backgrounds are crucial.

[0003] The human visual attention mechanism is a selective attention mechanism that can make orderly selections of information received from the outside world, so it can always quickly screen out important information from a large amount of information. Therefore, inspired by the human visual attention mechanism, some researchers have proposed introducing the human visual attention mechanism into target detection. First, through the human visual attention mechanism model, the salient and prominent target areas in the image are found, that is, the target area is located in a complex background, and then the target detection task is performed. The algorithm that constructs a saliency map for the image and extracts the salient areas in the image by simulating the human visual attention mechanism is called a saliency detection algorithm.

[0004] Currently, most saliency detection algorithms are designed for optical images, but the imaging mechanisms of optical images and SAR images are different and there are significant differences. Secondly, most existing saliency detection algorithms obtain the saliency of each subdomain in the image by calculating the contrast between the surrounding adjacent areas. Although this can detect salient areas, the algorithm focuses on local features, resulting in higher saliency values ​​at the edges of salient object regions, while lower saliency values ​​for salient internal objects. This makes it difficult to present the entire salient area uniformly, which in turn leads to false detection problems in subsequent SAR image target detection, resulting in low SAR image target detection accuracy. Therefore, starting from global features and combining the inherent characteristics of SAR images, a human visual attention mechanism model suitable for SAR images is implemented, and this model is applied to SAR image ship detection, which is a current problem to be solved. Summary of the Invention

[0005] In order to overcome the defects of the above-mentioned prior art, the present invention proposes a SAR image ship detection method based on the human eye visual attention mechanism, introduces the human eye visual attention mechanism into SAR image ship target detection, and finds significant areas by modifying the Itti model using brightness, local contrast features, and global contrast features, thereby overcoming the problem of uneven display of significant areas caused by only considering local features, which in turn leads to low accuracy of subsequent target detection; then, HOG features are extracted, the SVM classifier is trained, and the ship targets are detected using a sliding window; finally, the test results of the HRSID dataset show that the present invention can solve the problem of false detection of SAR image ship detection under complex backgrounds, and improve the accuracy of SAR image ship target detection to 95.56%.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for detecting ships in SAR images based on the human eye's visual attention mechanism comprises the following steps:

[0008] Step 1: Calculate the brightness feature matrix, local contrast feature matrix, and global contrast feature matrix of the input SAR image, and then construct a Gaussian pyramid. Use the Gaussian pyramid to calculate the brightness feature map, local contrast feature map, and global contrast feature map at multiple scales.

[0009] Step 2: Normalize each feature map separately, then add the normalized brightness feature maps at multiple scales to obtain a brightness feature saliency map, add the local contrast feature maps at multiple scales to obtain a local contrast feature saliency map, and add the global contrast feature maps at multiple scales to obtain a global contrast feature saliency map;

[0010] Step 3: Perform linear weighted addition of the brightness feature saliency map, the local contrast feature saliency map, and the global contrast feature saliency map to obtain a visual saliency map;

[0011] Step 4: Extract HOG features from the visual saliency map and train an SVM classifier;

[0012] Step 5: Use the SVM classifier to detect ship targets using a sliding window, and eliminate overlapping windows through non-maximum suppression to obtain the final detection result.

[0013] Furthermore, step 1 is specifically as follows:

[0014] Step 1-1: Calculate the brightness feature matrix of the input SAR image:

[0015]

[0016] Among them, f lis the brightness feature matrix, I is the pixel matrix of the input SAR image, and (i, j) is the pixel coordinate;

[0017] Step 1-2: Calculate the local contrast feature matrix of the input SAR image;

[0018] Use a sliding window to scan the entire input SAR image and calculate the local contrast characteristics of each window: divide the window T(i,j) into 9 equal areas; for the central area, search for the maximum value L of the pixel points in the area max ; For other areas except the central area, calculate the pixel mean in each area and search for the maximum value of the pixel mean The local contrast feature is calculated as:

[0019]

[0020] Among them, f loc is the local contrast feature matrix, n is the preset power value;

[0021] Step 1-3: Calculate the global contrast feature matrix of the input SAR image:

[0022]

[0023] g(i,j)=I(i,j)-μ

[0024]

[0025] Among them, f glob is the global contrast feature matrix, μ represents the pixel mean of the input SAR image, N is the total number of pixels of the input SAR image, and α is the preset threshold;

[0026] Steps 1-4: Downsample the brightness feature matrix, local contrast feature matrix, and global contrast feature matrix of the input SAR image multiple times to obtain the brightness feature Gaussian pyramid, local contrast feature Gaussian pyramid, and global contrast feature Gaussian pyramid, respectively;

[0027] Steps 1-5: Based on the brightness feature Gaussian pyramid, local contrast feature Gaussian pyramid, and global contrast feature Gaussian pyramid, the central-peripheral feature is used to simulate the working mode of the receptive field to obtain the brightness feature map l(c, s), local contrast feature map loc(c, s), and global contrast feature map g(c, s) at multiple scales, respectively. Among them, c and s are both scales in the Gaussian pyramid.

[0028] Furthermore, step 2 is specifically as follows:

[0029] Step 2-1: Normalize each brightness feature map l(c,s), local contrast feature map loc(c,s) or global contrast feature map g(c,s) separately: First, normalize the feature map to the range [0,M], where M is the global maximum value in the feature map; then calculate the average value of the local maximum values ​​excluding the global maximum value M. Then normalize the feature map and coefficient Multiply; finally, set the size of the multiplied feature map to the same as the input SAR image; after normalization, the corresponding brightness feature map l′(c, s), local contrast feature map loc′(c, s) or global contrast feature map g′(c, s) is obtained;

[0030] Step 2-2: Add the feature maps at multiple scales to obtain the brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map respectively:

[0031]

[0032]

[0033]

[0034] Among them, L, Loc and G are brightness feature saliency map, local contrast feature saliency map and global contrast feature saliency map respectively.

[0035] Furthermore, step 3 is specifically as follows:

[0036] Step 3-1: Normalize the brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map to the range of [0,1] to obtain the normalized brightness feature saliency map. Local contrast feature saliency map and global contrast feature saliency map

[0037] Step 3-2: Add the normalized brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map through linear weighted addition to obtain the visual saliency map:

[0038]

[0039] Among them, SM is the visual saliency map; ω1, ω2 and ω3 are all preset weights.

[0040] Furthermore, step 4 is specifically as follows:

[0041] Step 4-1: Extract HOG features from the visually salient image and obtain the HOG feature vector;

[0042] Step 4-2: Use the HOG feature vector as the input of the SVM model for training to obtain the SVM classifier.

[0043] Furthermore, step 5 is specifically as follows:

[0044] Step 5-1: Use a sliding window to scan the entire visual saliency map, and use the SVM classifier to classify each window into background and target.

[0045] Step 5-2: Use non-maximum suppression to eliminate overlapping boxes and obtain the final detection result.

[0046] The beneficial effects of the present invention are:

[0047] The present invention provides a method for detecting ships in SAR images based on the human eye's visual attention mechanism, which has the following advantages:

[0048] 1) The method of the present invention adopts the human eye's visual attention mechanism to accurately find the "interested" region in a complex background, narrowing the scope of subsequent target detection and laying the foundation for subsequent target detection; at the same time, it also suppresses background clutter, reduces the difficulty of subsequent target detection, and improves the target detection accuracy to 95.56%.

[0049] 2) This paper improves the Itti model. Ships appear as bright spots in SAR images. To obtain salient regions, the image brightness features are first extracted. Secondly, to further highlight salient regions in SAR images, the contrast between the ship target and the surrounding sea background is enhanced by utilizing the difference between the two, and local contrast features are extracted. Finally, to further highlight salient regions in the entire image, not only the local relationship between the target and its neighbors is considered, but also the global relationship between the target region and the entire image, extracting global contrast features from the SAR image. Through these improvements, the model can accurately locate salient regions and evenly highlight them. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a flow chart of the ship detection method in SAR images based on the human eye visual attention mechanism of the present invention.

[0051] Figure 2 Schematic diagram of the local contrast sliding window in the present invention.

[0052] Figure 3 Graphs showing simulation test results in an embodiment of the present invention, where (a) is the original graph and (b) is the improved Itti model result graph.

[0053] Figure 4Figure 1 is a simulation test result diagram in an embodiment of the present invention, where (a) is the original image, (b) is the manually labeled standard target result diagram (Ground Truth, GT), (c) is the SAR image target detection result diagram based on the human eye visual attention mechanism, and (d) is the SAR image ship detection result diagram without the human eye visual attention mechanism. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and effects of the present invention more clear and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0055] This embodiment provides a method for detecting ships in SAR images based on the human eye's visual attention mechanism. The process is as follows: Figure 1 As shown, the specific steps include:

[0056] Step 1: Calculate the brightness feature matrix, local contrast feature matrix, and global contrast feature matrix of the input SAR image, and then construct a Gaussian pyramid. Use the Gaussian pyramid to calculate the brightness feature map, local contrast feature map, and global contrast feature map at multiple scales; specifically:

[0057] Step 1-1: Calculate the brightness feature matrix of the input SAR image. The calculation formula is:

[0058]

[0059] Among them, f l is the brightness feature matrix, I is the pixel matrix of the input SAR image, (i, j) is the pixel coordinate, that is, I(i, j) is the pixel value at the coordinate point (i, j) in the input SAR image;

[0060] Step 1-2: Calculate the local contrast feature matrix of the input SAR image;

[0061] The entire input SAR image is scanned using a sliding window with a window size of 9×9 and a sliding step size of 1, and the windows T(i, j) are obtained in sequence, as shown in Figure 2 As shown; calculate the local contrast features of each window:

[0062] Divide the window T(i,j) into 9 equal regions (each region is 3×3); for the central region, search for the maximum value of the pixels in the region, denoted as L max ; For other areas except the central area, calculate the pixel mean in each area and search for the maximum value of the pixel mean, recorded as The local contrast feature is calculated as:

[0063]

[0064] Among them, f loc is the local contrast feature matrix, n is the preset power value, and the value is 5 according to experience;

[0065] Step 1-3: Calculate the global contrast feature matrix of the input SAR image. The calculation formula is:

[0066]

[0067] g(i,j)=I(i,j)-μ

[0068]

[0069] Among them, f glob is the global contrast feature matrix, μ represents the pixel mean of the input SAR image, N is the total number of pixels of the input SAR image; α is the preset threshold, usually α=μ 2 / σ, σ represents the pixel mean square error of the input SAR image;

[0070] Steps 1-4: Downsample the brightness feature matrix, local contrast feature matrix, and global contrast feature matrix of the input SAR image 8 times respectively, and obtain 9 layers of brightness feature Gaussian pyramid, local contrast feature Gaussian pyramid, and global contrast feature Gaussian pyramid respectively;

[0071] Steps 1-5: Based on the brightness feature Gaussian pyramid, local contrast feature Gaussian pyramid, and global contrast feature Gaussian pyramid, the central-peripheral feature is used to simulate the working mode of the receptive field to obtain 6 brightness feature maps l(c, s), 6 local contrast feature maps loc(c, s), and 6 global contrast feature maps g(c, s), respectively. Among them, c and s are the scales in the Gaussian pyramid, c = 2, 3, 4, and s = c+3, c+4;

[0072] Step 2: Normalize each feature map separately, then add the normalized brightness feature maps at multiple scales to obtain a brightness feature saliency map, add the local contrast feature maps at multiple scales to obtain a local contrast feature saliency map, and add the global contrast feature maps at multiple scales to obtain a global contrast feature saliency map; specifically:

[0073] Step 2-1: Normalize each brightness feature map l(c,s), local contrast feature map loc(c,s) or global contrast feature map g(c,s) separately: First, normalize the feature map to the range [0,M], where M is the global maximum value in the feature map; then calculate the average value of the local maximum values ​​excluding the global maximum value M. Then normalize the feature map and coefficient Multiply; finally, set the size of the multiplied feature map to the same as the input SAR image; after normalization, the corresponding brightness feature map l′(c, s), local contrast feature map loc′(c, s) or global contrast feature map g′(c, s) is obtained;

[0074] Step 2-2: Add the feature maps at multiple scales to obtain the brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map. The calculation formula is:

[0075]

[0076]

[0077]

[0078] Among them, L, Loc and G are the brightness feature saliency map, local contrast feature saliency map and global contrast feature saliency map of the input SAR image respectively;

[0079] Step 3: Perform linear weighted addition of the brightness feature saliency map, the local contrast feature saliency map, and the global contrast feature saliency map to obtain a visual saliency map; specifically:

[0080] Step 3-1: Normalize the brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map to the range of [0,1] to obtain the normalized brightness feature saliency map. Local contrast feature saliency map and global contrast feature saliency map

[0081] Step 3-2: Add the normalized brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map through linear weighted addition to obtain the visual saliency map (SM). The calculation formula is:

[0082]

[0083] Among them, SM is the visual saliency map; ω1, ω2 and ω3 are the preset weights of the brightness feature saliency map, the local contrast feature saliency map and the global contrast feature saliency map respectively, all of which are empirical values, and the values ​​are 0.3, 0.3 and 0.4 respectively according to experience;

[0084] Step 4: Extract HOG features from the visual saliency map and train an SVM classifier; specifically:

[0085] Step 4-1: Extract HOG features from the visually salient image and obtain the HOG feature vector;

[0086] Step 4-2: Use the HOG feature vector as the input of the SVM model for training to obtain the SVM classifier;

[0087] Step 5: Use the sliding window to detect the ship target and eliminate the overlapping windows through non-maximum suppression to obtain the final detection result; specifically:

[0088] Step 5-1: Use a sliding window with a window size of 64×64 and a sliding step size of 8 to scan the entire visual saliency map, and use the SVM classifier to classify each window into background and target.

[0089] Step 5-2: Use non-maximum suppression to eliminate overlapping boxes and obtain the final detection result.

[0090] The following is a simulation test to further illustrate the beneficial effects of the present invention: In this embodiment, a simulation experiment was conducted based on the HRSID dataset. The image resolution in the HRSID dataset is 0.5m, 1m, and 3m, and the image size is 800×800. The software environment is python3.6+opencv4.5+scikit-learn0.20, and the PyCharm development tool is used for algorithm development. Recall, accuracy, and F1-score are used as evaluation indicators. In order to prove the effectiveness of the present invention, a comparison is made with the HOG+SVM model that does not introduce the human eye visual attention mechanism. The comparison results are shown in Table 1:

[0091] Table 1: Comparison of the results of the present invention and the HOG+SVM model

[0092] Model Recall Precision F1-score HOG+SVM 90.05％ 62.64％ 73.88％ The present invention 83.70％ 95.56％ 89.24％

[0093] As shown in Table 1, although the recall rate of the proposed method is slightly lower than that of the HOG+SVM model without the human eye visual attention mechanism, the accuracy rate is much higher than that of the HOG+SVM model, which is improved by 52.55%. Furthermore, the F1-score of the proposed method is better than that of the HOG+SVM model, which is improved by 20.79%. Figure 3 、 Figure 4 The results shown, from Figure 3 As can be seen from (a) and (b), the improved Itti model can accurately locate the area where the ship target is located and present the area evenly, rather than only the edge information of the target area, while also suppressing background clutter. Figure 4 As shown in (b), (c), and (d), false detection occurs in SAR ship target detection without the human eye visual attention mechanism. After adding the human eye visual attention mechanism, the false detection problem of SAR image ship detection under complex backgrounds can be better solved, and the target detection accuracy is improved to 95.56%.

[0094] In summary, the present invention addresses the problems existing in the ship target detection algorithm for SAR images under complex backgrounds. Starting from improving the accuracy of the existing SAR image ship detection algorithm, a SAR image ship detection algorithm based on the human eye visual attention mechanism is proposed. By constructing a saliency map for the image, finding the salient areas, and then performing ship detection in the salient areas, the detection accuracy of the original detection model is improved. The method is effective and suitable for SAR image ship detection.

[0095] The above description is only a specific embodiment of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes; all disclosed features, or all steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A method for ship detection in SAR images based on the human eye's visual attention mechanism, comprising the following steps: Step 1: Calculate the brightness feature matrix, local contrast feature matrix, and global contrast feature matrix of the input SAR image, and then construct a Gaussian pyramid. Use the Gaussian pyramid to calculate the brightness feature map, local contrast feature map, and global contrast feature map at multiple scales; specifically: Step 1-1: Calculate the brightness feature matrix of the input SAR image: Among them, f l is the brightness feature matrix, I is the pixel matrix of the input SAR image, and (i, j) is the pixel coordinate; Step 1-2: Calculate the local contrast feature matrix of the input SAR image; Use a sliding window to scan the entire input SAR image and calculate the local contrast characteristics of each window: divide the window T(i,j) into 9 equal areas; for the central area, search for the maximum value L of the pixel points in the area max ; For other areas except the central area, calculate the pixel mean in each area and search for the maximum value L of the pixel mean max ; Calculate the local contrast feature as: Among them, f loc is the local contrast feature matrix, n is the preset power value; Step 1-3: Calculate the global contrast feature matrix of the input SAR image: g(i,j)=I(i,j)-μ Among them, f glob is the global contrast feature matrix, μ represents the pixel mean of the input SAR image, N is the total number of pixels of the input SAR image, and α is the preset threshold; Steps 1-4: Downsample the brightness feature matrix, local contrast feature matrix, and global contrast feature matrix of the input SAR image multiple times to obtain the brightness feature Gaussian pyramid, local contrast feature Gaussian pyramid, and global contrast feature Gaussian pyramid, respectively; Steps 1-5: Based on the brightness feature Gaussian pyramid, local contrast feature Gaussian pyramid, and global contrast feature Gaussian pyramid, the central-peripheral feature is used to simulate the working mode of the receptive field to obtain the brightness feature map l(c, s), local contrast feature map loc(c, s), and global contrast feature map g(c, s) at multiple scales, respectively. Among them, c and s are the scales in the Gaussian pyramid; Step 2: Normalize each feature map separately, then add the normalized brightness feature maps at multiple scales to obtain a brightness feature saliency map, add the local contrast feature maps at multiple scales to obtain a local contrast feature saliency map, and add the global contrast feature maps at multiple scales to obtain a global contrast feature saliency map; Step 3: Perform linear weighted addition of the brightness feature saliency map, the local contrast feature saliency map, and the global contrast feature saliency map to obtain a visual saliency map; Step 4: Extract HOG features from the visual saliency map and train an SVM classifier; Step 5: Use the SVM classifier to detect ship targets using a sliding window, and eliminate overlapping windows through non-maximum suppression to obtain the final detection result.

2. The method for ship detection in SAR images based on the human eye visual attention mechanism according to claim 1, characterized in that: Step 2 is as follows: Step 2-1: Normalize each brightness feature map l(c,s), local contrast feature map loc(c,s) or global contrast feature map g(c,s) separately: First, normalize the feature map to the range [0,M], where M is the global maximum value in the feature map; then calculate the average value of the local maximum values ​​excluding the global maximum value M. Then normalize the feature map and coefficient Multiply; finally, set the size of the multiplied feature map to the same as the input SAR image; after normalization, the corresponding brightness feature map l′(c, s), local contrast feature map loc′(c, s) or global contrast feature map g′(c, s) is obtained; Step 2-2: Add the feature maps at multiple scales to obtain the brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map respectively: Among them, L, Loc and G are brightness feature saliency map, local contrast feature saliency map and global contrast feature saliency map respectively.

3. The method for ship detection in SAR images based on human visual attention mechanism according to claim 1, characterized in that: Step 3 is as follows: Step 3-1: Normalize the brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map to the range of [0,1] to obtain the normalized brightness feature saliency map. Local contrast feature saliency map and global contrast feature saliency map Step 3-2: Add the normalized brightness feature saliency map, local contrast feature saliency map, and global contrast feature saliency map through linear weighted addition to obtain the visual saliency map: Among them, SM is the visual saliency map; ω1, ω2 and ω3 are all preset weights.

4. The method for ship detection in SAR images based on human visual attention mechanism according to claim 1, characterized in that: Step 4 is as follows: Step 4-1: Extract HOG features from the visually salient image and obtain the HOG feature vector; Step 4-2: Use the HOG feature vector as the input of the SVM model for training to obtain the SVM classifier.

5. The method for ship detection in SAR images based on human visual attention mechanism according to claim 1, characterized in that: Step 5 is as follows: Step 5-1: Use a sliding window to scan the entire visual saliency map, and use the SVM classifier to classify each window into background and target. Step 5-2: Use non-maximum suppression to eliminate overlapping boxes and obtain the final detection result.

Images