A ship target recognition method based on deep learning in remote sensing images based on threshold constraints

Abstract

The present invention discloses a method for ship target recognition in remote sensing images based on deep learning based on threshold constraints, comprising the following steps: (1) performing threshold segmentation on remote sensing images using the OTSU threshold segmentation method to achieve land-sea separation; (2) extracting shape features of the remote sensing images; (3) performing multi-scale connection and fusion on the underlying pyramid network structure based on the deep learning YOLOv5 algorithm; (4) designing anchor frames based on the shape features of ship targets in remote sensing images; (5) introducing focused classification loss as the loss function of the YOLOv5 algorithm for regression convergence; (6) inputting ship samples for training to obtain a model based on the improved YOLOv5 algorithm; and (7) recognizing ship targets in remote sensing images based on the trained model. The present invention optimizes the anchor frames and loss function based on the shape features of the ships, thereby improving the generalization performance of the model and enhancing the accuracy of ship recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition, and in particular relates to a method for ship target recognition in remote sensing images based on deep learning of threshold constraints. Background Art

[0002] With the rapid development of remote sensing technology, deep learning is being used to quickly and accurately identify targets from satellite imagery. This can, on the one hand, replace repetitive and tedious human tasks, freeing people from the arduous task of interpreting remote sensing images. On the other hand, establishing an end-to-end model architecture can not only increase the processing rate of remote sensing data, but also achieve high recognition accuracy. Introducing deep learning into target detection tasks can break free from the constraints of traditional detection algorithms, which require manual design of detection features. By autonomously learning and extracting relevant features through model networks, the algorithm's detection efficiency and reliability are improved. Rapid and accurate target detection of ships based on deep learning algorithms can automate target detection, significantly reducing human resource consumption and improving detection speed and accuracy in large amounts of data.

[0003] Ship detection is a key application in remote sensing, with broad applications in both military and civilian sectors. In the military, ship detection can be used to monitor illegal vessels, such as those involved in terrorist attacks and illegal immigration, as well as for combat effectiveness assessment, wartime sea surveillance, and local vessel reconnaissance. In the civilian sector, ship target detection technology can be applied to monitoring and managing marine fisheries, controlling port navigation, and monitoring and measuring marine pollution. For practical applications, corresponding ship detection systems must be highly automated and timely. However, traditional methods are limited by prior knowledge and struggle to achieve the required efficiency and accuracy. Therefore, the application of deep learning technology to remote sensing ship identification is of great significance.

[0004] Object detection algorithms in deep learning can be broadly divided into two categories: two-stage region proposal-based algorithms, exemplified by algorithms like Fast R-CNN, Faster R-CNN, and R-FCN; and single-stage regression-based algorithms, exemplified by YOLO and SSD. YOLO has garnered considerable attention for ship recognition due to its lightweight model strategy, fast recognition speed, and excellent performance in detecting small objects. The latest YOLO method, YOLOv5, has been updated. However, as it remains a single-stage regression-based algorithm, it prioritizes efficiency at the expense of accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning remote sensing image ship target recognition method based on threshold constraint, which improves the recognition efficiency of ship targets while ensuring the recognition rate.

[0006] The technical solution adopted by the present invention to achieve its purpose is:

[0007] A method for ship target recognition in remote sensing images based on deep learning and threshold constraints includes the following steps:

[0008] Step 1: Use the OTSU threshold segmentation method to perform threshold segmentation on the remote sensing image to separate land and sea;

[0009] Step 2: Extract shape features from the sea area remote sensing image after land and sea separation;

[0010] Step 3: Based on the deep learning YOLOv5 algorithm and combined with the feature pyramid structure, the underlying pyramid network structure is connected and fused at multiple scales;

[0011] Step 4: Design the anchor frame based on the shape characteristics of the ship target in the remote sensing image;

[0012] Step 5: Introduce focused classification loss as the loss function of the YOLOv5 algorithm for regression convergence;

[0013] Step 6: Based on the improved YOLOv5 algorithm, input the pre-labeled ship samples for training to obtain a trained deep learning model;

[0014] Step 7: Perform ship target recognition on remote sensing images based on the trained model.

[0015] Furthermore, the step 1 specifically includes: determining a threshold based on the grayscale difference between the ocean and the land using the threshold value, and then separating the ocean and the land using the selected grayscale value as the threshold value:

[0016] The OTSU threshold segmentation method uses the inter-class method measurement, which is calculated as follows:

[0017] σ 2 (T)=W a (μ a -μ) 2 +W b (μ b -μ) 2

[0018] Among them, σ 2 represents the inter-class method, W a Represents the ratio of the target area to the total image area, μ a Represents the mean value of all pixels in the target area, W bRepresents the ratio of the background area to the total image area, μ b Represents the average value of pixels in the background area, μ represents the grayscale average of all pixels in the full image, T traverses from 0 to 255, so that σ 2 The maximum value of T is the optimal threshold for image segmentation.

[0019] Furthermore, the step 2 specifically includes: based on the characteristics of the ship itself, its shape is a long strip, and considering the ability to clearly describe the target's outer contour elements under the multi-scale influence of remote sensing images, three features are selected: compactness, aspect ratio, and rectangularity. The calculation formula is as follows:

[0020] (1) Firmness:

[0021]

[0022] (2) Aspect ratio:

[0023]

[0024] (3) Rectangularity:

[0025]

[0026] Where A represents the area of ​​the target area, p represents the perimeter of the target area, l, w, and S represent the length, width, and area of ​​the circumscribed rectangle of the target area, respectively.

[0027] Furthermore, the step 3 specifically includes: focusing on the lateral connections of the low-level feature layers, performing feature enhancement on the low-level feature layers, and removing the top-level connections while ensuring the computational complexity and speed.

[0028] Furthermore, the step 4 specifically includes: the calculation formula for designing the anchor frame according to its shape characteristics is:

[0029]

[0030] Where m refers to the number of feature maps, S k Indicates the ratio of the anchor box size to the image, S mt浔 and S max Indicates the minimum and maximum values ​​of the ratio,

[0031] The aspect ratio calculation formula of the anchor box is:

[0032]

[0033]

[0034] in s kIt is calculated by the previous formula, adding a default box scale for the case where the aspect ratio is 1 The default coordinate center of the box is |f k | is the size of the corresponding feature map, i,j∈[0,|f k |).

[0035] Furthermore, the step 5 specifically includes: introducing the focus classification loss as the loss function of the YOLOv5 algorithm to calculate the regression convergence formula as follows:

[0036]

[0037] Where: L loc (b * ,b) is the positioning loss function, and its specific calculation method is: b is the predicted value, b * is the true label box value, is the classification loss function, which is calculated as follows: t i,c For the correct forecast probability, is the background prediction probability, a t and r are kernel parameters.

[0038] Compared with the prior art, the advantages of the present invention are:

[0039] Compared with traditional deep learning algorithms, this invention targets the characteristics of ship targets. On the one hand, it uses the prior knowledge that the ship background is the sea area as a constraint, separates land and sea before recognition, narrows the recognition range and reduces the false recognition rate; on the other hand, it combines the shape characteristics of the ship, designs the anchor frame and optimizes the loss function, so that it can greatly improve the recognition efficiency while ensuring the recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the main process of the threshold-constrained deep learning remote sensing image ship target recognition method of the present invention. DETAILED DESCRIPTION

[0041] Combine Figure 1 A method for ship target recognition in remote sensing images based on deep learning and threshold constraints includes the following steps:

[0042] Step 1: Use the OTSU threshold segmentation method to perform threshold segmentation on the remote sensing image to separate land and sea;

[0043] Step 2: Extract shape features from the sea area remote sensing image after land and sea separation;

[0044] Step 3: Based on the deep learning YOLOv5 algorithm and combined with the feature pyramid structure, the underlying pyramid network structure is connected and fused at multiple scales;

[0045] Step 4: Design the anchor frame based on the shape characteristics of the ship target in the remote sensing image;

[0046] Step 5: Introduce focused classification loss as the loss function of the YOLOv5 algorithm for regression convergence;

[0047] Step 6: Based on the improved YOLOv5 algorithm, input the pre-labeled ship samples for training to obtain a trained deep learning model;

[0048] Step 7: Perform ship target recognition on remote sensing images based on the trained model.

[0049] Furthermore, the step 1 specifically includes: determining a threshold based on the grayscale difference between the ocean and the land using the threshold value, and then separating the ocean and the land using the selected grayscale value as the threshold value:

[0050] The OTSU threshold segmentation method uses the inter-class method measurement, which is calculated as follows:

[0051] σ 2 (T)=W a (μ a -μ) 2 +W b (μ b -μ) 2

[0052] Among them, σ 2 represents the inter-class method, W a Represents the ratio of the target area to the total image area, μ a Represents the mean value of all pixels in the target area, W b Represents the ratio of the background area to the total image area, μ b Represents the average value of pixels in the background area, μ represents the grayscale average of all pixels in the full image, T traverses from 0 to 255, so that σ 2 The maximum value of T is the optimal threshold for image segmentation.

[0053] Furthermore, the step 2 specifically includes: based on the characteristics of the ship itself, its shape is a long strip, and considering the ability to clearly describe the target's outer contour elements under the multi-scale influence of remote sensing images, three features are selected: compactness, aspect ratio, and rectangularity. The calculation formula is as follows:

[0054] (1) Firmness:

[0055]

[0056] (2) Aspect ratio:

[0057]

[0058] (3) Rectangularity:

[0059]

[0060] Where A represents the area of ​​the target area, p represents the perimeter of the target area, l, w, and S represent the length, width, and area of ​​the circumscribed rectangle of the target area, respectively.

[0061] Furthermore, the step 3 specifically includes: focusing on the lateral connections of the low-level feature layers, performing feature enhancement on the low-level feature layers, and removing the top-level connections while ensuring the computational complexity and speed.

[0062] Furthermore, the step 4 specifically includes: the calculation formula for designing the anchor frame according to its shape characteristics is:

[0063]

[0064] Where m refers to the number of feature maps, S k Indicates the ratio of the anchor box size to the image, S mt浔 and S max Indicates the minimum and maximum values ​​of the ratio,

[0065] The aspect ratio calculation formula of the anchor box is:

[0066]

[0067]

[0068] in s k It is calculated by the previous formula, adding a default box scale for the case where the aspect ratio is 1 The default coordinate center of the box is |f k | is the size of the corresponding feature map, i,j∈[0,|f k |).

[0069] Furthermore, the step 5 specifically includes: introducing the focus classification loss as the loss function of the YOLOv5 algorithm to calculate the regression convergence formula as follows:

[0070]

[0071] Where: L loc (b * ,b) is the positioning loss function, and its specific calculation method is: b is the predicted value, b * is the true label box value, is the classification loss function, which is calculated as follows: t i,c For the correct forecast probability, is the background prediction probability, a t and r are kernel parameters.

[0072] Example

[0073] During implementation, the DOTA dataset was selected as the data sample for verifying this patent, and suitable remote sensing images were selected from this dataset for this implementation process. After image preprocessing and annotation, a total of 6,000 images were selected, including 4,800 images (80%) for the training set, 900 images (15%) for the validation set, and 300 images (5%) for the test set. mAP (mean Average Precision) was used as the evaluation indicator. The specific processing steps of this patented method are as follows:

[0074] (1) Dataset land and sea separation. The dataset is segmented using the OTSU threshold segmentation method to obtain the sea area image.

[0075] (2) Shape feature extraction and fusion. Three shape features, namely compactness, aspect ratio, and rectangularity, are extracted from the dataset. The multi-scale connection feature fusion is performed on the underlying pyramid network structure in combination with the image.

[0076] (3) Anchor frame design. For each pixel in the feature maps of different sizes, a ship target anchor frame of a predetermined size and number is generated. An anchor frame is generated in the feature map of each scale according to the formula. According to the shape characteristics of the ship, the aspect ratio of the anchor frame is set to {{1, 2, 1 / 2}}. The maximum and minimum scales of the anchor frame are determined by the median of the ship target area to the image area.

[0077] (4) Input the data set and train the deep learning model. Through the labeled data set, regression is performed according to the loss function to obtain the model parameters and generate the training model. The maximum number of training rounds is set to 50,000, the initial learning rate is 0.0015, the number of learning rate reduction rounds is 5,000, the learning rate adjustment parameter is set to 0.12, and the regularization weight decay parameter is 0.0005;

[0078] (5) Input data and identify the ship. For each generated anchor frame, the total loss function value is calculated using the classification focus loss function and the regression loss function to obtain the position and confidence of the anchor frame;

[0079] (6) Statistical test and validation sample accuracy and effect diagram. Generate Table 1 based on the results.

[0080] Table 1

[0081]

[0082] This paper addresses the shortcomings of this model for ship recognition. Firstly, by combining prior knowledge of ships and the sea area, the proposed method uses OTSU threshold segmentation to constrain the recognition scene and improve the error rate. Secondly, the proposed method optimizes the anchor boxes and loss function based on the ship's shape characteristics, improving the model's generalization performance and enhancing ship recognition accuracy.

Claims

1. A method for ship target recognition in remote sensing images based on deep learning with threshold constraints, characterized by: The steps include: Step 1: Use the OTSU threshold segmentation method to perform threshold segmentation on the remote sensing image to separate land and sea; Step 2: Extract shape features from the sea area remote sensing image after land and sea separation; Step 3: Based on the deep learning YOLOv5 algorithm and combined with the feature pyramid structure, the underlying pyramid network structure is connected and fused at multiple scales; The step 3 specifically includes: focusing on the horizontal connections of the lower feature layers, enhancing their features, and removing the top-level connections while ensuring the computational complexity and speed; Step 4: Design the anchor frame based on the shape characteristics of the ship target in the remote sensing image; The step 4 specifically includes: the calculation formula for designing the anchor frame according to its shape characteristics is: Where m refers to the number of feature maps, S k Indicates the ratio of the anchor box size to the image, S min and S max Indicates the minimum and maximum values ​​of the ratio, The aspect ratio calculation formula of the anchor box is: in s k It is calculated by the previous formula, adding a default box scale for the case where the aspect ratio is 1 The default coordinate center of the box is |f k | is the size of the corresponding feature map, i,j∈[0,|f k |); Step 5: Introduce focused classification loss as the loss function of the YOLOv5 algorithm for regression convergence; Step 6: Based on the improved YOLOv5 algorithm, input the pre-labeled ship samples for training to obtain a trained deep learning model; Step 7: Perform ship target recognition on remote sensing images based on the trained model.

2. The method for ship target recognition based on deep learning remote sensing images based on threshold constraints according to claim 1 is characterized in that: The step 1 specifically includes: determining a threshold based on the grayscale difference between the ocean and the land using the threshold value, and then separating the ocean and the land using the selected grayscale value as the threshold value: The OTSU threshold segmentation method uses the inter-class method measurement, which is calculated as follows: s 2 (T)=W a (m a -m) 2 +W b (m b -m) 2 Among them, σ 2 represents the inter-class method, W a Represents the ratio of the target area to the total image area, μ a Represents the mean value of all pixels in the target area, W b Represents the ratio of the background area to the total image area, μ b Represents the average value of pixels in the background area, μ represents the grayscale average of all pixels in the full image, T traverses from 0 to 255, so that σ 2 The maximum value of T is the optimal threshold for image segmentation.

3. The method for ship target recognition based on deep learning remote sensing images based on threshold constraints according to claim 2 is characterized in that: The step 2 specifically includes: based on the characteristics of the ship itself, its shape is long and narrow, and considering the ability to clearly describe the target's outer contour elements under the multi-scale influence of remote sensing images, three features are selected: compactness, aspect ratio, and rectangularity. The calculation formula is as follows: (1) Firmness: (2) Aspect ratio: (3) Rectangularity: Where A represents the area of ​​the target area, p represents the perimeter of the target area, l, w, and S represent the length, width, and area of ​​the circumscribed rectangle of the target area, respectively.

4. The method for ship target recognition based on deep learning remote sensing images based on threshold constraints according to claim 3 is characterized in that: The step 5 specifically includes: introducing the focus classification loss as the loss function of the YOLOv5 algorithm to calculate the regression convergence formula as follows: Where: L loc (b * ,b) is the positioning loss function, and its specific calculation method is: b is the predicted value, b * is the true label box value, is the classification loss function, which is calculated as follows: t i,c For the correct forecast probability, is the background prediction probability, a t and r are kernel parameters.

Images