A method for ship target detection in optical remote sensing images
By using the D-RFB module and the elastic hourglass module to extract feature information in optical remote sensing images, and build an elastic hourglass path aggregation network, solve the problem of low target detection accuracy and insufficient information of the optical remote sensing image, and achieve higher information. The detection accuracy and less missed inspection.
Patent Information
- Application Number
- CN202210249333.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-14
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2042-03-14
AI Technical Summary
The existing target detection method When detecting ship targets in optical remote sensing images, there are many missing targets for small and medium-sized ship targets, and there are fewer pixels for small-sized ship targets, and the location information and semantic information are insufficient.
Use the D-RFB module to further extract the feature diagrams in optical remote sensing images, provide different standards to feel the wild, and extract richer semantic information and location information through the elastic hourglass module. Based on these modules, the elastic hourglass path aggregation network is constructed to promote the dissemination of characteristic information between different layers and reduce information loss.
The detection accuracy of small and medium -sized ship targets has been improved, the missed and misunderstanding is reduced, and the detection capacity of the target of small -sized ships is enhanced.
Smart Images
Figure CN114612781B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of optical remote sensing image detection, and in particular relates to a method for detecting ship targets in optical remote sensing images. Background Art
[0002] With the development of remote sensing technology, the amount of ground image data acquired by remote sensing satellites has exploded, and how to process these remote sensing data is imminent. Ship targets in optical remote sensing images have the characteristics of diverse scales, complex background environments, dense distribution, and mostly small targets. When the existing target detection algorithms are applied to remote sensing image ship target detection, the missed detection rate and false detection rate of small and medium-sized targets are high.
[0003] With the development of deep learning, a large number of optical remote sensing image detection methods based on deep learning have been proposed by researchers. However, due to the scale characteristics of ships (most ships are relatively small targets) and distribution characteristics (ships are densely distributed), many methods have serious problems of missed detection and false detection when applied to ship target detection in optical remote sensing images. Therefore, there is still a lot of room for improvement and research significance for ship target detection algorithms in optical remote sensing images based on deep learning. Summary of the invention
[0004] The present invention proposes a method for detecting ship targets in optical remote sensing images in view of the problems that existing target detection methods have poor detection accuracy for ship targets in optical remote sensing images, many small and medium-sized ship targets are missed, and small-sized ship targets have fewer pixels and insufficient position information and semantic information.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A method for detecting ship targets in optical remote sensing images, comprising:
[0007] Step 1: Construct a D-RFB module, and further extract information from the feature maps extracted from the optical remote sensing image based on the D-RFB module, while providing appropriate receptive fields for feature maps of different scales;
[0008] Step 2: Build an elastic hourglass module, and build an elastic hourglass path aggregation network based on the elastic hourglass module;
[0009] Step 3: Based on the elastic hourglass path aggregation network, feature maps of different scales are fused and the final ship target detection result is output.
[0010] Furthermore, the D-RFB module uses dilated convolutions with different expansion rates to enhance the receptive field of the feature map, wherein each dilated convolution branch contains a different number of 3×3 ordinary convolutions to extract feature information.
[0011] Furthermore, the D-RFB module includes a convolution branch and a residual branch; in the convolution branch, the input feature map is first subjected to a 1×1 convolution to increase the dimension, and then different numbers of convolution operations are used to extract information to enhance the receptive field. Each convolution branch contains a different number of 3×3 ordinary convolutions and hole convolutions with different expansion rates, and the network structure is gradually deepened. Each convolution branch provides a receptive field of different scales for the feature map. After the convolution extracts the features, the feature information of different dimensions is fused together. Finally, each convolution branch is subjected to a 1×1 convolution to reduce the dimension and is residually connected to the input feature map.
[0012] Furthermore, the elastic hourglass module includes an elastic branch and an hourglass module; the hourglass module extracts information in a high dimension and performs identity mapping and spatial transformation at the same time; the elastic branch performs scale transformation of the feature map through upsampling and downsampling functions, and the scale transformation process includes a series of convolution operations.
[0013] Furthermore, the elastic hourglass module includes an hourglass module and an elastic branch; the hourglass module includes a residual branch and a trunk convolution branch, the input feature map first undergoes a 3×3 depthwise separable convolution, and then uses two 1×1 convolutions to perform dimensionality transformation, first reducing the dimension and then increasing the dimension, the ratio of the reduced dimension to the increased dimension remains consistent, and finally undergoes a 3×3 depthwise separable convolution; the residual branch directly performs a residual connection from the input feature map and the output feature map; the elastic branch first undergoes downsampling through a normal convolution with a size of 3 and a step size of 2, and then uses the trunk convolution branch of the hourglass module to extract information, and finally uses upsampling to restore the feature map size.
[0014] Furthermore, the structure of the hourglass module is expressed by formula (1):
[0015] G=φ e (φ r (F))+F (1)
[0016] F represents the input tensor, G represents the output tensor, φ e and φ r Represent the point convolution of channel expansion and reduction respectively.
[0017] Furthermore, the structure of the elastic hourglass module is as shown in formula (2):
[0018] F(x)=U ri (g(D ri(x)))+g(x)+x (2)
[0019] Where x represents the input of the elastic hourglass module, g() is the main convolution branch operation of the hourglass module, and D ri () represents the downsampling function, U ri () represents the upsampling function.
[0020] Furthermore, the elastic hourglass path aggregation network fuses feature maps of different scales through upsampling, downsampling and parallel branches: in the elastic hourglass path aggregation network, the high-level feature map containing semantic information is fused with the low-level position information after upsampling, and then the low-level feature map containing position information is fused with the high-level semantic information after downsampling. The fused feature maps are passed through two elastic hourglass modules for information extraction, and finally the feature maps of different scales are detected.
[0021] Compared with the prior art, the present invention has the following beneficial effects:
[0022] The present invention aims to solve the problems that existing target detection methods have poor accuracy in detecting ship targets in optical remote sensing images and miss many small and medium-sized ship targets, and proposes a method for detecting ship targets in optical remote sensing images. The present invention proposes a D-RFB module to further extract information from feature maps extracted from optical remote sensing images, and provide suitable receptive fields for feature maps of different scales. Secondly, in order to solve the problem that small-sized ship targets have fewer pixels and insufficient position information and semantic information, an elastic hourglass module is proposed, which can extract richer semantic information and position information. At the same time, based on the elastic hourglass module, an elastic hourglass path aggregation network is proposed, which can promote the propagation of feature information between different layers, and reduce the loss of feature information caused during the propagation process. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 A basic flow chart of a method for detecting ship targets in optical remote sensing images according to an embodiment of the present invention;
[0024] Figure 2 : is a structural diagram of a D-RFB module according to an embodiment of the present invention, where + represents a residual connection;
[0025] Figure 3 This is a structural diagram of an elastic hourglass module according to an embodiment of the present invention, where + represents a residual connection;
[0026] Figure 4 This is a diagram of the elastic hourglass path aggregation network structure of an embodiment of the present invention, wherein ESG represents an elastic hourglass module. DETAILED DESCRIPTION
[0027] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:
[0028] like Figure 1 As shown, a method for detecting ship targets in optical remote sensing images comprises:
[0029] Step S101: construct a D-RFB module, and further extract information from the feature map extracted from the optical remote sensing image based on the D-RFB module, and provide a suitable receptive field for feature maps of different scales; As an implementable method, before step S101, the feature map can be extracted from the optical remote sensing image by CSPDarknet53;
[0030] Step S102: constructing an elastic hourglass module, and constructing an elastic hourglass path aggregation network based on the elastic hourglass module;
[0031] Step S103: Fusing feature maps of different scales based on the elastic hourglass path aggregation network and outputting the final ship target detection result.
[0032] Specifically, the problem of ship targets being difficult to detect in optical remote sensing images is that the receptive fields do not match and the semantic information is less. To solve the above problems, firstly, the present invention proposes a D-RFB module, which provides receptive fields of different scales for multi-scale ship targets through dilated convolutions with different expansion rates, and uses depthwise separable convolutions to reduce the amount of calculation. Secondly, we propose an elastic hourglass module, which performs identity mapping and spatial transformation at a high level, and a deeper network structure enables it to extract more semantic information, while the elastic mechanism can capture feature information of different scales through scaling strategies. Finally, an elastic hourglass path aggregation network is proposed based on the elastic hourglass module. The elastic hourglass path aggregation network is used for the fusion of feature maps of different scales, and can avoid the loss of feature information caused by the fusion process and retain more feature information.
[0033] (I) D-RFB module
[0034] The D-RFB module uses dilated convolutions with different expansion rates to enhance the receptive field of feature maps. Each dilated convolution branch contains a different number of 3×3 normal convolutions to extract feature information. At the same time, the existence of the residual branch avoids the gradient vanishing problem caused by the network being too deep. This module can extract richer semantic information while enhancing the receptive field of feature maps, improving the detection accuracy of small and medium-sized ship targets.
[0035] Figure 2The following is the structure diagram of the D-RFB module. This module mainly includes convolution branches and residual branches. In the convolution branch, the input feature map is first subjected to 1×1 convolution to increase the dimension, and then different numbers of convolution operations are used to extract information and enhance the receptive field. The three branches contain different numbers of 3×3 ordinary convolutions and dilated convolutions with different expansion rates, and the network structure gradually deepens. The three convolution branches can provide small, medium, and large scale receptive fields for the feature map from left to right. After extracting features through convolution, feature information of different dimensions is fused together to enrich the semantic information of the feature map. The introduction of depthwise separable convolution greatly reduces the amount of calculation and makes the convolution branch more lightweight. Finally, the convolution branch is subjected to 1×1 convolution to reduce the dimension and then is residually connected to the input feature map. The existence of the residual branch avoids the gradient vanishing problem caused by the network being too deep and accelerates the convergence of the network.
[0036] (II) Elastic hourglass module
[0037] The elastic hourglass module includes an elastic branch and an hourglass module. The hourglass module extracts information in high dimensions, performs identity mapping and spatial transformation at the same time, and alleviates the information loss problem caused by dimensional transformation. The elastic branch performs scale transformation of the feature map through upsampling and downsampling functions, and the scale transformation process includes a series of convolution operations.
[0038] Figure 3 The elastic hourglass module structure diagram. The elastic hourglass module mainly consists of two parts, the hourglass module and the elastic branch. The hourglass module contains a residual branch and a trunk convolution branch. The input feature map first passes through a 3×3 depth-separable convolution. Then two 1×1 convolutions are used for dimensional transformation. First reduce the dimension, then increase the dimension, the ratio of the reduced dimension to the increased dimension is consistent, and finally pass through a 3×3 depth-separable convolution. The hourglass module solves the problem of information loss caused by dimensional transformation by performing dimensional transformation in high dimensions. The residual branch directly performs residual connection from the input feature map and the output feature map.
[0039] Specifically, the construction of the hourglass module can be expressed by formula (1):
[0040] G=φ e (φ r (F))+F (1)
[0041] Without considering the depth of convolution and activation function, if the input tensor is The output tensor is where φ e and φ r Denote the point convolution of channel expansion and reduction, D f represents the feature map size, and M represents the number of feature map channels.
[0042] Table 1 shows the basic operation of the hourglass module, where t and s represent the channel ratios of expansion and reduction, respectively, Dwise Conv represents depthwise separable convolution, and N represents the number of feature map channels.
[0043] Table 1 Basic operations of the hourglass module
[0044]
[0045] The elastic branch is first downsampled by a normal convolution of size 3×3 and stride 2, and then a series of convolutions (hourglass module trunk convolution branch) are used to further extract information. Finally, upsampling is used to restore the size of the feature map.
[0046] The elastic hourglass module can not only capture feature information of different scales and dimensions, but also reduce the information loss problem caused by dimensional transformation due to its internal convolution operation. It effectively improves the detection accuracy of small and medium-sized ship targets. The construction of the elastic hourglass module is shown in formula (2):
[0047] F(x)=U ri (g(D ri (x)))+g(x)+x (2)
[0048] Where x represents the input of the module (and the residual branch), g() is the main convolution branch operation of the hourglass module, and D ri () represents the downsampling function, U ri () represents the upsampling function.
[0049] (III) Elastic Hourglass Path Aggregation Network
[0050] The elastic hourglass path aggregation network fuses feature maps of different scales through upsampling, downsampling and parallel branches. The high-level feature map contains rich semantic information and is fused with the low-level position information after upsampling. Then the low-level feature map contains rich position information and is fused with the high-level semantic information after downsampling. The fused feature map is passed through two elastic hourglass modules for information extraction. Finally, the feature maps of three scales are detected. Assuming that the size of the input image is 416×416, the elastic hourglass path aggregation network finally outputs 13×13, 26×26, and 52×52 feature maps, which are sent to the detection head for detection. Figure 4 The structure of the elastic hourglass path aggregation network is shown.
[0051] The multi-scale fusion of elastic hourglass path aggregation network is very important for ship detection. It can provide sufficient semantic information and location information for ship targets, greatly improving the accuracy of ship detection.
[0052] In summary, the present invention proposes a D-RFB module, which can extract feature information at different levels, and at the same time provide a suitable receptive field for the feature map through hole convolution with different expansion rates, avoiding the situation of missed detection and false detection of ship targets caused by mismatch of receptive fields. The present invention proposes an elastic hourglass module, which can reduce the information loss caused by dimensional transformation, and at the same time, the elastic branch can extract information from feature maps of different scales. The module can extract rich feature information, reduce the loss of feature information, and improve the detection accuracy of small-sized ships. On the basis of the elastic hourglass module, the present invention also proposes an elastic hourglass path aggregation network, which propagates and fuses feature information between different layers through upsampling and downsampling and parallel branches, and then uses the elastic hourglass module to further extract the fused feature information. The network reduces the information loss problem caused by the propagation of feature information of different layers, and at the same time promotes the fusion of feature information between different layers, which can effectively improve the accuracy of ship detection in optical remote sensing images.
[0053] The present invention aims to solve the problem that the existing target detection methods have poor accuracy in detecting ship targets in optical remote sensing images and miss many small and medium-sized ship targets, and proposes a method for detecting ship targets in optical remote sensing images. The present invention proposes a D-RFB module to further extract information from the feature map extracted from the optical remote sensing image by CSPDarknet53, and provide a suitable receptive field for feature maps of different scales. Secondly, in order to solve the problem that small-sized ship targets have fewer pixels and insufficient position information and semantic information, an elastic hourglass module is proposed, which can extract richer semantic information and position information. At the same time, based on the elastic hourglass module, an elastic hourglass path aggregation network is proposed, which can promote the propagation of feature information between different layers and reduce the loss of feature information caused by the propagation process.
[0054] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A method for detecting ship targets in optical remote sensing images, characterized in that: include: Step 1: Construct a D-RFB module, and further extract information from the feature maps extracted from the optical remote sensing image based on the D-RFB module, while providing appropriate receptive fields for feature maps of different scales; The D-RFB module includes a residual branch, a first convolution branch, a second convolution branch and a third convolution branch; the first convolution branch includes a 1×1 convolution and a 3×3 dilated convolution; the second convolution branch includes a 1×1 convolution, a 3×3 depth-separable convolution and a 3×3 dilated convolution; the third convolution branch includes a 1×1 convolution, two 3×3 depth-separable convolutions and a 3×3 dilated convolution; different convolution branches use 3×3 dilated convolutions with different expansion rates to enhance the receptive field of the feature map; after extracting features through convolution, feature information of different dimensions is fused together, and finally each convolution branch is subjected to a 1×1 convolution to reduce the dimension and perform a residual connection with the input feature map; Step 2: construct an elastic hourglass module, and construct an elastic hourglass path aggregation network based on the elastic hourglass module; the elastic hourglass module includes an elastic branch and an hourglass module; the hourglass module extracts information in a high dimension, and performs identity mapping and spatial transformation at the same time; the elastic branch performs scale transformation of the feature map through upsampling and downsampling functions, and the scale transformation process includes a series of convolution operations; The elastic hourglass module includes an hourglass module and an elastic branch; the hourglass module includes a residual branch and a trunk convolution branch, the input feature map first undergoes a 3×3 depthwise separable convolution, and then uses two 1×1 convolutions for dimensional transformation, first reducing the dimension and then increasing the dimension, the ratio of the reduced dimension and the increased dimension is kept consistent, and finally undergoes a 3×3 depthwise separable convolution; the residual branch directly performs a residual connection from the input feature map and the output feature map; the elastic branch first undergoes a normal convolution with a size of 3 and a step size of 2 for downsampling, and then uses the trunk convolution branch of the hourglass module to extract information, and finally uses upsampling to restore the size of the feature map; Step 3: Based on the elastic hourglass path aggregation network, feature maps of different scales are fused and the final ship target detection result is output.
2. The method for detecting ship targets in optical remote sensing images according to claim 1, characterized in that: The structure of the hourglass module is expressed by formula (1): G=φ e (f r (F))+F (1) F represents the input tensor, G represents the output tensor, φ e and φ r Represent the point convolution of channel expansion and reduction respectively.
3. The method for detecting ship targets in optical remote sensing images according to claim 1, characterized in that: The structure of the elastic hourglass module is shown in formula (2): F(x)=U ri (g(D ri (x)))+g(x)+x (2) Where x represents the input of the elastic hourglass module, g() is the main convolution branch operation of the hourglass module, and D ri () represents the downsampling function, U ri () represents the upsampling function.
4. The method for detecting ship targets in optical remote sensing images according to claim 1, characterized in that: The elastic hourglass path aggregation network fuses feature maps of different scales through upsampling, downsampling and parallel branches: in the elastic hourglass path aggregation network, the high-level feature map containing semantic information is fused with the low-level position information after upsampling, and then the low-level feature map containing position information is fused with the high-level semantic information after downsampling. The fused feature maps are passed through two elastic hourglass modules for information extraction, and finally the feature maps of different scales are detected.