A method and system for detecting a directional marine vessel

Abstract

The present application discloses a method and system for directed detection of marine ships, and specifically relates to the field of image analysis technology. The present application extracts remote sensing image features through multiple convolution kernels of different sizes and depths, and fuses them through a spatial attention selection mechanism, thereby effectively improving the accuracy of feature extraction; and by introducing angle cosine mapping encoding to process angle information in target detection, the problems of angle discontinuity and target-like squareness in the prior art are solved, and the boundary problem in directed target detection is avoided, thereby ultimately improving the efficiency and accuracy of directed ship detection.

Description

Technical Field

[0001] The present application belongs to the field of image analysis technology, and in particular, relates to a method and system for directional detection of ships based on remote sensing images. Background Art

[0002] Ships are important military targets at sea, and detecting marine ships is of great significance to both maritime transportation and military security. Traditional ship detection methods are mainly based on artificially extracted features of remote sensing images, such as edges and textures. This method is time-consuming and laborious. Due to the complexity and variability of the actual marine environment, the robustness and generalization ability of such methods are often unsatisfactory, and it is difficult to achieve automatic and accurate ship recognition by the system.

[0003] With the rise and progress of deep learning technology, the directional detection method of marine remote sensing ships based on deep learning has been increasingly studied and applied. Existing directional detection methods for marine ships often unify the task into the general remote sensing target detection framework, using a fixed-size convolution kernel to extract remote sensing image features, ignoring the size difference of the ship itself compared to targets such as vehicles and buildings, and the large horizontal and vertical ratios, which leads to missed target detection. In addition, in the current directional detection method, the angle information is generally converted into classification by angle regression to deal with the periodic boundary problem of the direction angle itself, but when the shape of the detected object approaches a square, this method will fail, resulting in serious errors and affecting the performance of the detector. Summary of the invention

[0004] In order to solve the technical problems of target detection errors and inability to detect nearly square ships in the prior art directed ship detection methods, the present application proposes a method and system for constructing a marine ship directed detection model.

[0005] In order to achieve the above objectives, in a first aspect, the present application provides a method for constructing a directional detection model for marine ships, which specifically includes:

[0006] Step S1: pre-acquire remote sensing images of ocean ships, and pre-process the ship images to obtain image training data sets;

[0007] Step S2: constructing a marine ship directional detection model, the model comprising: a feature extraction module, a feature fusion module and a detection regression module;

[0008] Step S3: input the preprocessed image data set into a feature extraction module, which extracts features of receptive fields of different scales in the image through a multi-scale convolution kernel, and obtains a feature map in combination with spatial attention weighted selection;

[0009] Step S4: input the obtained feature map information into a feature fusion module, which constructs a feature pyramid by top-down and lateral connections and combines different feature layers, fuses the bottom-level high-resolution features with the top-level low-resolution features, and generates a feature representation with rich semantic information;

[0010] Step S5: input the feature representation into the detection regression module, perform positioning regression and classification on the feature map, perform cosine phase shift encoding on the input angle information, calculate the model loss function, perform back propagation to train and obtain model parameters.

[0011] In order to enhance the detection capability of multi-scale targets, the feature extraction module in step S3 extracts features of receptive fields of different scales in the image through multi-scale convolution kernels, including:

[0012] The size of the input image F is 1024×1024, and the size of the i-th depth convolution kernel in the multi-scale convolution is k i , expansion rate e i And the receptive field size R i The definition is as follows:

[0013] k i-1 ≤k i ; e1=1,e i-1 <e i ≤R i-1

[0014] R1=k1,R i =e i (k i -1)+R i-1 .

[0015] Furthermore, the step S3 combines the spatial attention weighted selection to obtain the feature map, including:

[0016] Through a series of decomposition convolutions with different receptive fields, the feature information U of the image is obtained according to the image F i , the specific formula is:

[0017] U0=F

[0018] U i+1 =dw(U i )

[0019] Among them dw(U i ) is a convolution kernel with k i , expansion rate e i Depth convolution;

[0020] The features obtained by convolution kernels of different sizes with different receptive fields are connected to obtain the connected feature U, and then the channel-based average pooling AvgPool and maximum pooling operation MaxPool are used to extract the spatial relationship, and the pooled features are converted into a spatial attention map through the convolution layer.

[0021] For each spatial attention map The sigmoid activation function is applied to obtain a separate spatial selection mask, and then each spatial attention map is selected, weighted and fused through the corresponding spatial selection mask and convolution layer to obtain the spatial attention S. The corresponding formula is as follows:

[0022]

[0023] Where σ() represents the sigmoid activation function and X is the number of convolution kernels.

[0024] The feature map O finally output by the feature extraction module is the item-by-item product of the input image feature F and the spatial attention S:

[0025] O=F·S

[0026] In order to adapt to the situation that the rectangular frame is close to a square and the angle information is solved in a one-to-many manner, the cosine phase shift encoding of the input angle information in step S5 includes:

[0027] For the input angle α, since the angle period of the rectangular frame is π, and the period of the cosine function is 2π, the input angle is first mapped:

[0028] β=2α

[0029] One value of the cosine function corresponds to multiple angles. In order to obtain the correct angle value in the decoding stage, the angle needs to be phase shifted multiple times in the encoding stage so that one angle value has a unique encoding sequence:

[0030]

[0031] Where step is the number of phase shifts, which can be adjusted by setting model hyperparameters, n = 1, 2, 3, …, step.

[0032] Furthermore, the step S5 of performing cosine phase shift encoding on the input angle information also includes: the decoding formula of the angle information is:

[0033]

[0034] When training the model, it is necessary to determine the loss function. The calculation of the loss function in step S5 includes:

[0035] L=w1L ang +w2L cls +w3L box

[0036] L ang is the loss of the angle branch, L cls is the loss of the classification branch, L box is the prediction box regression loss, w1, w2, w3 are weighted respectively, which can be set by model hyperparameters during training; among them, the loss L of the angle branch ang =|GT-Pred|, GT is the labeled value of the image angle information, and Pred is the angle value predicted by the model.

[0037] In a second aspect of the present application, the present application provides a method for directional detection of marine ships, comprising:

[0038] Acquire remote sensing images of ocean ships to be detected;

[0039] Inputting the remote sensing image of the ocean ship to be detected into the model constructed by the method for constructing a directional detection model of an ocean ship described in any one of the first aspects;

[0040] Output the position and angle information of ocean ship targets.

[0041] In a third aspect of the present application, the present application provides a marine ship directional detection system for implementing the marine ship directional detection method in the second aspect, the system comprising:

[0042] An image preprocessing module is used to process the ocean remote sensing ship images input into the system;

[0043] The image processing module is used to perform directional detection on the preprocessed ship image and output the position and angle information of the ship; the image processing module includes a feature extraction module, a feature fusion module, and a detection regression module; the feature extraction module uses a multi-scale convolution kernel to extract feature information of different scales in the image, and the detection regression module uses angle cosine mapping encoding to process the angle information to solve the angle discontinuity problem and the target square problem in directional target detection.

[0044] The result output module is used to visualize the position and angle information of the ship and output it to the user.

[0045] In a fourth aspect, the present application provides a computer device comprising a memory, a processor, and program instructions stored in the memory for execution by the processor, wherein the processor executes the program instructions to implement the steps of any one of the methods described in the first aspect.

[0046] In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method described in any one of the first aspect and the second aspect is implemented.

[0047] In summary, based on the inventive concept in this application, the following technical effects can be achieved:

[0048] (1) This application extracts remote sensing image features through multiple convolution kernels of different sizes and depths, and fuses them through a spatial attention selection mechanism to effectively improve the accuracy of feature extraction. Specifically, a new feature extraction module is first designed to extract targets of different sizes from remote sensing images, and spatial attention is used to calculate the weights of features obtained from different receptive fields, so that better features of marine ship targets of different sizes can be obtained, effectively improving the accuracy of target classification and positioning.

[0049] (2) The present application processes the angle information by introducing angle cosine mapping encoding in target detection, thereby solving the problems of angle discontinuity and square-like targets in the prior art, avoiding boundary problems in directional target detection, and ultimately improving the efficiency and accuracy of remote sensing directional ship detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Flow chart of the model construction method in this application.

[0051] Figure 2 This is the overall network architecture diagram of the model in this application.

[0052] Figure 3 This is the flow chart of the feature extraction module in this application.

[0053] Figure 4 This is a diagram of the multi-scale convolution kernel layer architecture in this application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to further clearly and completely describe the technical solutions in the embodiments of the present application. It should be noted that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0055] In order to make the invention purpose, technical scheme and advantages of this application clearer, the technical scheme of the present invention is described clearly and completely in conjunction with the accompanying drawings and specific implementation schemes. It should be pointed out that for ordinary technicians in this field, several changes and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of this application.

[0056] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings:

[0057] See attached Figure 1 Embodiment 1 of the present application provides a method for constructing a directional detection model for marine ships, which specifically includes:

[0058] Step S1: Acquire remote sensing images of ocean ships in advance, and preprocess the ship images to obtain an image training data set.

[0059] Remote sensing images are commonly used image data in ship target detection. In order to achieve the accuracy of directional detection of ship targets, marine ship images are pre-acquired as target detection data. Since the data concerned by target detection is relatively limited, the image is first segmented in image preprocessing. In order to improve the accuracy and efficiency of target detection, the acquired images are subjected to data enhancement and standardization operations to generate a preprocessed image dataset. It should be noted that this part of the image dataset is divided into a training dataset and a test dataset. The training dataset and the test dataset are used to train the model constructed later, improve the classification and positioning accuracy of the target detection model, and further improve the target detection accuracy of the model.

[0060] Step S2: constructing a directed detection model for marine ships, the model comprising: a feature extraction module, a feature fusion module and a detection regression module.

[0061] In this embodiment, the target detection model structure (such as Figure 2 The directional detection model of marine ships is constructed based on the directional detection model of marine ships (as shown in Figure 1). The directional detection model of marine ships includes feature extraction module, feature fusion module and detection regression module. The feature extraction module extracts features with multi-scale receptive fields through multi-scale feature selection. The feature fusion module fuses low-level high-resolution features with high-level low-resolution features. The detection regression head module classifies and locates the input features and outputs the final detection results.

[0062] For general target detection feature extraction modules, such as Resnet, ConvNeXt, etc., although they have made some progress by utilizing stacked residual structures and increasing the size of convolution kernels, they ignore valuable information such as the surrounding environment and the shape, direction and other features of objects. For marine remote sensing ship images, the direction and size of the ship itself contain important information. In order to obtain the prior information contained in marine ships of different sizes, the multi-scale feature extraction module proposed in this application is as follows Figure 3As shown in the figure, its structure is similar to ConvNeXt. Each module consists of two residual structure sub-blocks: a spatial attention selection sub-block and a feedforward network sub-block. The attention selection sub-block dynamically adjusts the receptive field as needed. The feedforward network sub-block is used for channel mixing and feature refinement. It consists of a series of full convolutional layers, deep convolutions, and ReLU activation functions.

[0063] Step S3: The preprocessed image data set is input into a feature extraction module, which extracts features of receptive fields of different scales in the image through a multi-scale convolution kernel, and obtains a feature map in combination with spatial attention weighted selection.

[0064] After the marine ship directional detection model is constructed, the preprocessed image data set is input into the feature extraction module. The feature extraction module in this embodiment extracts the features of the receptive fields of different scales in the image through the multi-scale convolution kernel. The multi-scale convolution kernel needs to be pre-defined. Specifically, the size of the input image F is 1024×1024, and the first i The depth of the convolution kernel size is k i , expansion rate e i And the receptive field size R i The definition is as follows:

[0065] k i-1 ≤k i ; e1=1,e i-1 <e i ≤R i-1

[0066] R1=k1,R i =e i (k i -1)+R i-1 .

[0067] Setting the size of the convolution kernel and increasing the dilation rate can ensure that the receptive field increases rapidly during training. In order to prevent the introduction of gaps in the convolution, the upper limit of the dilation rate can be set through the model hyperparameters.

[0068] The advantage of this design is that it directly generates multiple features with different receptive fields, which facilitates the selection of convolution kernels. Compared with directly increasing the convolution kernel, multi-scale convolution fusion also reduces the number of calculation parameters.

[0069] Through a series of decomposition convolutions with different receptive fields, the feature information U of the image is obtained according to the image F i , the specific formula is:

[0070] U0=F

[0071] U i+1 =dw(Ui )

[0072] Among them dw(U i ) is a convolution kernel with k i , expansion rate e i Specifically, suppose there are X convolution kernels, each of which consists of a convolution layer Conv of size 1. 1×1 Further processing:

[0073] dw(U i )=Conv 1×1 (U i ).

[0074] See also Figure 4 As shown in the figure, after processing the image F according to convolution kernels of different scales and obtaining the feature information of the image, the features obtained by convolution kernels of different sizes with different receptive fields are connected to obtain the connected features U, and then the channel-based average pooling AvgPool and maximum pooling operation MaxPool are used to extract the spatial relationship, and the pooled features are converted into a spatial attention map through the convolution layer.

[0075] For each spatial attention map The sigmoid activation function is applied to obtain a separate spatial selection mask, and then each spatial attention map is selected, weighted and fused through the corresponding spatial selection mask and convolution layer to obtain the spatial attention S. The corresponding formula is as follows:

[0076]

[0077] Where σ() represents the sigmoid activation function and X is the number of convolution kernels.

[0078] The feature map O finally output by the feature extraction module is the term-by-term product of the input image feature F and the spatial attention S, and the specific formula is: O=F·S.

[0079] Step S4: Input the obtained feature map information into the feature fusion module. The feature fusion module constructs a feature pyramid by top-down and lateral connections and combines different feature layers, fuses the bottom-level high-resolution features with the high-level low-resolution features, and generates a feature representation with rich semantic information.

[0080] After the multi-scale feature extraction module obtains the feature map information of the image, since the scales and resolutions of each feature information are different, in order to optimize the resolution and feature information of the acquired feature information, this embodiment loads the feature map information into the multi-scale feature fusion module, and constructs a feature pyramid by top-down and lateral connections and combining different feature layers, and fuses the bottom-level high-resolution features with the high-level low-resolution features to generate features with rich semantic information.

[0081] Step S5: input the feature representation into the detection regression module, perform positioning regression and classification on the feature map, perform cosine phase shift encoding on the input angle information, calculate the model loss function, perform back propagation to train and obtain model parameters.

[0082] According to the target detection model, the feature representation after feature fusion is input into the detection regression module, and the detection regression module classifies and locates the target according to the input feature representation information. However, when angle information is involved, the periodicity of the angle often causes boundary problems. When the input angle is 0° and 180°, since operations such as convolution are continuous functions, the predicted angle cannot jump from 180° to 0°, but gradually decreases to 0°. At this time, the accuracy of the model is seriously reduced. In the current directed detection method, the angle information is generally converted into classification to deal with the periodic boundary problem of the direction angle itself. Because the classification box is generally a rectangle, including angle information in the definition of the prediction box can indeed solve the problem of positive and negative angle rotation. However, when the shape of the detected object approaches a square, this method will fail, resulting in serious errors and affecting the performance of the detector.

[0083] In order to prevent boundary problems in oriented target detection in this embodiment, the cosine value of the angle is predicted so that the output of the network is a continuous quantity.

[0084] In order to adapt to the situation where the rectangular frame is close to a square and the angle information is solved with errors, the cosine phase shift encoding of the input angle information in step S5 includes:

[0085] For the input angle α, since the angle period of the rectangular frame is π, and the period of the cosine function is 2π, the input angle is first mapped:

[0086] β=2α

[0087] One value of the cosine function corresponds to multiple angles. In order to obtain the correct angle value in the decoding stage, the angle needs to be phase shifted multiple times in the encoding stage so that one angle value has a unique encoding sequence:

[0088]

[0089] Where step is the number of phase shifts, which can be adjusted by setting model hyperparameters, n = 1, 2, 3, …, step.

[0090] Furthermore, the step S5 of performing cosine phase shift encoding on the input angle information also includes: the decoding formula of the angle information is:

[0091]

[0092] When training the model, it is necessary to determine the loss function. The calculation of the loss function in step S5 includes:

[0093] L=w1L ang +w2L cls +w3L box

[0094] L ang is the loss of the angle branch, L cls is the loss of the classification branch, L box is the prediction box regression loss, w1, w2, w3 are weighted respectively, which can be set by model hyperparameters during training; among them, the loss L of the angle branch ang =|GT-Pred|, GT is the labeled value of the image angle information, and Pred is the angle value predicted by the model.

[0095] The constructed ocean ship directed detection model is trained and tested according to the minimum loss function, and the model parameters are updated in reverse according to the test results. After multiple rounds of iterations, a converged model for ocean remote sensing ship directed target detection is obtained.

[0096] In the second embodiment of the present application, the present application provides a method for detecting a directional marine ship, which uses the model constructed in any one of the methods for constructing a directional marine ship detection model in the first embodiment to perform directional detection on the ship. Specifically, the method includes:

[0097] Acquire remote sensing images of ocean ships to be detected;

[0098] Inputting the remote sensing image of the ocean ship to be detected into the model constructed by the method for constructing a directional detection model of an ocean ship described in any one of the first aspects;

[0099] Output the position and angle information of ocean ship targets.

[0100] In addition, the acquired remote sensing images of the ocean ships to be detected are preprocessed to obtain preprocessed images, which are input into the trained model to perform target detection on the ocean ship images to complete the classification and directional positioning of the ships.

[0101] In the third embodiment of the present application, the present application provides a marine ship directional detection system, the system comprising:

[0102] An image preprocessing module is used to process the ocean remote sensing ship images input into the system;

[0103] The image processing module is used to perform directional detection on the preprocessed ship image and output the position and angle information of the ship; the image processing module includes a feature extraction module, a feature fusion module, and a detection regression module; the feature extraction module uses a multi-scale convolution kernel to extract feature information of different scales in the image, and the detection regression module uses angle cosine mapping encoding to process the angle information to solve the angle discontinuity problem and the target square problem in directional target detection.

[0104] The result output module is used to visualize the position and angle information of the ship and output it to the user.

[0105] It should be noted that the image processing module in the marine ship directional detection system in this embodiment can be a model constructed based on the ship directional detection model construction method in the first embodiment.

[0106] In embodiment 4 of the present application, the present application provides a computer device, which includes a memory, a processor, and program instructions stored in the memory for execution by the processor, wherein the processor executes the program instructions to implement the steps in the method described in any one of embodiments 1 and 2.

[0107] In the fifth embodiment of the present application, the present application provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method described in any one of the first and second embodiments is implemented.

[0108] The present application discloses a method and system for directed detection of marine ships, which extracts remote sensing image features through multiple convolution kernels of different sizes and depths, and fuses them through a spatial attention selection mechanism, effectively improving the accuracy of feature extraction; and by introducing angle cosine mapping encoding to process angle information in target detection, solves the problems of angle discontinuity and target-like squareness in the prior art, avoids boundary problems in directed target detection, and ultimately improves the efficiency and accuracy of directed ship detection.

Claims

1. A method for constructing a directional detection model for marine ships, characterized in that: The method comprises: Step S1: pre-acquire remote sensing images of ocean ships, and pre-process the ship images to obtain image training data sets; Step S2: constructing a directed detection model for marine ships, the model comprising: a feature extraction module, a feature fusion module and a detection regression module; Step S3: input the preprocessed image data set into a feature extraction module, which extracts features of receptive fields of different scales in the image through a multi-scale convolution kernel, and obtains a feature map in combination with spatial attention weighted selection; Step S4: input the obtained feature map information into a feature fusion module, which constructs a feature pyramid by top-down and lateral connections and combines different feature layers, fuses the bottom-level high-resolution features with the top-level low-resolution features, and generates a feature representation with rich semantic information; Step S5: input the feature representation into the detection regression module, perform positioning regression and classification on the feature map, perform cosine phase shift encoding on the input angle information, calculate the model loss function, perform back propagation to train and obtain model parameters.

2. The method for constructing a directional detection model for marine vessels according to claim 1, characterized in that: The feature extraction module in step S3 extracts features of receptive fields of different scales in the image through a multi-scale convolution kernel, including: The size of the input image F is 1024×1024, and the size of the i-th depth convolution kernel in the multi-scale convolution is k i , expansion rate e i And the receptive field size R i The definition is as follows: k i-1 ≤k i ;e1=1,e i-1 <e i ≤R i-1 ; R1=k1,R i =e i (k i -1)+R i-1 。 3. The method for constructing a directional detection model for a marine vessel according to claim 2, characterized in that: The feature map obtained by combining the spatial attention weighted selection in step S3 includes: Through a series of decomposition convolutions with different receptive fields, the feature information U of the image is obtained according to the image F i , the specific formula is: U0=F; U i+1 =dw(U i ); Among them dw(U i ) is a convolution kernel with k i , expansion rate e i Depth convolution; The features obtained by convolution kernels of different sizes with different receptive fields are connected to obtain the connected feature U, and then the channel-based average pooling AvgPool and maximum pooling operation MaxPool are used to extract the spatial relationship, and the pooled features are converted into a spatial attention map through the convolution layer. For each spatial attention map, the sigmoid activation function is applied to obtain a separate spatial selection mask. Then each spatial attention map is selected, weighted and fused through the corresponding spatial selection mask and convolution layer to obtain the spatial attention S. The corresponding formula is as follows: Where σ() represents the sigmoid activation function, and X is the number of convolution kernels; The feature map O finally output by the feature extraction module is the term-by-term product of the input image feature F and the spatial attention S, and the specific formula is: O=F·S.

4. The method for constructing a directional detection model for a marine vessel according to claim 3, characterized in that: The step S5 of performing cosine phase shift encoding on the input angle information comprises: For the input angle α, since the angle period of the rectangular frame is π, and the period of the cosine function is 2π, the input angle is first mapped: β＝2α； One value of the cosine function corresponds to multiple angles. In order to obtain the correct angle value in the decoding stage, the angle needs to be phase shifted multiple times in the encoding stage so that one angle value has a unique encoding sequence: x step =cosβ+2nπstep; Where step is the number of phase shifts, which can be adjusted by setting model hyperparameters, n = 1, 2, 3, ..., step.

5. The method for constructing a directional detection model for a marine vessel according to claim 4, characterized in that: The step S5 of performing cosine phase shift encoding on the input angle information also includes: the decoding formula of the angle information is:

6. The method for constructing a directional detection model for a marine vessel according to claim 5, characterized in that: Calculating the loss function in step S5 includes: L=w1L ang +w2L cls +w3L box ; L ang is the loss of the angle branch, L cls is the loss of the classification branch, L box is the prediction box regression loss, w1, w2, w3 are the weights taken by each, which can be set by the model hyperparameters during training; among them, the loss L of the angle branch ang =|GT-Pred|, GT is the labeled value of the image angle information, and Pred is the angle value predicted by the model.

7. A method for detecting a directional marine vessel, characterized in that: include: Acquire remote sensing images of ocean ships to be detected; Inputting the remote sensing image of the ocean ship to be detected into the model constructed by the method for constructing a directional detection model of an ocean ship as described in any one of claims 1 to 6; Output the position and angle information of ocean ship targets.

8. A marine vessel directional detection system for implementing the method described in claim 7, characterized in that: include: An image preprocessing module is used to process the ocean remote sensing ship images input into the system; An image processing module is used to perform directional detection on the preprocessed ship image and output the position and angle information of the ship; the image processing module includes a feature extraction module, a feature fusion module, and a detection regression module; the feature extraction module uses a multi-scale convolution kernel to extract feature information of different scales in the image, and the detection regression module uses angle cosine mapping encoding to process the angle information to solve the angle discontinuity problem and the square-like problem of the target in directional target detection; The result output module is used to visualize the position and angle information of the ship and output it to the user.

9. A computer device comprising a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images