Lightweight visible ship target detection method based on edge feature guidance

Abstract

The application provides a lightweight visible light ship target detection method based on edge feature guidance, relates to the technical field of ship detection image data processing, and comprises the following steps: collecting remote sensing satellite images, screening and labeling the images, and randomly distributing the images to obtain a training set and a verification set; a backbone network module comprises a plurality of Conv modules and C3k2 modules which are stacked with each other; a neck module comprises a convolution module with detail enhancement and a hierarchical pyramid module based on dynamic feature aggregation; in a head module, the features of all detection layers are subjected to independent convolution processing, and then subjected to feature transformation through a multi-branch detail enhancement convolution module; data enhancement is performed on the training set; and through a back propagation algorithm and a gradient descent optimization method, a trained ship target detection model is obtained. The application realizes lightweight and precision improvement of the detection head, enhances the robustness of the model to light changes, and balances the detection of small targets and large targets by fusing global semantic information and local detail features.

Description

Technical Field

[0001] The present application relates to the technical field of ship detection image data processing, and more specifically, to a lightweight visible light ship target detection method based on edge feature guidance. Background Art

[0002] Ship detection is of vital importance in the maritime domain. It is widely used in maritime traffic control, military reconnaissance, marine resource development, and other areas, playing a key role in ensuring maritime security and safeguarding national maritime rights and interests. In recent years, deep learning-based object detection technology has rapidly gained popularity in many fields due to its outstanding performance. Its high accuracy has brought new breakthroughs to ship target detection tasks.

[0003] In the existing technology, the ship target detection methods based on deep learning include:

[0004] (1) Feature Mapping: The Chinese invention patent with publication number CN110647802A discloses a method for detecting ship targets in remote sensing images based on deep learning. A residual network is used to extract features from images in a remote sensing image dataset to generate a feature map. A region generation network is used to generate a predicted calibration frame on the feature map. The predicted calibration frame is judged to be a target calibration frame and regression analysis is performed to generate an accurate calibration frame. The accurate calibration frame is fused with the feature map to generate a fixed feature map for target positioning. The category of each fixed feature map is judged to achieve target detection. A similar solution to this solution is the Chinese invention patent "A method for detecting ship targets in high-resolution remote sensing images based on deep learning" (publication number CN108830224A). This solution is more suitable for target tracking and lacks adaptability to multi-scale targets.

[0005] (2) Edge Detection: The Chinese invention patent with publication number CN108121991A discloses a deep learning ship target detection method based on edge candidate region extraction. It uses a structured random forest model to perform edge detection on remote sensing images under different conditions, obtains purposeful edge detection results, and suppresses the influence of sea conditions and lighting on edge results. It uses a candidate region extraction algorithm based on edge detection results to extract candidate regions where targets may exist in remote sensing images with a wide width. The selected results are used as input to the deep learning network, thereby improving the efficiency of network processing and accelerating detection speed. The training of this scheme relies on binary true value annotation and connected area recognition. Therefore, when it is actually run on unlabeled images, it cannot obtain the high gradient edges of the training set, and essentially does not provide the ability to suppress the influence of sea conditions and lighting on edge results.

[0006] In summary, none of the aforementioned existing solutions address the problem of adapting to multi-scale target detection. In optical remote sensing ship target detection, the ocean background exhibits dynamic and variable optical properties due to factors such as wave texture and cloud shadows. This significantly reduces the grayscale / spectral contrast of ship targets in remote sensing imagery, significantly weakening target saliency. In particular, medium-resolution images exhibit a large pixel-level span of ship targets, making it difficult to maintain a balance between local and global features in detecting both small and large targets. Furthermore, deep learning networks typically employ a multi-layer convolutional network architecture, resulting in high computational complexity and long inference times. Summary of the Invention

[0007] To solve the above problems, the technical solution adopted in this application is a lightweight visible light ship target detection method based on edge feature guidance, including:

[0008] Dataset construction: remote sensing satellite images are collected, images containing ship targets are screened, ship targets in the images are annotated, and the annotated images are randomly distributed to obtain training and validation sets;

[0009] Model construction: The ship target detection model includes a backbone network module, a neck module and a head module. The backbone network module includes multiple stacked Conv modules and C3k2 modules, which are used to perform multiple convolutions on the input image data and output feature maps of multiple layers of effective feature layers to the neck module; the neck module includes a detail enhancement convolution module and a hierarchical pyramid module based on dynamic feature aggregation. The detail enhancement convolution module includes a convolution module based on edge-spatial features; the hierarchical pyramid module based on dynamic feature aggregation is a semantic-channel-space three-branch structure. Each branch structure acts on the feature pyramid level at different stages and integrates the multiple layers of effective feature layers output by the backbone network module; in the head module, the features of all detection layers are transformed through a shared module after independent convolution processing. The shared module is a multi-branch detail enhancement convolution module.

[0010] Model training: perform data augmentation on the training set; configure the ship target detection model parameters and initialize the weights of the ship target detection model; use the data augmented training set to train the ship target detection model; obtain the trained ship target detection model through the back propagation algorithm and gradient descent optimization method.

[0011] Optionally, the detail enhancement convolution module is controlled by the c3k parameters, including:

[0012] When c3k=False, after the feature map enters the detail enhancement convolution module, it first undergoes a 1×1 convolution, then splits the feature map, and a part of the feature map is processed by multiple ESConv modules in sequence. Finally, the feature map after 1×1 convolution, the feature map after the Split operation that has not been processed by the ESConv module, and the feature map obtained after the ESConv module are spliced ​​together. After the splicing is completed, it undergoes another 1×1 convolution and is output;

[0013] When c3k=True, after the feature map enters the detail enhancement convolution module, it first undergoes a 1×1 convolution, then splits the feature map, and then processes it in multiple C3k_ESC modules in sequence. Finally, the feature map after the 1×1 convolution, the feature map after the Split operation that has not been processed by the C3k_ESC module, and the feature map obtained after the C3k_ESC module are concatenated. After the splicing is completed, it undergoes another 1×1 convolution and is output;

[0014] After the feature map enters the C3k_ESC module, it first undergoes a 1×1 convolution, and then enters the upper C3k_ESC module branch and the lower C3k_ESC module branch respectively. The upper C3k_ESC module branch includes a 1×1 convolution, and the lower C3k_ESC module branch includes multiple ESConv modules connected in series. After the upper C3k_ESC module branch and the lower C3k_ESC module branch are processed, they are spliced ​​together and then undergo another 1×1 convolution output.

[0015] Optionally, the ESConv module is a convolution module based on edge-spatial features, including a Sobel edge detection branch, a spatial convolution branch and residual fusion. In the Sobel edge detection branch, the Sobel operator is used to extract edge features of the feature map. In the spatial convolution branch, 3×3 convolution is used to extract features of the feature map. The features extracted by the Sobel edge detection branch and the spatial convolution branch are spliced, and 1×1 convolution is used for channel compression, which is output after residual fusion.

[0016] Optionally, the Sobel operator is used to extract edge features from the feature map: , use the Sobel operator to extract edge features from the feature map as follows:

[0017] ;

[0018] Where, and They are 3D convolution operations in the horizontal and vertical directions, respectively, and the convolution kernel in the X direction K x and the convolution kernel in the Y direction K y It is defined as follows:

[0019] ;

[0020] .

[0021] Optionally, the hierarchical pyramid module based on dynamic feature aggregation is a semantic-channel-space three-branch structure. The semantic branch downsamples and compresses channels on large-scale features through the ADown module. The spatial branch reconstructs the small-scale feature structure through an upsampling operation, and then outputs it through a 1×1 convolution. The channel branch processes the medium-scale features through a 1×1 convolution. After the three branches are processed, the first splicing is performed. After splicing, the depthwise separable convolution modules with different convolution kernel sizes are used for processing. After processing with the depthwise separable convolution modules with different convolution kernel sizes, the feature maps obtained by the first splicing are spliced ​​for the second time. After the second splicing, channel aggregation is performed through 1x1 convolution, and then the feature maps obtained by the first splicing are spliced ​​for the third time and output.

[0022] Optionally, the ADown module includes making the input feature map First, through 2×2 average pooling, the average pooled features are split into two parts along the channel dimension and , The branch uses 3×3 convolution to directly downsample and compress the feature map size to , The branch first applies 3×3 maximum pooling to highlight local salient features, then adjusts the channel dimension through 1×1 convolution; finally, the outputs of the two branches are spliced.

[0023] Optionally, the depth-wise separable convolution module includes depth-wise convolution and point-wise convolution. In the depth-wise convolution, an independent k×k convolution kernel is applied to each input channel of the input feature map. After the depth-wise convolution, the point-wise convolution is used to linearly combine the channels of the depth-wise convolution output through the 1×1 convolution kernel, thereby realizing the fusion of information between channels and obtaining the final output feature map.

[0024] Optionally, the neck module includes a front neck module and a back neck module with the same structure, the front neck module includes an upper branch of the front neck module, a middle branch of the front neck module, and a lower branch of the front neck module, the upper branch of the front neck module includes a Conv module, a splicing module and a detail-enhanced convolution module, the middle branch of the front neck module includes a hierarchical pyramid module based on dynamic feature aggregation, and the lower branch of the front neck module includes an upsampling module, a splicing module and a detail-enhanced convolution module.

[0025] Optionally, the head module includes a target angle recognition branch and a target position recognition branch. The target angle recognition branch includes processing the large-scale features, medium-scale features and small-scale features through the Conv_GN module, the multi-branch detail enhancement convolution module and the two-dimensional convolution layer module respectively, and splicing them after the processing is completed to obtain the target angle information. The Conv_GN module is composed of a two-dimensional convolution layer, group normalization and SiLU activation function; the target position recognition branch includes processing the large-scale features, medium-scale features and small-scale features through the Conv_GN module respectively, and then entering two shared multi-branch detail enhancement convolution modules in series, and then processing them through the parallel regression convolution layer and the classification convolution layer respectively. After the processing is completed, splicing is performed, and the size information and position information of the target bounding box are output after splicing.

[0026] Optionally, the multi-branch detail enhancement convolution module includes parallel standard convolution branches, center differential convolution branches, horizontal differential convolution branches, vertical differential convolution branches and angular differential convolution branches. After the results of all the above branches are added together, they are output after batch normalization and activation function ReLU.

[0027] The beneficial effects of the lightweight visible light ship target detection method based on edge feature guidance provided by this application are:

[0028] (1) The detection head of shared detail enhancement convolution proposed in this application uniformly processes multi-scale features through shared convolution layers, thereby achieving lightweight detection head and improving accuracy. To address the problem of target feature blurring caused by variable background illumination conditions on the sea surface, a convolution based on edge-space features is proposed, and the Sobel operator is used to capture the contours and edge details of ship targets, thereby enhancing the model's robustness to illumination changes. To address the multi-scale problem of targets, a hierarchical pyramid structure based on dynamic feature aggregation is proposed, and a semantic-channel-space three-branch structure is designed to integrate global semantic information with local detail features to balance the detection of small and large targets, which is particularly suitable for ship target detection in medium-resolution images.

[0029] (2) The detection accuracy of ships in visible light remote sensing images exceeds 89%. The model accurately detects ships under different environmental conditions and has good recognition effects on multiple offshore targets, high-light sea surface targets, large pixel-level targets, low-light sea surface targets, cloud and fog interference targets, wave interference targets, multiple nearshore targets, lightly exposed targets, and small pixel-level targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art.

[0031] Figure 1This is a flow chart of a lightweight visible light ship target detection method based on edge feature guidance provided by an embodiment of the present application;

[0032] Figure 2 This is an overall algorithm structure diagram of the lightweight visible light ship target detection method based on edge feature guidance provided in an embodiment of the present application;

[0033] Figure 3 This is a schematic diagram of the C3k2_ESConv module structure provided in an embodiment of the present application;

[0034] Figure 4 This is a schematic diagram of the DFA-FPN module structure provided in an embodiment of the present application;

[0035] Figure 5 This is a schematic diagram of the ADown module structure provided in an embodiment of the present application;

[0036] Figure 6 DSConv is a schematic diagram of the structure of the depthwise separable convolution module provided in the embodiment of the present application;

[0037] Figure 7 This is a schematic diagram of the SDEC-Head module structure provided in an embodiment of the present application;

[0038] Figure 8 This is a diagrammatic illustration of an offshore multi-target visualization experiment provided in an embodiment of the present application;

[0039] Figure 9 This is a diagrammatic illustration of a low-light sea surface visualization experiment provided in an embodiment of the present application;

[0040] Figure 10 This is a diagrammatic illustration of a nearshore multi-target visualization experiment provided in an embodiment of the present application;

[0041] Figure 11 This is a diagrammatic illustration of a high-light sea surface visualization experiment provided in an embodiment of the present application;

[0042] Figure 12 This is a diagrammatic illustration of a cloud and fog interference visualization experiment provided in an embodiment of the present application;

[0043] Figure 13 This is a diagrammatic illustration of a light exposure visualization experiment provided in an embodiment of the present application;

[0044] Figure 14 This is a schematic diagram of the large pixel-level target visualization experiment annotation provided in the embodiment of the present application;

[0045] Figure 15 This is a diagrammatic illustration of a wave interference visualization experiment provided in an embodiment of the present application;

[0046] Figure 16 This is a schematic diagram of the small pixel-level target visualization experiment annotation provided in the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] Example 1

[0049] like Figure 1 As shown, the present application provides a lightweight visible light ship target detection method based on edge feature guidance, including:

[0050] Dataset construction: remote sensing satellite images are collected, images containing ship targets are screened, ship targets in the images are annotated, and the annotated images are randomly distributed to obtain training and validation sets;

[0051] Model construction: The ship target detection model includes a backbone network module, a neck module and a head module. The backbone network module includes multiple stacked Conv modules and C3k2 modules, which are used to perform multiple convolutions on the input image data and output feature maps of multiple layers of effective feature layers to the neck module; the neck module includes a detail enhancement convolution module and a hierarchical pyramid module based on dynamic feature aggregation. The detail enhancement convolution module includes a convolution module based on edge-spatial features; the hierarchical pyramid module based on dynamic feature aggregation is a semantic-channel-space three-branch structure. Each branch structure acts on the feature pyramid level at different stages and integrates the multiple layers of effective feature layers output by the backbone network module; in the head module, the features of all detection layers are transformed through a shared module after independent convolution processing. The shared module is a multi-branch detail enhancement convolution module.

[0052] Model training: perform data augmentation on the training set; configure the ship target detection model parameters and initialize the weights of the ship target detection model; use the data augmented training set to train the ship target detection model; obtain the trained ship target detection model through the back propagation algorithm and gradient descent optimization method.

[0053] The term "medium-resolution imagery" in this application refers to the spatial resolution of satellite imagery. The term "medium resolution" in this application refers to Li Miaomiao. Research on Remote Sensing Estimation Methods for Vegetation Cover [D]. Graduate School of the Chinese Academy of Sciences (Institute of Remote Sensing Applications), 2003. P40. Table 2 Remote Sensing Data Used for Vegetation Cover Monitoring, defining "medium spatial resolution" as 10-100m. Compared to high-resolution images, medium-resolution remote sensing satellite images can cover a larger area under the same imaging time and satellite orbit conditions. Using high-resolution images to monitor ships over a vast ocean requires multiple stitching and consumes significant time and satellite resources. Medium-resolution images, on the other hand, can cover a larger area in one go, allowing for a quick understanding of the overall distribution of ships in the area and improving image processing efficiency. Compared to low-resolution images, medium-resolution remote sensing satellite images can more clearly depict the outlines and structural features of ships. In areas with densely populated ships, medium-resolution images can more clearly separate different ship targets, leading to more accurate ship counts. Due to the low resolution of low spatial resolution images, adjacent ships are prone to aliasing.

[0054] Despite the aforementioned advantages, medium-resolution images also present challenges in recognition. This is primarily due to the large pixel-level span of ship targets, making it difficult to maintain a balance between local and global features when detecting both small and large objects. To address this, the aforementioned lightweight visible light ship detection method, guided by edge features, is employed to improve ship detection.

[0055] In this embodiment, the data set construction specifically includes:

[0056] A dataset is constructed based on remote sensing satellite images collected at different times and locations. In this example, the dataset is from Weihai-1 Satellite 01 and 02. The original images are converted to JPEG format, and images containing ship targets are screened out. Large-size images are segmented appropriately to obtain medium-size images (typically 1024 pixels × 1024 pixels, RGB three-channel).

[0057] Ship targets are manually identified and labeled, and then converted into YOLO format after labeling. The labeled remote sensing satellite images are randomly divided into training and validation sets according to the proportion.

[0058] After segmentation, a total of 2,119 medium-sized images were obtained. These images contain a total of 4,535 ship targets. The dataset was randomly split into two independent subsets: a training set and a validation set. The training set consists of 1,564 images, including 3,284 ship targets; the validation set consists of 555 images, including 1,251 ship targets. The dataset encompasses detection scenarios such as low light conditions, cloud and fog interference, land interference, small targets, and densely packed targets, effectively reflecting real-world detection requirements.

[0059] Construct a model that includes Backbone module, Neck module and Head module. The overall structure of the model is as follows: Figure 2 shown.

[0060] The Backbone module is composed of multiple Conv modules and C3k2 modules stacked together. It performs multiple convolutions on the input image and outputs multiple layers of effective feature layers to the Neck module. The deeper layers of the Backbone module also include the SPPF module and the C2PSA module.

[0061] In this embodiment, the Backbone module includes the Conv module (the output spatial scale is 512×512 and the channel scale is 64), the Conv module (the output spatial scale is 256×256 and the channel scale is 128), the C3k2 module (the output spatial scale is 256×256 and the channel scale is 256), the Conv module (the output spatial scale is 128×128 and the channel scale is 256), the C3k2 module (the output spatial scale is 128×128 and the channel scale is 512), and the Conv module. v module (the output spatial scale is 64×64 and the channel scale is 512), C3k2 module (the output spatial scale is 64×64 and the channel scale is 512), Conv module (the output spatial scale is 32×32 and the channel scale is 1024), C3k2 module (the output spatial scale is 32×32 and the channel scale is 1024), SPPF module (the output spatial scale is 32×32 and the channel scale is 1024) and C2PSA module (the output spatial scale is 32×32 and the channel scale is 1024).

[0062] The Conv module includes a two-dimensional convolution layer (conv2d), a batch normalization layer (BatchNorm), and a SiLU activation function layer.

[0063] The "2" in the C3k2 module is the value of N. The value of the c3k parameter is controlled according to the actual performance of the model. For reference, in the setting parameters of the backbone network in this embodiment, the c3k parameters of the first two C3k2 modules from bottom to top are False, and the c3k parameters of the last two are True. The convolution kernel size of all Conv modules is 3 and the step size is 2.

[0064] The C3k2 module is controlled by the c3k parameter and the N parameter:

[0065] When c3k=False, after the feature map enters the C3k2 module, it first passes through a Conv module, then splits the feature map, and a part of the feature map is processed by N Bottleneck modules in sequence. Finally, the feature map processed by the Conv module, the feature map after the Split operation that has not been processed by the Bottleneck module, and the feature map obtained after the Bottleneck module are spliced ​​together. After the splicing is completed, it is processed by a Conv module again and output;

[0066] When c3k=True, after the feature map enters the C3k2 module, it first passes through a Conv module, then splits the feature map, and then passes through N C3k N=2 modules in sequence. Finally, the feature map processed by the Conv module, the feature map after the Split operation but not processed by the C3k N=2 module, and the feature map obtained after the C3k N=2 module are concatenated. After the splicing is completed, it is processed again by a Conv module and output;

[0067] After the feature map enters the C3k N=2 module (the module is represented as C3k N=? in the figure), it is first processed by the Conv module, and then enters the upper C3k N=2 module branch and the lower C3k N=2 module branch respectively. The upper C3k N=2 module branch includes a Conv module, and the lower C3k_ESC module branch includes N (N=2) Bottleneck modules connected in series. After processing, the upper C3k N=2 module branch and the lower C3k N=2 module branch are spliced ​​together. After splicing, they are processed again by a Conv module and then output.

[0068] The Bottleneck module includes two Conv modules connected in series, and the input of the Bottleneck module is directly connected to the output processed by the two Conv modules in series through a residual connection, and the input of the Bottleneck module is directly connected to the output processed by the two Conv modules in series through an addition operation.

[0069] After receiving the feature map input, the SPPF module first processes it through a Conv module, and then processes the convolved feature map in parallel through multiple (3 in this embodiment) maximum pooling layers (Maxpool2d), extracting feature information of different scales respectively, splicing the feature maps processed by multiple maximum pooling layers and the feature maps processed by the Conv module, fusing the multi-scale feature information, and processing the spliced ​​feature map with the Conv module to obtain the output feature map.

[0070] After receiving the feature map input, the C2PSA module first processes it through a Conv module. After processing, it passes through multiple serially connected PSAblock (PSA module) layers to splice the feature maps processed by multiple PSAblock layers and the feature maps processed by the Conv module. The spliced ​​feature maps are processed by the Conv module to obtain the output feature map.

[0071] After receiving the feature map input, the PSA module calculates the attention weights for the input feature map through the attention layer to generate an attention feature map. The output of the attention layer is first added to the feature map received by the PSA module. After the addition, it is processed by two Conv modules. The feature map processed by the two Conv modules is then added to the feature map obtained by the first addition for a second time.

[0072] The Neck part includes the detail enhancement convolution module C3k2_ESConv and the hierarchical pyramid DFA-FPN based on dynamic feature aggregation. The detail enhancement convolution module C3k2_ESConv adopts the convolution ESConv based on edge-spatial features. The hierarchical pyramid DFA-FPN based on dynamic feature aggregation adopts a semantic-channel-space three-branch structure, which acts on the feature pyramid levels at different stages and effectively fuses the multi-layer effective feature layers output by the Backbone part.

[0073] like Figure 3 As shown, the detail enhancement convolution module C3k2_ESConv is controlled by the N parameter and the c3k parameter. The "2" in the C3k2_ESConv module is the value of N, and the c3k parameter is specifically set according to the model performance. In this embodiment, c3k is True in the C3k2_ESConv on the upper branch of the front neck module, c3k is True in the C3k2_ESConv on the upper branch of the back neck module, c3k is False in the C3k2_ESConv on the lower branch of the front neck module, and c3k is False in the C3k2_ESConv on the lower branch of the back neck module, including:

[0074] When c3k=False, after the feature map enters the detail enhancement convolution module, it first undergoes a 1×1 convolution, then splits the feature map, and then is processed by multiple ESConv modules in sequence. Finally, the feature maps obtained after the 1×1 convolution, Split operation, and ESConv module processing are spliced ​​together. After the splicing is completed, it undergoes another 1×1 convolution and is output;

[0075] When c3k=True, after the feature map enters the detail enhancement convolution module, it first undergoes a 1×1 convolution, then splits the feature map, and then is processed by multiple C3k_ESC modules in sequence. Finally, the feature maps obtained after the 1×1 convolution, Split operation, and C3k_ESC module processing are concatenated. After the splicing is completed, it undergoes another 1×1 convolution and is output;

[0076] After the feature map enters the C3k_ESC module, it first undergoes a 1×1 convolution, and then enters the upper C3k_ESC module branch and the lower C3k_ESC module branch respectively. The upper C3k_ESC module branch includes a 1×1 convolution, and the lower C3k_ESC module branch includes multiple ESConv modules connected in series. After the upper C3k_ESC module branch and the lower C3k_ESC module branch are processed, they are spliced ​​together and then undergo another 1×1 convolution output.

[0077] like Figure 3 As shown in the figure, the ESConv module is a convolution module based on edge-spatial features, including a Sobel edge detection branch, a spatial convolution branch and residual fusion. The spatial convolution branch is used for spatial feature extraction, and the Sobel edge detection branch is used for edge detail feature extraction. In the Sobel edge detection branch, the Sobel operator is used to extract edge features from the feature map. In the spatial convolution branch, the features of the feature map are extracted through 3×3 convolution. The features extracted by the Sobel edge detection branch and the spatial convolution branch are spliced, and 1×1 convolution is used for channel compression. The output is after residual fusion.

[0078] Specifically, when the input feature map is ,in is the number of input channels, H and W are the feature map height and width, and the module target output channel number is .

[0079] For the input feature map , use the Sobel operator to extract edge features from the feature map, and calculate the Sobel edge detection branch as follows:

[0080] ;

[0081] Where, and They are 3D convolution operations in the horizontal and vertical directions, respectively, and the convolution kernel in the X direction K x and the convolution kernel in the Y direction K y It is defined as follows:

[0082] ;

[0083] ;

[0084] The spatial convolution branch is calculated as follows, that is, the features of the original image are extracted through 3×3 convolution:

[0085] ;

[0086] Where, It is a standard convolutional layer with BatchNorm and activation function.

[0087] Finally, the edge convolution and spatial convolution structures are concatenated to obtain:

[0088] ;

[0089] Use 1×1 convolution to concatenate Channel compression back Channel, get X feature .

[0090] Output after residual connection:

[0091] .

[0092] like Figure 4 As shown, DFA-FPN (a hierarchical pyramid module based on dynamic feature aggregation) constructs a three-branch structure: semantic-channel-spatial. The semantic branch downsamples and compresses large-scale features (denoted as P3 in the figure) through the ADown module, aggregating global semantic information. The spatial branch reconstructs small-scale features (denoted as P5 in the figure) through upsampling, enhancing the spatial detail of the feature map, and then outputs them through a 1×1 convolution. The channel branch reduces the number of parameters while preserving spatial information, processing medium-scale features (denoted as P4 in the figure) through 1×1 convolution. The DFA-FPN structure enhances the semantics of large-scale features and supplements the details of small-scale features. Depthwise separable convolutions are used to achieve a balance between computational efficiency and feature expression.

[0093] The "small-scale features", "medium-scale features" and "large-scale features" referred to in this application are used to distinguish the output feature maps of different convolutional layers, and refer to the relative sizes between the output feature maps of different convolutional layers, and only refer to the spatial scale of the output feature map of the module, excluding the channel scale.

[0094] After the three-branch processing is completed, the first splicing is performed. After splicing, depthwise separable convolution modules with different convolution kernel sizes are used for processing. After processing with depthwise separable convolution modules with different convolution kernel sizes, the feature map obtained by the first splicing is spliced ​​for the second time. After the second splicing, channel aggregation is performed through 1x1 convolution, and then the feature map obtained by the first splicing is spliced ​​for the third time and output.

[0095] DFA-FPN acts on the feature pyramid levels at different stages, improving the ability to represent multi-scale targets in dense scenes without significantly increasing the computational complexity.

[0096] The structure of the ADown module is as follows Figure 5 As shown in the figure, the ADown module achieves lightweight downsampling through multi-branch processing of average pooling, channel splitting, different convolution and maximum pooling operations, reducing the number of parameters while retaining image information and improving target detection accuracy.

[0097] Specifically, the input feature map First, the smoothness and robustness of the features are enhanced by 2×2 average pooling. The specific calculation method is shown in the following formula:

[0098] ;

[0099] Where, X avg [ i,j,c ] indicates that the feature map obtained after the 2×2 average pooling operation is at position ( i,j ),aisle c The eigenvalue at m and n It is the index of the convolution kernel in the spatial dimension, which averages the pooled features. Split into two parts along the channel dimension and , The branch uses 3×3 convolution to directly downsample and compress the feature map size to The specific calculation method is as follows

[0100] ;

[0101] Where, [ i,j, ]express xThe feature map obtained after the 3×3 convolution operation of branch 1 is at position ( i,j ),aisle The eigenvalue at [ m,n,c, ]express x The convolution kernel weights used in the 1-branch convolution operation.

[0102] According to the following formula, The branch first applies 3×3 maximum pooling to highlight local salient features, with a stride of 2 and a padding of 1:

[0103] ;

[0104] Where, [ i,j, ]express The feature map obtained after the branch undergoes a 3×3 maximum pooling operation is at position ( i,j ),aisle c The eigenvalue at .

[0105] Then adjust the channel dimension through 1×1 convolution according to the following formula:

[0106] ;

[0107] Where, [ i,j, ]express x The feature map obtained after the 1×1 convolution operation of the 2 branches is at position ( i,j ),aisle The eigenvalue at .

[0108] Finally, press the formula to output the two branches and Splicing:

[0109] .

[0110] like Figure 6 As shown in the figure, depthwise separable convolution (DSC) consists of depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, and pointwise convolution is used to extract channel features. Depthwise separable convolution groups convolutions in the feature dimension, performs independent depthwise convolution on each channel, and uses a 1x1 convolution (pointwise convolution) to aggregate all channels before the output.

[0111] In standard convolution, assuming the input feature map , convolution kernel , the calculation formula of the output feature map Y is:

[0112] ;

[0113] Where, is the coordinate of the output feature map, is the output channel index, m and n are the coordinates within the convolution kernel, The input channel index.

[0114] In depth-wise convolution, for input features Figure X , whose dimensions are . Apply a separate k×k convolution kernel to each input channel. The output of the depthwise convolution The calculation formula is:

[0115] ;

[0116] in, is the convolution kernel of each channel, with dimension Depthwise convolution performs convolution operations on each channel separately, without changing the number of channels, but only extracting the spatial information of each channel. After depthwise convolution, pointwise convolution is used to combine the information of different channels. Pointwise convolution uses a 1×1 convolution kernel. For the output of depthwise convolution , the final output Y is obtained by point-by-point convolution, and the calculation formula is:

[0117] ;

[0118] in, Is the convolution kernel of point-by-point convolution, with dimension , is the number of output channels of depthwise convolution. Pointwise convolution linearly combines the channels of depthwise convolution output through 1×1 convolution kernel, thereby achieving the fusion of information between channels and obtaining the final output feature map.

[0119] like Figure 2 As shown, the neck module includes a front neck module and a back neck module with the same structure. The front neck module includes an upper branch of the front neck module, a middle branch of the front neck module, and a lower branch of the front neck module. The upper branch of the front neck module includes a Conv module, a splicing module, and a detail-enhanced convolution module (C3k2_ESConv module). The middle branch of the front neck module includes a hierarchical pyramid module based on dynamic feature aggregation (DFA-FPN module). The lower branch of the front neck module includes an upsampling module, a splicing module, and a detail-enhanced convolution module.

[0120] The back neck module includes the upper branch of the back neck module, the middle branch of the back neck module, and the lower branch of the back neck module. The upper branch of the back neck module includes the Conv module, the splicing module and the detail-enhanced convolution module (C3k2_ESConv). The middle branch of the back neck module includes a hierarchical pyramid module based on dynamic feature aggregation. The lower branch of the back neck module includes the upsampling module, the splicing module and the detail-enhanced convolution module.

[0121] The hierarchical pyramid module based on dynamic feature aggregation in the front neck module receives the output of the fifth layer C3k2 module, the output of the seventh layer C3k2 module and the output of the C2PSA module from the backbone network module, respectively, to obtain large-scale features (in Figure 4 In the figure, it is represented as P3), mesoscale features (in Figure 4 P4 in the Figure 4 After processing, the hierarchical pyramid module based on dynamic feature aggregation outputs the information to the Conv module on the upper branch of the front neck module, the hierarchical pyramid module based on dynamic feature aggregation on the middle branch of the front neck module, and the upsampling module on the lower branch of the front neck module.

[0122] The splicing module of the upper branch of the front neck module splices the output of the Conv module of the upper branch of the front neck module and the output of the C2PSA module of the backbone network module; the splicing module of the lower branch of the front neck module splices the output of the upsampling module of the lower branch of the front neck module and the output of the fifth-layer C3k2 module of the backbone network module.

[0123] The hierarchical pyramid module based on dynamic feature aggregation in the middle branch of the back neck module receives the output of the upper branch of the front neck module, the output of the middle branch of the front neck module and the output of the lower branch of the front neck module as small-scale features (in Figure 4 In the figure, it is represented as P5), mesoscale features (in Figure 4 P4 in the Figure 4 After processing by the hierarchical pyramid module based on dynamic feature aggregation, the output is respectively sent to the Conv module of the upper branch of the back neck module, the head module, and the upsampling module of the lower branch of the back neck module.

[0124] The splicing module on the upper branch of the back neck module splices the output of the Conv module on the upper branch of the front neck module, the output of the detail-enhanced convolution module (C3k2_ESConv) on the upper branch of the front neck module, and the output of the Conv module on the upper branch of the back neck module; the splicing module on the lower branch of the back neck module splices the output of the upsampling module on the lower branch of the front neck module, the output of the detail-enhanced convolution module (C3k2_ESConv) on the lower branch of the front neck module, and the output of the upsampling module on the lower branch of the back neck module.

[0125] The head module receives the output of the branch on the back neck module as a small-scale feature (in Figure 7 denoted as P5 in the figure), receiving the output of the branch in the back neck module as the mid-scale feature (in Figure 7 denoted as P4 in the figure), receives the output of the lower branch of the back neck module as a large-scale feature (in Figure 7 The specific structure of the SDEC-Head detection head of the head module is as follows Figure 7 As shown in the figure, the characteristic is that the features of all detection layers are transformed through a shared module after independent convolution processing, and the shared module is a convolution structure constructed by multi-branch detail enhancement convolution.

[0126] The head module includes a target angle recognition branch and a target position recognition branch. The target angle recognition branch includes processing the large-scale features, medium-scale features and small-scale features through the Conv_GN module, the multi-branch detail enhancement convolution module and the two-dimensional convolution layer module respectively. After processing, they are spliced ​​together to obtain the target angle information. The Conv_GN module consists of a two-dimensional convolution layer, group normalization and SiLU activation function; the target position recognition branch includes processing the large-scale features, medium-scale features and small-scale features through the Conv_GN module respectively, and then entering two shared multi-branch detail enhancement convolution modules in series, and then processing them through the parallel regression convolution layer and classification convolution layer respectively. After processing, they are spliced ​​together and the size information and position information of the target bounding box are output after splicing.

[0127] Multi-branch detail enhanced convolution (MDEConv) is a composite convolution module that integrates multiple convolution variants. It includes parallel standard convolution branches, center differential convolution branches, horizontal differential convolution branches, vertical differential convolution branches, and angular differential convolution branches. The results of all these branches are summed and then output using batch normalization and the ReLU activation function. By combining different convolution operations, the MDEConv module fuses gradient information from different directions to enhance feature extraction capabilities. Each branch in MDEConv uses a different convolution mode. Specifically, let the input feature map be:

[0128] ;

[0129] in, is the number of input channels, and H×W is the spatial size.

[0130] First, standard convolution is used to provide basic local feature extraction capabilities, weights W std The calculation is as follows:

[0131] ;

[0132] In the formula Represents a specific element in the convolution kernel weight matrix, and the subscript indicates the position. Represents the element in the first row and first column, the same below;

[0133] At the same time, the central difference convolution is used to enhance the local contrast, highlight the central features, and weight W cd The calculation is as follows:

[0134] ;

[0135] Indicates the sum of all elements in the central difference convolution weight matrix;

[0136] Horizontal differential convolution is used to extract horizontal edge features, weights W hd The calculation is as follows:

[0137] ;

[0138] Vertical differential convolution is used to extract vertical edge features, weight W vd The calculation is as follows:

[0139] ;

[0140] Use angle difference convolution to enhance diagonal gradient and weight W ad The calculation is as follows:

[0141] ;

[0142] Where, is the differential strength hyperparameter.

[0143] After all branch results are added together, they are batch normalized and activated using the ReLU function. This structure significantly improves the perception of edges, textures, and local contrast through multi-directional differential operations, significantly enhancing the geometric perception of feature representation while maintaining translation invariance.

[0144] Model training includes:

[0145] Perform data enhancement on the training set constructed from the dataset, including rotation, flipping, scaling, and mosaic data enhancement;

[0146] According to the algorithm obtained from model construction, configure the ship target detection model parameters, including network structure, learning rate, and batch size; at the same time, initialize the weights of the ship target detection model;

[0147] The experimental environment configuration is shown in Table 1. Regarding hyperparameter configuration, the optimizer used stochastic gradient descent with momentum (with a momentum value of 0.937), and the initial learning rate was set to 0.01. A cosine annealing strategy was used to dynamically adjust the learning rate, causing it to decay along a cosine curve from its initial value. The learning rate decayed by 0.001 at the end of training. Each training batch used 8 images, and 500 training epochs were performed.

[0148] Table 1 Experimental environment configuration

[0149]

[0150] The ship target detection model is trained using the data-enhanced training set; the trained ship target detection model is obtained through the back propagation algorithm and gradient descent optimization method.

[0151] Ship target detection model verification:

[0152] The validation set divided in the dataset construction is used to verify the ship target detection model. The data collection and annotation quality standards of the validation set and the training set are consistent.

[0153] The performance of the trained ship target detection model is evaluated using the validation set. The performance evaluation includes calculating Precision, Recall, mAP50 (average precision with an Intersection over Union (IoU) threshold of 0.5), mAP50-95 (average precision with an IoU threshold from 0.5 to 0.95), Parameters, FPS (frames per second), and GFLOPs / G (gigafloating-point operations per second).

[0154] In order to evaluate the detection performance of this method, this application used several state-of-the-art target detection methods for a comprehensive quantitative evaluation, and adopted a variety of evaluation indicators from different perspectives. The evaluation results are shown in Table 2.

[0155] Table 2 Evaluation of target detection methods

[0156]

[0157] In the table:

[0158] Roi transformer: an object detection method that processes regions of interest (RoIs) based on the Transformer architecture.

[0159] redet: an object detection method optimized for specific object detection tasks;

[0160] GlidingVertex: An object detection method that generates an object's bounding box by sliding vertices.

[0161] OrientedRCNN: An improved target detection method based on the classic RCNN (Region-based Convolutional Neural Network) specifically for detecting targets with orientation

[0162] Rotated FasterRcnn: An object detection method based on an improved version of Faster R-CNN, focusing on rotated object detection;

[0163] Oriented RepPoints: An object detection method that combines RepPoints (representative points) and processing of object orientation.

[0164] R3det: an object detection method specifically designed for rotated object detection.

[0165] S2anet: Single-Shot Alignment Network, an object detection method for rotated object detection.

[0166] YOLOv5: The fifth generation of the You Only Look Once (YOLO) family of object detection methods;

[0167] YOLOv8: The eighth generation of the You Only Look Once (YOLO) family of object detection methods.

[0168] YOLOv11: The eleventh generation of object detection methods in the You Only Look Once (YOLO) series;

[0169] Ours: represents the target detection method proposed in this application.

[0170] As can be seen from the table, this method has achieved significant performance improvements compared with other algorithms in multiple indicators. The recall rate, precision rate and mAP50 of this method reached 82.0%, 89.4% and 89.4% respectively, which are significantly better than RotatedFasterRcnn, YOLOv8 and YOLOv11.

[0171] In addition, it also demonstrates significant advantages in terms of parameter count and computational complexity. To evaluate the model's computational efficiency in real-world scenarios, we compared its inference time. The results show that the algorithm can process an average of 91.4 images per second, achieving a very high inference speed compared to RotatedFasterRcnn, R3det, and others. Compared to the YOLO series, it is comparable in computational complexity and inference speed, but has improved accuracy and mAP.

[0172] To verify the effectiveness of each module in the edge-feature-guided lightweight visible light ship target detection method proposed in this application for ship detection, we integrated the modules of this method into YOLOv11 and conducted ablation experiments on the resulting dataset. The experimental results are shown in Table 3. Compared with YOLOv11-OBB, the precision, recall, mAP50, and mAP50-95 all improved, verifying the effectiveness of the proposed modules.

[0173] Table 3 Ablation test results after YOLOv11 integrates each module of this application

[0174]

[0175] In the visualization experiment part, the detection accuracy of this method is verified on the validation set constructed from the dataset, such as Figures 8-16 shown.

[0176] This method has a detection accuracy of over 89% for ships in visible light remote sensing images. The model accurately detects ships under different environmental conditions, demonstrating the stability and reliability of the algorithm in practical applications, further confirming its adaptability and robustness.

[0177] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A lightweight visible light ship target detection method based on edge feature guidance, characterized by: include: Dataset construction: remote sensing satellite images are collected, images containing ship targets are screened, ship targets in the images are annotated, and the annotated images are randomly distributed to obtain training and validation sets; Model construction: The ship target detection model includes a backbone network module, a neck module and a head module. The backbone network module includes multiple stacked Conv modules and C3k2 modules, which are used to perform multiple convolutions on the input image data and output feature maps of multiple layers of effective feature layers to the neck module; the neck module includes a detail enhancement convolution module and a hierarchical pyramid module based on dynamic feature aggregation. The detail enhancement convolution module includes a convolution module based on edge-spatial features; the hierarchical pyramid module based on dynamic feature aggregation is a semantic-channel-space three-branch structure, each branch structure acts on the feature pyramid level at different stages, and integrates the multiple layers of effective feature layers output by the backbone network module; in the head module, the features of all detection layers are subjected to independent convolution processing and then feature transformed by a shared module, which is a multi-branch detail enhancement convolution module; Model training: Perform data augmentation on the training set; configure the ship target detection model parameters and initialize the weights of the ship target detection model; use the data augmented training set to train the ship target detection model; obtain the trained ship target detection model through the back propagation algorithm and gradient descent optimization method; The neck module includes a front neck module and a back neck module with the same structure. The front neck module includes an upper branch of the front neck module, a middle branch of the front neck module, and a lower branch of the front neck module. The upper branch of the front neck module includes a Conv module, a splicing module, and a detail-enhanced convolution module. The middle branch of the front neck module includes a hierarchical pyramid module based on dynamic feature aggregation. The lower branch of the front neck module includes an upsampling module, a splicing module, and a detail-enhanced convolution module. The head module includes a target angle recognition branch and a target position recognition branch. The target angle recognition branch includes processing large-scale features, medium-scale features and small-scale features through a Conv_GN module, a multi-branch detail enhancement convolution module and a two-dimensional convolution layer module respectively, and splicing them after the processing is completed to obtain target angle information. The Conv_GN module is composed of a two-dimensional convolution layer, a group normalization and a SiLU activation function; the target position recognition branch includes processing large-scale features, medium-scale features and small-scale features through a Conv_GN module respectively, and then entering two shared multi-branch detail enhancement convolution modules in series, and then respectively processing through a parallel regression convolution layer and a classification convolution layer. After the processing is completed, splicing is performed, and after splicing, the size information and position information of the target bounding box are output.

2. The lightweight visible light ship target detection method based on edge feature guidance according to claim 1 is characterized by: The detail enhancement convolution module is controlled by the c3k parameters, including: When c3k=False, after the feature map enters the detail enhancement convolution module, it first undergoes a 1×1 convolution, then the feature map is split, and part of the feature map is processed by multiple ESConv modules in sequence. Finally, the feature map after 1×1 convolution, the feature map after the Split operation that has not been processed by the ESConv module, and the feature map obtained after the ESConv module are spliced ​​together. After the splicing is completed, it undergoes another 1×1 convolution and is output; When c3k=True, after the feature map enters the detail enhancement convolution module, it first undergoes a 1×1 convolution, then splits the feature map, and then sequentially processes it through multiple C3k_ESC modules. Finally, the feature map after the 1×1 convolution, the feature map after the Split operation that has not been processed by the C3k_ESC module, and the feature map obtained after the C3k_ESC module are concatenated. After the splicing is completed, it undergoes another 1×1 convolution and is output; After the feature map enters the C3k_ESC module, it first undergoes a 1×1 convolution, and then enters the upper C3k_ESC module branch and the lower C3k_ESC module branch respectively. The upper C3k_ESC module branch includes a 1×1 convolution, and the lower C3k_ESC module branch includes multiple ESConv modules connected in series. After the upper C3k_ESC module branch and the lower C3k_ESC module branch are processed, they are spliced ​​together and then undergo another 1×1 convolution output.

3. The lightweight visible light ship target detection method based on edge feature guidance according to claim 2 is characterized by: The ESConv module is a convolution module based on edge-spatial features, including a Sobel edge detection branch, a spatial convolution branch, and residual fusion. In the Sobel edge detection branch, the Sobel operator is used to extract edge features from the feature map. In the spatial convolution branch, 3×3 convolution is used to extract features from the feature map. The features extracted by the Sobel edge detection branch and the spatial convolution branch are spliced, and 1×1 convolution is used for channel compression. The output is after residual fusion.

4. The lightweight visible light ship target detection method based on edge feature guidance according to claim 3 is characterized by: The Sobel operator is used to extract edge features from the feature map: , use the Sobel operator to extract edge features from the feature map as follows: ； Where, and They are 3D convolution operations in the horizontal and vertical directions, respectively, and the convolution kernel in the X direction K x and the convolution kernel in the Y direction K y It is defined as follows: ； 。 5. The lightweight visible light ship target detection method based on edge feature guidance according to claim 1 is characterized by: The hierarchical pyramid module based on dynamic feature aggregation is a semantic-channel-space three-branch structure. The semantic branch downsamples and compresses channels on large-scale features through the ADown module. The spatial branch reconstructs small-scale feature structures through upsampling operations, and then outputs them through 1×1 convolution. The channel branch processes medium-scale features through 1×1 convolution. After the three-branch processing is completed, the first splicing is performed. After splicing, depthwise separable convolution modules with different convolution kernel sizes are used for processing. After processing with depthwise separable convolution modules with different convolution kernel sizes, the feature maps obtained by the first splicing are spliced ​​for the second time. After the second splicing, channel aggregation is performed through 1x1 convolution, and then the feature maps obtained by the first splicing are spliced ​​for the third time and output.

6. The lightweight visible light ship target detection method based on edge feature guidance according to claim 5 is characterized by: The ADown module includes the input feature map First, through 2×2 average pooling, the average pooled features are split into two parts along the channel dimension and , The branch uses 3×3 convolution to directly downsample and compress the feature map size to , The branch first applies 3×3 maximum pooling to highlight local salient features, then adjusts the channel dimension through 1×1 convolution; finally, the outputs of the two branches are spliced.

7. The lightweight visible light ship target detection method based on edge feature guidance according to claim 5 is characterized by: The depth-wise separable convolution module includes depth-wise convolution and point-wise convolution. In the depth-wise convolution, an independent k×k convolution kernel is applied to each input channel of the input feature map. After the depth-wise convolution, the point-wise convolution is used to linearly combine the channels of the depth-wise convolution output through the 1×1 convolution kernel, thereby realizing the fusion of information between channels and obtaining the final output feature map.

8. The lightweight visible light ship target detection method based on edge feature guidance according to claim 1 is characterized by: The multi-branch detail enhancement convolution module includes parallel standard convolution branches, center differential convolution branches, horizontal differential convolution branches, vertical differential convolution branches and angle differential convolution branches. After the results of all the above branches are added together, they are output after batch normalization and activation function ReLU.

Images