SAR ship image segmentation method and system based on bifunctional neural network

Abstract

The invention provides an SAR ship image segmentation method and system based on a bifunctional neural network, and belongs to the technical field of SAR ship image segmentation. The method comprises the steps of obtaining an SAR ship image data set and performing data set division; training a bifunctional neural network by using the divided SAR ship image data set; carrying out image segmentation on the to-be-segmented ship image based on the trained bifunctional neural network; wherein the difunctional neural network comprises a backbone network and a difunctional alignment network, and during image segmentation, attention distribution of the network is dynamically modulated based on a dual-network mechanism of the difunctional alignment network so as to pay attention to a plurality of discrimination areas on the ship image to be segmented. According to the method, on the basis of improving the SAR ship image segmentation efficiency, multiple discrimination regions in the image segmentation process can be concerned to the greatest extent, so that omission of segmentation details is prevented, and the SAR ship image segmentation precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of synthetic aperture radar (SAR) ship image segmentation, and particularly relates to a SAR ship image segmentation method and system based on a dual-functional neural network. Background Art

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of satellite and remote sensing technologies, a large number of high-resolution remote sensing images have been generated, posing great challenges to manual operation and processing. Therefore, the automatic analysis and understanding of remote sensing images and synthetic aperture radar (SAR) images have become increasingly important for various civilian applications. In recent years, it has attracted considerable attention in the field of deep learning. Most of the existing models for object detection and instance segmentation have achieved success in traditional forward-looking scenarios. However, when directly applied to remote sensing images, a large number of related methods will inevitably encounter performance degradation. Compared with natural images, SAR images are usually observed from the top to capture limited object differences over a large area. For the instance segmentation of ships, it can help shipping companies optimize shipping routes and avoid congestion.

[0004] However, there are generally some technical problems in the existing SAR ship image segmentation methods, such as:

[0005] (1) The existing SAR ship image segmentation methods generally use a deep neural network based on a residual network as a segmentation tool. However, the receptive field of this basic network is small, and it can only capture local information in the image, unable to obtain global context information, resulting in the neural network only focusing on the most significant discriminant regions. At the same time, the design of the attention mechanism is also unreasonable, exacerbating the defect that the model only focuses on the most significant part of the image while ignoring other equally important information. Therefore, the segmentation of SAR ship images often has omissions and poor accuracy.

[0006] (2) To avoid omission problems, there are also methods for ship image segmentation based on morphological filtering algorithms. Although such methods can achieve good segmentation results, their computational complexity is high, resulting in slow processing speed. In addition, the threshold segmentation method is also a commonly used SAR image segmentation algorithm, but how to accurately select the threshold is a difficult problem. Especially in the case of poor image quality or noise interference, the selection of the threshold often affects the accuracy and efficiency of segmentation. Summary of the Invention

[0007] To overcome the deficiencies of the above-mentioned prior art, the present invention provides a SAR ship image segmentation method and system based on a dual-functional neural network, which can, on the basis of improving the efficiency of SAR ship image segmentation, pay the greatest attention to multiple discriminant regions in the image segmentation process to prevent the omission of segmentation details, thereby improving the accuracy of SAR ship image segmentation.

[0008] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:

[0009] The first aspect of the present invention provides a SAR ship image segmentation method based on a dual-functional neural network.

[0010] The SAR ship image segmentation method based on a dual-functional neural network includes:

[0011] Obtain the ship image to be segmented and the SAR ship image dataset, and divide the obtained SAR ship image dataset;

[0012] Use the divided SAR ship image dataset to train the dual-functional neural network;

[0013] Perform image segmentation on the ship image to be segmented based on the trained dual-functional neural network; wherein, the dual-functional neural network includes a backbone network and a dual-functional alignment network, and when performing image segmentation on the ship image to be segmented, the attention distribution of the network is dynamically modulated based on the dual-network mechanism of the dual-functional alignment network to pay attention to multiple discriminant regions on the ship image to be segmented.

[0014] Further, the backbone network combines the Residual Network (Resnet), the Feature Pyramid Network (FPN), and the Region Proposal Network (RPN) at the same time.

[0015] Further, the Residual Network (Resnet) is composed of multiple residual blocks, and each residual block includes multiple convolutional layers; the Feature Pyramid Network (FPN) constructs a feature pyramid using a top-down path, upsamples the high-level features, and then makes a horizontal connection with the feature maps of the lower levels; the Region Proposal Network (RPN) generates bounding boxes with a fixed size and aspect ratio at each position of the obtained feature map to generate candidate regions from the feature map.

[0016] Further, the dual-functional alignment network includes a dual-functional alignment module matrix and a dual-functional alignment head module.

[0017] Further, the dual-functional alignment module substrate includes a first functional alignment module and a second functional alignment module; wherein, the first functional alignment module uses classification alignment convolution to extract classification features on the semantic feature map to generate a classification feature map; the second functional alignment module uses regression all-alignment convolution to extract regression features on the semantic feature map to generate a regression feature map.

[0018] Further, after receiving the classification feature map and the regression feature map generated by the dual-functional alignment module substrate, the dual-functional alignment head module performs analysis based on the attention mechanism to generate a target region and a classification score.

[0019] Further, based on the trained dual-functional neural network, the ship image to be segmented is segmented. After the image features are extracted based on the dual-functional neural network and the RN box and the classification score are obtained, the final segmentation mask is output based on the mask segmentation head to achieve SAR ship image segmentation.

[0020] The second aspect of the present invention provides a SAR ship image segmentation system based on a dual-functional neural network.

[0021] The SAR ship image segmentation system based on a dual-functional neural network includes:

[0022] An image acquisition module, configured to: acquire the ship image to be segmented and the SAR ship image dataset, and divide the obtained SAR ship image dataset;

[0023] A model training module, configured to: use the divided SAR ship image dataset to train the dual-functional neural network;

[0024] A SAR ship image segmentation module, configured to: segment the ship image to be segmented based on the trained dual-functional neural network; wherein, the dual-functional neural network includes a backbone network and a dual-functional alignment network. When segmenting the ship image to be segmented, the attention distribution of the network is dynamically modulated based on the dual-network mechanism of the dual-functional alignment network to focus on multiple discriminant regions on the ship image to be segmented.

[0025] The third aspect of the present invention provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps in the SAR ship image segmentation method based on a dual-functional neural network as described in the first aspect of the present invention are implemented.

[0026] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored on the memory and executable on the processor. When the processor executes the program, the steps in the SAR ship image segmentation method based on a dual-functional neural network as described in the first aspect of the present invention are implemented.

[0027] The above one or more technical solutions have the following beneficial effects:

[0028] (1) The present invention performs image segmentation on the ship image to be segmented based on a dual-functional neural network; wherein, the dual-functional neural network includes a backbone network and a dual-functional alignment network. When performing image segmentation on the ship image to be segmented, the attention distribution of the network is dynamically modulated based on the dual-network mechanism of the dual-functional alignment network to focus on multiple discriminant regions on the ship image to be segmented. This avoids the problem that the existing image segmentation models are prone to missing the objects to be segmented. Therefore, the present invention can effectively improve the accuracy of SAR ship image segmentation.

[0029] (2) The present invention uses the dual-functional alignment network to generate a classification feature map and a regression feature map to complete image feature extraction; after completing image feature extraction based on the dual-functional neural network and obtaining the RN box and classification score, the final segmentation mask is output based on the mask segmentation head to achieve SAR ship image segmentation. This application uses the mask segmentation head (MaskHead) as the output head, which is designed based on the lightweight convolution framework commonly used in variable object detection networks and instance segmentation networks, and can reduce the running time of the model and greatly reduce the cache generated during operation. Therefore, the present invention has better segmentation efficiency.

[0030] The advantages of the additional aspects of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0032] Figure 1 It is a flowchart of the SAR ship image segmentation method based on the dual-functional neural network in the first embodiment of the present invention.

[0033] Figure 2 It is a schematic overall framework diagram of the function implementation and front-back connection of the dual-functional neural network in the first embodiment of the present invention.

[0034] Figure 3 It is a schematic structural diagram of the dual-functional alignment module matrix in the first embodiment of the present invention.

[0035] Figure 4 It is a schematic structural diagram of the mask segmentation head in the first embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.

[0038] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0039] Embodiment 1

[0040] This embodiment discloses a SAR ship image segmentation method based on a dual-functional neural network.

[0041] As Figure 1 shown, the SAR ship image segmentation method based on a dual-functional neural network includes:

[0042] Step S1, obtain the ship image to be segmented and the SAR ship image dataset, and divide the obtained SAR ship image dataset;

[0043] Step S2, use the divided SAR ship image dataset to train the dual-functional neural network;

[0044] Step S3, perform image segmentation on the ship image to be segmented based on the trained dual-functional neural network; wherein, the dual-functional neural network includes a backbone network and a dual-functional alignment network. When performing image segmentation on the ship image to be segmented, the attention distribution of the network is dynamically modulated based on the dual-network mechanism of the dual-functional alignment network to focus on multiple discriminant regions on the ship image to be segmented.

[0045] Based on the above process, the present invention can, on the basis of improving the efficiency of SAR ship image segmentation, maximize the attention to multiple discriminant regions in the image segmentation process to prevent the omission of segmentation details, and then improve the accuracy of SAR ship image segmentation. For the convenience of further understanding of the technical solution of the present invention, the following further explains and illustrates the specific implementation steps in the technical solution of the present invention.

[0046] Step S1, obtain the ship image to be segmented and the SAR ship image dataset, and divide the obtained SAR ship image dataset.

[0047] In this embodiment, the publicly available dataset HRSID (High Resolution SAR Images Dataset) is used as the SAR ship image dataset. This dataset is a high-resolution synthetic aperture radar (SAR) image dataset specifically for ship instance segmentation tasks; it contains 5,604 high-resolution SAR images and 16,951 ship instances, covering various scenarios such as different resolutions, polarizations, sea states, sea areas, and coastal ports. The resolution of this HRSID is divided into 0.5 meters, 1 meter, and 3 meters, covering multiple cities; at the same time, this dataset covers a variety of complex scenarios and a wide variety of multi-scale target ships.

[0048] After obtaining the ship image to be segmented and the SAR ship image dataset, preprocessing operations need to be performed on them. Specifically:

[0049] First, denoising is performed. The Lee filter is used for denoising. The Lee filter is a filter based on local region statistical characteristics and is suitable for removing speckle noise in SAR images. When performing denoising, it can be completed in four small steps, namely:

[0050] The first step is to select the window size, and a fixed 3×3 window is selected in the neighborhood area of the pixel to be filtered; the second step is to calculate the local mean and local variance to reflect the overall characteristics of the area; the third step is to calculate the global mean and global variance of the image for comparison with the previous step; the fourth step is to perform weighted averaging. By the relationship between the local mean and the global mean, local variance, and global variance, the weight factor is calculated to determine the weighting degree of the local mean and the global mean in the filtering process.

[0051] Furthermore, the calculation process of the weight factor can be completed by the following formula, that is:

[0052]

[0053] where w represents the weight factor, represents the variance of the local region, represents the variance of the noise. The larger the weight factor w, the greater the pixel change in the local region. At this time, the weight will rely more on the local mean; on the contrary, if the local variance is small, it means that the region is relatively smooth and the weight will tend to the global mean.

[0054] Subsequently, image enhancement is performed. The image is enhanced through operations such as rotation, cropping (randomly cropping a part of the image to avoid model overfitting), and geometric transformation (such as translation or affine transformation to increase the diversity of training data). Finally, the normalization operation is performed to normalize the image pixel values to the range of [0,1].

[0055] Adopt the same format as the Microsoft Common Objects in Context (MS COCO) dataset, and divide the SAR ship image dataset into a training set, a validation set, and a test set according to the ratio of 7:2:1, that is, three JavaScript Object Notation (json) files containing training set data, validation set data, and test set data. Among them, the MS COCO dataset is a large image dataset developed and maintained by Microsoft.

[0056] Step S2: Use the divided SAR ship image dataset to train the dual-functional neural network.

[0057] Input the training set into the dual-functional neural network to train the dual-functional neural network. To ensure the training effect, after training, immediately perform the validation operation and the test operation in sequence. When both the validation result and the test result are in an ideal state, it is considered that the training is completed.

[0058] As an optional embodiment, the validation process is as follows: First, input the validation set images into the dual-functional neural network to generate RB-Boxes and instance segmentation masks. Subsequently, verify whether the model is ideal by calculating the Recall metric. Specifically, as an optional embodiment, the validation process includes two processes: the evaluation of the Recall rate of the rotated bounding box (RB-Boxes) and the evaluation of the Recall rate of the instance segmentation mask. First, regarding the evaluation of the Recall rate of the rotated bounding box: For each real target (represented by the annotated RB-Box), find the predicted RB-Box with the highest IoU (Intersection over Union, considering rotation) with it; if this highest IoU value exceeds the set threshold (e.g., 0.5), then it is considered that the real target is correctly detected; calculate the ratio of the number of correctly detected real targets to the total number of real targets to obtain the Recall rate of the RB-Boxes. Second, regarding the evaluation of the Recall rate of the instance segmentation mask: For the segmentation mask of each real target, find the predicted segmentation mask with the highest IoU (without considering rotation, because segmentation is usually performed on images in a fixed direction) with it; similarly, if the highest IoU value exceeds the set threshold (e.g., 0.5 or higher, depending on the task difficulty and dataset characteristics), then it is considered that the segmentation of the real target is correctly predicted; calculate the ratio of the number of correctly segmented real targets to the total number of real targets to obtain the Recall rate of the instance segmentation. The higher the Recall value, the better the performance of the model in detecting targets. In this embodiment, it is considered that the Recall values of the rotated bounding box and the instance segmentation are not less than 0.9 to meet the validation standard of the model.

[0059] As an alternative embodiment, the testing process is as follows: First, the validation set images are input into the dual-functional neural network to generate RB-Boxes and instance segmentation masks; subsequently, the AP metric is calculated to test whether the model is ideal. Among them, the testing criterion is that the higher the value of AP, the better the model performance. Specifically, as an alternative embodiment, first, the output results of the object detection algorithm are sorted from high to low according to the confidence level; then, the precision and recall are calculated at each confidence threshold; among them, the precision refers to the proportion of the instances predicted as positive classes by the model that are actually positive classes; the recall refers to the proportion of all actual positive class instances that are correctly predicted as positive classes by the model. Subsequently, according to the change curves of the precision and recall, a PR curve can be obtained; finally, the area under the PR curve is calculated as the AP value, that is, the average precision. The higher the AP value, the better the model performance in detecting the target. In this embodiment, it is considered that when the AP value is not lower than 0.7, the testing standard of the model is achieved.

[0060] Step S3: Perform image segmentation on the ship image to be segmented based on the trained dual-functional neural network; among them, the dual-functional neural network includes a backbone network and a dual-functional alignment network. When performing image segmentation on the ship image to be segmented, the attention distribution of the network is dynamically modulated based on the dual-network mechanism of the dual-functional alignment network to focus on multiple discriminant regions on the ship image to be segmented.

[0061] Use the dual-functional neural network to perform image segmentation on the ship image to be segmented; among them, the dual-functional neural network includes a backbone network and a dual-functional alignment network. Specifically: the backbone network combines the residual network Resnet, the feature pyramid network FPN, and the region proposal network RPN at the same time, that is, the backbone network is actually a combination of Resnet, FPN, and RPN. After the backbone network extracts features, they are transmitted to the dual-functional alignment network to generate classification feature maps and regression feature maps; then, they are input into the mask segmentation head (MaskHead) for segmentation; finally, the final segmentation result is output. The overall network structure diagram and the front-back connection relationship of the dual-functional neural network are as Figure 2 shown. This dual-functional neural network structure improves the accuracy of the ship strength segmentation task in SAR images.

[0062] Furthermore, the Residual Network (Resnet) in the backbone network is a deep convolutional neural network that introduces the concept of residual learning. Specifically, the Residual Network (Resnet) consists of multiple residual blocks, each of which includes multiple convolutional layers and directly adds the input to the output through a shortcut connection. This design enables the network to learn residuals, making it easier to optimize. With the help of residual connections, the Residual Network (ResNet) can build very deep networks (such as ResNet-50, ResNet-101, etc.) without easily suffering from the problem of vanishing gradients. At the same time, deep residual networks can learn more abstract and richer hierarchical features, and have very strong representation capabilities for complex image content.

[0063] Furthermore, the Feature Pyramid Network (FPN) in the backbone network is a multi-scale feature fusion network used for object detection and image segmentation. It enhances the model's detection ability for objects of different scales by constructing a feature pyramid to fuse information at different scales. Specifically, the Feature Pyramid Network (FPN) constructs a feature pyramid using a top-down path, upsamples high-level features, and then makes lateral connections with lower-level feature maps. This approach enables the integration of features at different levels and retains detailed information. By using FPN, the model can more effectively handle small and large objects, avoid information loss, and improve detection accuracy.

[0064] Furthermore, the Region Proposal Network (RPN) in the backbone network is used to generate high-quality candidate regions from the feature map. The design of RPN aims to achieve fast and efficient object detection and reduce the computational overhead of traditional methods. RPN generates multiple anchor boxes (bounding boxes) with fixed sizes and aspect ratios at each position in the feature map, and performs foreground / background classification and bounding box regression on each anchor box through a classification subnet and a regression subnet. After non-maximum suppression (NMS) processing, RPN filters out high-quality candidate regions for use in subsequent object detection steps.

[0065] The present invention uses Resnet, FPN, and RPN as backbone networks. Mixing the three can maximize their advantages, helping to improve deep and diverse features to process objects of different scales, and then efficiently generating candidate regions. Specifically, the workflow of the backbone network is as follows: First, the input object (the ship image to be segmented) arrives at the Resnet network (a series of Resnet feature maps are generated through multiple convolutional layers and residual blocks, and ResNet provides deep semantic information and local detail information); immediately afterwards, the ResNet feature maps are input into the FPN network, and the FPN network generates multi-scale feature maps through upsampling and feature fusion (FPN ensures that the network can process objects of different sizes); subsequently, the multi-scale feature maps output by the FPN network are sent to the RPN network to generate object candidate regions (the RPN network determines the accurate position of each candidate region through object classification and bounding box regression); finally, a semantic feature map is output through the RPN network, and the semantic feature map is input into the dual-functional alignment network.

[0066] As Figure 2 , Figure 3 shown, the dual-functional alignment network adopted by the present invention includes a dual-functional alignment module matrix and a dual-functional alignment head module.

[0067] Furthermore, the dual-functional alignment module matrix (i.e., the DAM module) includes a first functional alignment module and a second functional alignment module; among them, the first functional alignment module can use classification alignment convolution to extract classification features on the semantic feature map to generate a classification feature map; the second functional alignment module can use regression all alignment convolution to extract regression features on the semantic feature map to generate a regression feature map. Therefore, by using the dual-functional alignment module matrix, two different feature maps with different features can be extracted, that is, using the DAM module can better obtain the shape information of the representative features of the ship and improve the accuracy of segmentation.

[0068] The dual-functional alignment module matrix can enhance the model's attention to key regions in the image by dynamically adjusting the attention weights, so as to enhance the segmentation accuracy; among them, the dual-functional alignment module matrix takes the semantic feature map as the input. Further, regarding how to enhance the model's attention to key regions in the image by dynamically adjusting the attention weights, it can be achieved through the following process: First, this embodiment proposes a dynamic sampling strategy. RAConv (Regression Alignment Convolution): Predict the center coordinates, width, height, and rotation angle of the convolution kernel through the R-Anchor branch, and dynamically adjust the sampling area to cover the boundary and surrounding context of the target; this dynamic adjustment enables the model to pay more attention to the boundary region of the target, thereby improving the accuracy of the regression task. CAConv (Classification Alignment Convolution): Predict the sampling offset through the C-Offset branch, densely sample the internal region of the target, and ignore the redundant background; this dynamic adjustment enables the model to pay more attention to the internal region of the target, thereby improving the accuracy of the classification task. Subsequently, task-driven feature extraction is performed. Regression task: The sampling points of RAConv are evenly distributed around the target region and its boundary, ensuring that the model can capture the boundary information of the target and enhance the attention to the boundary region. Classification task: The sampling points of CAConv are densely distributed in the internal region of the target, ensuring that the model can capture the overall semantic information of the target and enhance the attention to the internal region of the target. Then, parametric feature alignment is performed. R-Anchor branch: Predict the geometric parameters (center coordinates, width, height, and rotation angle) of the convolution kernel through a lightweight convolutional network, and dynamically adjust the position and shape of the convolution kernel to align it with the direction and shape of the target. C-Offset branch: Predict the sampling offset through a lightweight convolutional network, and dynamically adjust the position of the sampling points to make them densely distributed in the internal region of the target. Finally, feature fusion and weighting are performed. RAConv and CAConv: Perform weighted summation on the feature map through deformable convolution, and dynamically adjust the weight of each sampling point to make it pay more attention to the features of the key region. Specifically, RAConv and CAConv extract the regression feature map and the classification feature map respectively. After these feature maps are weighted and fused, they can more accurately reflect the key region information of the target.

[0069] The first functional alignment module can use classification alignment convolution to extract classification features on the semantic feature map to generate a classification feature map; the second functional alignment module can use regression all alignment convolution to extract regression features on the semantic feature map to generate a regression feature map. Specifically, as Figure 3 shown, the semantic feature map F m first undergoes three convolutions each, and in the upper part (the second functional alignment module) passes through the kernel convolution C k , and in the lower part (the first functional alignment module) passes through the sampling offset O, and respectively reaches the two permutation branches of the regression alignment convolution RAConv and the classification alignment convolution CAConv. Wm ×H m × The number of channels represents a feature map; where, W m represents the bandwidth, and H m represents the height. FmRAC is the obtained regression feature map, and FmCAC is the obtained classification feature map.

[0070] The process of the first functional alignment module and the second functional alignment module obtaining classification features and regression features can be achieved through the following process:

[0071] First, perform a deformation process on the initial sampling position to obtain the sampling position with kernel alignment; where, the calculation process of the sampling position with kernel alignment is shown in the following formula:

[0072]

[0073] Where, represents the convolution kernel alignment sampling position of the regression alignment convolution RAConv, represents the convolution kernel alignment sampling position of the classification alignment convolution (CAConv); P represents the initial sampling position of the standard convolution kernel, which is specifically defined as {(x,y)|x = -1,0,1; y = -1,0,1}, that is, the 9 sampling points of a 3x3 convolution kernel. O represents the 9 sampling offsets of CAConv, which are specifically defined as {(O xi ,O yi )|i = 1,2,...,9}. ΔW and ΔH respectively represent 1 / 3 of the convolution kernel width and height, that is * represents element-wise multiplication.

[0074] Subsequently, perform rotation and translation on the sampling position with kernel alignment to obtain the sampling position with feature alignment; where, the calculation process of the sampling position with feature alignment is shown in the following formula:

[0075]

[0076] Where, represents the feature alignment sampling position of the regression alignment convolution RAConv, represents the feature alignment sampling position of the classification alignment convolution CAConv; (X,Y) represents the center coordinates of the convolution kernel. R θ represents the rotation matrix, which is specifically defined as R θ =(cos(θ), -sin(θ); sin(θ), cos(θ)), and θ represents the rotation angle of the convolution kernel.

[0077] Finally, the calculated sampling position is aligned with the ship through the convolution kernel, and the specific alignment process is shown in the following formula:

[0078]

[0079] Among them, represents the regression feature map extracted by the regression alignment convolution RAConv, represents the classification feature map extracted by the classification alignment convolution CAConv. and respectively represent the convolution weights of the regression alignment convolution RAConv and the classification alignment convolution CAConv; F m () represents the feature value at the position within () in the feature map, represents the feature alignment sampling position of the regression alignment convolution RAConv at the i-th position, represents the feature alignment sampling position of the classification alignment convolution CAConv at the i-th position.

[0080] Furthermore, after receiving the classification feature map and the regression feature map generated by the dual-functional alignment module matrix, the dual-functional alignment head module can analyze based on the attention mechanism to generate RB boxes and classification scores; among them, the RB box is used to locate and focus on the target area in the image, which can help the model recognize and extract the features of each target instance in the image, so as to perform instance segmentation more precisely.

[0081] The dual-functional alignment head module DAM Head enhances the performance of instance segmentation by introducing the attention mechanism, especially when dealing with complex backgrounds or overlaps between objects.

[0082] As Figure 4 shown, after completing image feature extraction based on the dual-functional neural network and obtaining the RN boxes and classification scores, the final segmentation mask can be output based on the mask segmentation head to achieve SAR ship image segmentation. The present invention uses the mask segmentation head (MaskHead) as the output head, which is designed based on the lightweight convolution framework commonly used in variable object detection networks and instance segmentation networks, and can reduce the running time of the model and greatly reduce the cache generated during operation; among them, the MaskHead is the final instance mask obtained from the output of the three-layer convolution network and instance mask prediction to achieve pixel-level segmentation.

[0083] As Figure 4 shown, F m t represents the texture feature map of the set regression feature map and the classification feature map, which undergoes three-layer convolution output and then generates the final segmentation mask through mask instance prediction. Specifically: the instance mask in the regression prediction RB box (i.e., RB-Boxes) is predicted, and finally the mask is transferred to the original image.

[0084] Based on the above design of the ship image segmentation model in terms of structure, the present invention can, while improving the efficiency of SAR ship image segmentation, pay the greatest attention to multiple discriminant regions during the image segmentation process to prevent the omission of segmentation details, thereby improving the accuracy of SAR ship image segmentation.

[0085] Embodiment 2

[0086] This embodiment discloses a SAR ship image segmentation system based on a dual-functional neural network.

[0087] A SAR ship image segmentation system based on a dual-functional neural network includes:

[0088] An image acquisition module, configured to: acquire the ship image to be segmented and the SAR ship image dataset, and divide the obtained SAR ship image dataset;

[0089] A model training module, configured to: train the dual-functional neural network by using the divided SAR ship image dataset;

[0090] A SAR ship image segmentation module, configured to: perform image segmentation on the ship image to be segmented based on the trained dual-functional neural network; wherein, the dual-functional neural network includes a backbone network and a dual-functional alignment network, and when performing image segmentation on the ship image to be segmented, the attention distribution of the network is dynamically modulated based on the dual-network mechanism of the dual-functional alignment network to pay attention to multiple discriminant regions on the ship image to be segmented.

[0091] Embodiment 3

[0092] The purpose of this embodiment is to provide a computer-readable storage medium.

[0093] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the SAR ship image segmentation method based on a dual-functional neural network as described in Embodiment 1 of the present disclosure.

[0094] Embodiment 4

[0095] The purpose of this embodiment is to provide an electronic device.

[0096] An electronic device, including a memory, a processor, and a program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the SAR ship image segmentation method based on a dual-functional neural network as described in Embodiment 1 of the present disclosure.

[0097] In the devices of the above second, third, and fourth embodiments, the steps involved correspond to those of the method embodiment one. For specific implementation manners, reference may be made to the relevant description part of embodiment one. The term "computer-readable storage medium" should be understood to include a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to execute any method in the present invention.

[0098] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device for execution by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0099] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A SAR ship image segmentation method based on a dual-function neural network is characterized by: include: Obtaining a ship image to be segmented and a SAR ship image dataset, and dividing the obtained SAR ship image dataset; The dual-function neural network is trained using the divided SAR ship image dataset; Image segmentation is performed on the ship image to be segmented based on a trained dual-function neural network; wherein the dual-function neural network includes a backbone network and a dual-function alignment network. When image segmentation is performed on the ship image to be segmented, the network's attention distribution is dynamically modulated based on the dual-network mechanism of the dual-function alignment network to focus on multiple discrimination areas on the ship image to be segmented.

2. The SAR ship image segmentation method based on a dual-function neural network as claimed in claim 1, characterized in that: The backbone network combines the residual network Resnet, the feature pyramid network FPN and the region proposal network RPN.

3. The SAR ship image segmentation method based on a dual-function neural network as claimed in claim 2, characterized in that: The residual network Resnet is composed of multiple residual blocks, each of which includes multiple convolutional layers; the feature pyramid network FPN adopts a top-down path to build a feature pyramid, upsamples high-level features, and then laterally connects them with feature maps of lower layers; the region proposal network RPN generates a bounding box with a fixed size and aspect ratio at each position of the obtained feature map to generate a candidate region from the feature map.

4. The SAR ship image segmentation method based on a dual-function neural network as claimed in claim 1, characterized in that: The dual-function alignment network includes a dual-function alignment module base and a dual-function alignment head module.

5. The SAR ship image segmentation method based on dual-function neural network as claimed in claim 4, characterized in that: The dual-function alignment module matrix includes a first function alignment module and a second function alignment module; wherein the first function alignment module uses classification alignment convolution to extract classification features on a semantic feature map to generate a classification feature map; and the second function alignment module uses regression alignment convolution to extract regression features on a semantic feature map to generate a regression feature map.

6. The SAR ship image segmentation method based on a dual-function neural network according to any one of claims 4 to 5, characterized in that: After receiving the classification feature map and regression feature map generated by the dual-function alignment module base, the dual-function alignment head module performs analysis based on the attention mechanism to generate a target area and a classification score.

7. The SAR ship image segmentation method based on dual-function neural network according to claim 1, characterized in that: Image segmentation is performed on the ship image to be segmented based on the trained dual-function neural network. After completing image feature extraction and obtaining RN boxes and classification scores based on the dual-function neural network, the final segmentation mask is output based on the mask segmentation head to achieve SAR ship image segmentation.

8. SAR ship image segmentation system based on dual-function neural network, characterized in that: include: The image acquisition module is configured to: acquire the ship image to be segmented and the SAR ship image data set, and divide the obtained SAR ship image data set; The model training module is configured to: train the dual-function neural network using the divided SAR ship image data set; The SAR ship image segmentation module is configured to: perform image segmentation on the ship image to be segmented based on a trained dual-function neural network; wherein the dual-function neural network includes a backbone network and a dual-function alignment network, and when performing image segmentation on the ship image to be segmented, the dual-network mechanism of the dual-function alignment network is used to dynamically modulate the network's attention distribution to focus on multiple discrimination areas on the ship image to be segmented.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the SAR ship image segmentation method based on a dual-function neural network as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the SAR ship image segmentation method based on a dual-function neural network as described in any one of claims 1 to 7 are implemented.

Images