Underwater image instance segmentation method, system, storage medium and electronic device

Abstract

The present invention belongs to the field of image segmentation technology and provides a method, system, storage medium, and electronic device for underwater image instance segmentation. The image segmentation method includes extracting multi-level features of underwater images; reconstructing a global fine-grained feature map with the highest resolution based on a differential similarity graph attention mechanism to obtain a reconstructed global fine-grained feature map; downsampling a region of interest from the reconstructed global fine-grained feature map and fusing it with a local coarse-grained feature map to obtain foreground features and boundary features; extracting deep-level features of the foreground features and boundary features, respectively, to generate a first instance mask and a second instance mask; performing a two-fold upsampling operation on the first instance mask and replacing the boundary portion of the operation result with the corresponding portion of the second instance mask to obtain a final output instance mask; and optimizing the output instance mask using a total mask loss function to obtain an underwater image instance segmentation result.

Description

Technical Field

[0001] The present invention belongs to the technical field of image segmentation, and in particular relates to an underwater image instance segmentation method, system, storage medium and electronic equipment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of marine exploration and protection, underwater image analysis technology plays an increasingly important role. Underwater image instance segmentation is the basis and key step for underwater image analysis and understanding.

[0004] Currently, there is a lack of datasets specifically designed for instance segmentation in underwater images. Existing annotated data is related to instance segmentation applications for specific objects or is only applicable to specific tasks (such as object detection and semantic segmentation). These datasets are not suitable for multi-class instance segmentation in underwater images. Furthermore, due to wavelength- and distance-dependent attenuation and scattering, the quality of underwater images is inevitably degraded. Furthermore, marine snow, formed by plankton and other organisms in the ocean, can also cause varying degrees of noise, which severely degrades image quality. This makes existing natural image instance segmentation algorithms incapable of directly applying these degraded underwater images, thereby reducing the accuracy of object recognition in underwater images. Summary of the Invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method, system, storage medium and electronic device for underwater image instance segmentation, which can accurately perform instance segmentation on underwater images and ultimately improve the target recognition accuracy in underwater images.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A first aspect of the present invention provides a method for underwater image instance segmentation.

[0008] A method for underwater image instance segmentation, comprising:

[0009] Extract multi-level features of underwater images to obtain global fine-grained feature maps and local coarse-grained feature maps;

[0010] The highest resolution global fine-grained feature map is reconstructed based on the differential similarity graph attention mechanism to obtain the reconstructed global fine-grained feature map;

[0011] The region of interest is downsampled from the reconstructed global fine-grained feature map and fused with the local coarse-grained feature map to obtain foreground features and boundary features;

[0012] Extracting deep features of foreground features and boundary features respectively, and generating a first instance mask and a second instance mask respectively; performing a two-fold upsampling operation on the first instance mask, and replacing the boundary portion in the two-fold upsampling operation result with the corresponding portion of the second instance mask, to obtain the final output instance mask;

[0013] The output instance mask is optimized using the total mask loss function to obtain the underwater image instance segmentation result; wherein the total mask loss function includes the boundary area weight loss function.

[0014] A second aspect of the present invention provides an underwater image instance segmentation system.

[0015] An underwater image instance segmentation system, comprising:

[0016] A multi-level feature extraction module is used to extract multi-level features of underwater images to obtain global fine-grained feature maps and local coarse-grained feature maps;

[0017] The differential similarity graph attention module is used to reconstruct the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism to obtain a reconstructed global fine-grained feature map;

[0018] A multi-layer feature refinement module, which is used to downsample the region of interest from the reconstructed global fine-grained feature map and fuse it with the local coarse-grained feature map to obtain foreground features and boundary features;

[0019] An instance mask output module is used to extract deep features of foreground features and boundary features, respectively, and generate a first instance mask and a second instance mask respectively; a two-fold upsampling operation is performed on the first instance mask, and the boundary portion of the two-fold upsampling operation result is replaced with the corresponding portion of the second instance mask to obtain the final output instance mask;

[0020] The instance mask optimization module is used to optimize the output instance mask using the total mask loss function to obtain the underwater image instance segmentation result; wherein the total mask loss function includes the boundary area weight loss function.

[0021] A third aspect of the present invention provides a computer-readable storage medium.

[0022] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the underwater image instance segmentation method as described above.

[0023] A fourth aspect of the present invention provides a computer device.

[0024] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the underwater image instance segmentation method as described above when executing the program.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The present invention reconstructs the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism, which can help the model perform better inference by collecting as many degraded details as possible to compensate for the information loss caused by quality degradation and downsampling operations.

[0027] The present invention uses multi-layer feature refinement to fuse the reconstructed global fine-grained features into the local mask prediction as supplementary details, which can improve the quality of mask prediction and increase the model perception domain. The output instance mask is optimized using the total mask loss function, guiding the network to pay more attention to the segmentation of the boundary part and assigning more weight to the boundary area, thereby forcing the network to pay more attention to the classification within the boundary pixels and make more accurate predictions.

[0028] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0030] Figure 1 is a flow chart of an underwater image instance segmentation method according to an embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of a differential similarity graph attention module according to an embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of a multi-layer feature refinement module according to an embodiment of the present invention;

[0033] Figure 4 2 is a comparative effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0036] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0037] Example 1

[0038] This embodiment also produces a universal underwater image instance segmentation dataset UIIS. First, about 25,000 images from various fields were collected for underwater image enhancement, semantic segmentation, and object detection. These images include different natural underwater scenes and are suitable for various fields such as marine exploration, marine ecological maintenance, and human-machine intelligent collaborative applications. Afterwards, approximately 5,000 images filtered by underwater color image quality evaluation (UCIQE) and underwater image quality measurement indicators were obtained and annotated in detail. The dataset was annotated by volunteers with basic dataset annotation experience and basic knowledge of marine biology. Sparsely annotated polygons are used to annotate each instance in the image, and the resulting annotation data will be stored in the mainstream COCO style format to facilitate the use of the dataset by most common frameworks and models. Each image is annotated by at least three volunteers and then evaluated by another person to select the best annotation and refine it. To classify potentially confusing objects such as plants / coral reefs, vertebrates / invertebrates, etc., we followed the guidance in the MarineSpecies Identification Portal and The Ocean Animal Encyclopedia. We also filtered out images that could not reach a consensus or could not be finely labeled to ensure the rigor of the dataset. Finally, we obtained 4,628 images and corresponding annotations.

[0039] like Figure 1 As shown, this embodiment provides an underwater image instance segmentation method, which specifically includes the following steps:

[0040] Step 1: Extract multi-level features of underwater images to obtain global fine-grained feature maps and local coarse-grained feature maps.

[0041] In the specific implementation process, the backbone network is combined with the feature pyramid network to perform multi-level feature extraction on underwater images.

[0042] Step 2: Reconstruct the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism to obtain the reconstructed global fine-grained feature map.

[0043] like Figure 2 As shown in Figure 2, in step 2, the process of reconstructing the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism includes:

[0044] Step 2.1: Take the highest resolution global fine-grained feature map P2 as the input feature map and perform feature extraction to obtain the intermediate feature map;

[0045] For example: First, pass P2 to the convolution layer with a stride of s and a convolution kernel size of s×s. Step 2.2: Treat each pixel of the intermediate feature map as a graph node. The edge between each two nodes is determined by the Euclidean distance between the nodes. Information is aggregated through the edges to update the features of each graph node.

[0046] For example, each pixel of the output feature map is then considered as a graph node, and the edge between each two nodes is determined by the Euclidean distance between the nodes, so that GAT is used to aggregate information through the edge. At the same time, in order to collect as many different degraded residual detail features as possible and reduce computational cost and memory consumption, we only connect the k nodes farthest from each node. The attention coefficient a of graph node i and its first-order neighbor node j ij can be expressed as:

[0047]

[0048] where h i , h j are the features of graph node i and graph node j respectively, W and A are two shared feature conversion weights, N i represents all first-order neighbor nodes of node i, || represents feature concatenation; exp(·) represents the exponential of e; σ(·) represents an activation function.

[0049] Step 2.3: The updated intermediate feature map is reshaped and upsampled by deconvolution operations to obtain the residual information flow of the input image and add it to the input features to finally obtain the reconstructed global fine-grained feature map.

[0050] Among them, the graph nodes of the updated intermediate feature graph have the same dimension as the graph nodes of the intermediate feature graph.

[0051] After the information from neighboring nodes is aggregated, the features of each graph node are updated using the following formula:

[0052]

[0053] Here, δ(·) represents another activation function different from σ(·).

[0054] The graph nodes in the output feature map have the same dimension as the graph nodes in the input feature map. The output feature map is then reshaped and upsampled by deconvolution operations to obtain the residual information flow of the input image and add it to the input features to supplement the detail information lost in the original image when the quality is degraded and the network is downsampled. Finally, the reconstructed feature P is obtained. * .

[0055] Step 3: Downsample the region of interest from the reconstructed global fine-grained feature map and fuse it with the local coarse-grained feature map to obtain foreground features and boundary features.

[0056] like Figure 3 As shown, in step 3, the process of obtaining foreground features and boundary features includes:

[0057] Step 3.1: Perform convolution operations on the global fine-grained feature map and the local coarse-grained feature map to generate initial instance features;

[0058] Step 3.2: Using the global fine-grained feature map, iteratively refine the initial instance features using a multi-layer feature refinement step;

[0059] Step 3.3: At each iteration, extract local fine-grained features of the corresponding size from the reconstructed global fine-grained feature map, use the local fine-grained features to refine the features of the previous stage, perform 1×1 convolution and two-fold upsampling operations on the refined features to generate the output features of the corresponding stage;

[0060] Step 3.4: Perform the multi-layer feature refinement step twice, and output two features with different resolutions, namely foreground features and boundary features, which are used for foreground prediction and boundary prediction of the instance respectively.

[0061] For example, before multi-layer feature refinement, the features extracted from the feature pyramid by the 14×14 RoI Align operation are first sent to two 3×3 convolutional layers to generate the initial instance features F1. Afterwards, the fine-grained features iteratively refine the initial features F1 through multi-layer feature refinement. At each stage, the RoI Align operation is used to extract the features from the P * The local fine-grained features of the corresponding size are extracted and used to refine the features obtained in the previous stage. 1×1 convolution and two times upsampling operations are then performed on these features to generate the output features of this stage. Since the number of feature channels is reduced to half of the number of input channels before upsampling using 1×1 convolution, the multi-layer feature refinement operation does not introduce excessive computational overhead. The multi-layer feature refinement will be performed twice, with features F2 and F3 of different resolutions as output, which will be used as foreground prediction and boundary prediction of the instance, respectively.

[0062] Step 4: Extract the deep features of the foreground features and boundary features respectively, and generate the first instance mask and the second instance mask respectively; perform a two-fold upsampling operation on the first instance mask, and replace the boundary part in the two-fold upsampling operation result with the corresponding part of the second instance mask to obtain the final output instance mask.

[0063] In a specific implementation, in the process of generating the first instance mask and the second instance mask, a Laplacian operator is used to generate a boundary from a binary mask.

[0064] For example:

[0065] The features F2 and F3 generated by the multi-level feature refinement step are passed into the 1×1 convolution layer to generate instance masks M2 and M3 with different resolutions. We will perform a two-fold upsampling operation on M2 and replace its boundary part with the corresponding part of M3 as the final output. Design a convolution layer The Laplacian operator is used to generate boundaries from binary masks. The stride of the convolution layer is 1, the padding is 1, and the size of the convolution kernel weight p is b×b. The convolution kernel weight p can be specifically expressed as:

[0066]

[0067] Where i, j represents the position in p. Therefore, the boundary mask generation function can be expressed as:

[0068]

[0069] where μ is a hyperparameter that controls the size of the boundary, meaning that pixels in the binary mask are only considered if they are bounded by at least μb 2 A pixel is considered as a boundary pixel only when it is surrounded by pixels with values ​​not equal to the pixel. μ is set to 0.15 based on experience. The final output mask M out It can be defined as:

[0070] M out =f 2× (M2)⊙B 2× +M3⊙(1-B 2× )

[0071] where f 2× is a double upsampling operation, B 2× is defined as f 2× (B(M2)), ⊙ is the bit-by-bit multiplication of pixels.

[0072] Step 5: Use the mask total loss function to optimize the output instance mask to obtain the underwater image instance segmentation result; the mask total loss function includes the boundary area weight loss function.

[0073] For example, the total mask loss function is the sum of the boundary region weight loss function and the binary cross entropy loss function.

[0074] For the loss function part of the model, a boundary learning loss (BLL) is designed to assign more weights to the boundary area:

[0075]

[0076] Where H and W are the height and width of the mask M3, i represents the i-th pixel, the BCE function returns the binary cross entropy loss of the pixel, and ∨ represents the union of two regions. Indicates the boundary area that should be paid attention to, which is defined here as f 2× (B(M2)∨B(G2)), G k Representation and predicted mask M k The corresponding true value.

[0077] A 1×1 convolution is also performed on F1 to generate the mask M1 in the process, and BCE loss is used on M1 and M2 to ensure that the parameters learned by the network in the process are reliable, thereby obtaining the total loss L of the mask part mask for:

[0078]

[0079] Among them L BCE is the BCE (binary cross entropy loss function) loss, λ k is a hyperparameter of the balance weight (experimentally set to λ1 = 0.25 and λ2 = 0.65). Figure 4 (a) to (d) are shown in the figure, which respectively represent the original image group, the image group obtained by the existing Mask R-CNN algorithm, the image group obtained by the existing QueryInst algorithm, and the image group obtained by the underwater image instance segmentation of the present invention. This proves that the present embodiment exhibits strong performance under high saturation and quality degradation conditions.

[0080] Example 2

[0081] This embodiment provides an underwater image instance segmentation system, which specifically includes the following modules:

[0082] (1) A multi-level feature extraction module, which is used to extract multi-level features of underwater images and obtain global fine-grained feature maps and local coarse-grained feature maps;

[0083] Specifically, in the multi-level feature extraction module, a backbone network combined with a feature pyramid network is used to perform multi-level feature extraction on underwater images.

[0084] (2) a differential similarity graph attention module, which is used to reconstruct the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism to obtain a reconstructed global fine-grained feature map;

[0085] In the differential similarity graph attention module, the process of reconstructing the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism includes:

[0086] The highest resolution global fine-grained feature map is used as the input feature map and feature extraction is performed to obtain the intermediate feature map;

[0087] Each pixel in the intermediate feature map is considered as a graph node. The edge between each two nodes is determined by the Euclidean distance between the nodes. Information is aggregated through the edges to update the features of each graph node.

[0088] The updated intermediate feature map is reshaped and upsampled by deconvolution operations to obtain the residual information flow of the input image and add it to the input features to finally obtain the reconstructed global fine-grained feature map.

[0089] The differential similarity graph attention module reconstructs high-quality global detail features by learning different degraded residual detail features between small patches, and reconstructs image details by extracting and combining information through GAT operations.

[0090] (3) A multi-layer feature refinement module, which is used to downsample the region of interest from the reconstructed global fine-grained feature map and fuse it with the local coarse-grained feature map to obtain foreground features and boundary features.

[0091] A multi-layer feature refinement module downsamples regions of interest from the reconstructed global features and fuses them with local coarse-grained features to predict foreground and boundary masks with different resolutions.

[0092] Specifically, in the multi-layer feature refinement module, the process of obtaining foreground features and boundary features includes:

[0093] Perform convolution operations on the global fine-grained feature map and the local coarse-grained feature map to generate initial instance features;

[0094] Utilizing the global fine-grained feature map, the initial instance features are iteratively refined using a multi-layer feature refinement step;

[0095] In each iterative stage, local fine-grained features of corresponding size are extracted from the reconstructed global fine-grained feature map, and the local fine-grained features are used to refine the features of the previous stage. 1×1 convolution and two-fold upsampling operations are performed on the refined features to generate the output features of the corresponding stage;

[0096] The multi-layer feature refinement step is performed twice, and two features with different resolutions are output respectively, namely the foreground feature and the boundary feature, which are used for the foreground prediction and boundary prediction of the instance respectively.

[0097] (4) an instance mask output module, which is used to extract deep features of foreground features and boundary features, respectively, and generate a first instance mask and a second instance mask respectively; perform a two-fold upsampling operation on the first instance mask, and replace the boundary portion in the two-fold upsampling operation result with the corresponding portion of the second instance mask, thereby obtaining a final output instance mask;

[0098] Specifically, in the instance mask output module, in the process of generating the first instance mask and the second instance mask, a Laplacian operator is used to generate a boundary from the binary mask.

[0099] (5) An instance mask optimization module, which is used to optimize the output instance mask using the total mask loss function to obtain the underwater image instance segmentation result; wherein the total mask loss function includes the boundary area weight loss function.

[0100] The BMS (boundary mask strategy) of this embodiment optimizes the final output mask to achieve finer boundaries for underwater instances. The BLL (boundary loss function) assigns more weight to boundary regions, resulting in more accurate predictions.

[0101] Specifically, in the instance mask optimization module, the total mask loss function is the sum of the boundary region weight loss function and the binary cross entropy loss function.

[0102] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0103] This embodiment utilizes differential similarity graphs to reconstruct the highest-resolution features in the feature pyramid network to generate global fine-grained features. A multi-layer feature refinement module is then used to downsample regions of interest from the reconstructed global features and fuse them with local coarse-grained features from the feature pyramid network to predict foreground and boundary masks of varying resolutions. The final output mask is optimized using a boundary mask strategy, achieving finer boundaries for underwater instances. Furthermore, since quality degradation often leads to blurred boundaries of objects in underwater images, the boundary loss function assigns more weight to boundary regions, resulting in more accurate predictions.

[0104] Example 3

[0105] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the underwater image instance segmentation method described above are implemented.

[0106] Example 4

[0107] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the underwater image instance segmentation method described above are implemented.

[0108] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0109] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0111] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for underwater image instance segmentation, characterized in that: include: Extract multi-level features of underwater images to obtain global fine-grained feature maps and local coarse-grained feature maps; The highest resolution global fine-grained feature map is reconstructed based on the differential similarity graph attention mechanism to obtain the reconstructed global fine-grained feature map; The region of interest is downsampled from the reconstructed global fine-grained feature map and fused with the local coarse-grained feature map to obtain foreground features and boundary features; Extracting deep features of foreground features and boundary features respectively, and generating first instance masks and second instance masks accordingly; Perform a two-fold upsampling operation on the first instance mask, and replace the boundary portion of the two-fold upsampling operation result with the corresponding portion of the second instance mask to obtain a final output instance mask; The output instance mask is optimized using the total mask loss function to obtain the underwater image instance segmentation result; wherein the total mask loss function includes the boundary area weight loss function; The process of reconstructing the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism includes: The highest resolution global fine-grained feature map is used as the input feature map and feature extraction is performed to obtain the intermediate feature map; Each pixel in the intermediate feature map is considered as a graph node. The edge between each two nodes is determined by the Euclidean distance between the nodes. Information is aggregated through the edges to update the features of each graph node. The updated intermediate feature map is reshaped and upsampled by deconvolution operations to obtain the residual information flow of the input image and add it to the input features to finally obtain the reconstructed global fine-grained feature map.

2. The underwater image instance segmentation method according to claim 1, wherein: The graph nodes of the updated intermediate feature map have the same dimensions as the graph nodes of the intermediate feature map.

3. The underwater image instance segmentation method according to claim 1, wherein: The process of obtaining foreground features and boundary features includes: Perform convolution operations on the global fine-grained feature map and the local coarse-grained feature map to generate initial instance features; Utilizing the global fine-grained feature map, the initial instance features are iteratively refined using a multi-layer feature refinement step; In each iterative stage, local fine-grained features of corresponding size are extracted from the reconstructed global fine-grained feature map, and the local fine-grained features are used to refine the features of the previous stage. Convolution and two-fold upsampling operations to generate the output features of the corresponding stage; The multi-layer feature refinement step is performed twice, and two features with different resolutions are output respectively, namely the foreground feature and the boundary feature, which are used for the foreground prediction and boundary prediction of the instance respectively.

4. The underwater image instance segmentation method according to claim 1, wherein: In the process of generating the first instance mask and the second instance mask, a Laplacian operator is used to generate a boundary from the binary mask.

5. The underwater image instance segmentation method according to claim 1, wherein: The total mask loss function is the sum of the boundary area weight loss function and the binary cross entropy loss function.

6. The underwater image instance segmentation method according to claim 1, wherein: The backbone network is combined with the feature pyramid network to perform multi-level feature extraction on underwater images.

7. An underwater image instance segmentation system, characterized in that: include: A multi-level feature extraction module is used to extract multi-level features of underwater images to obtain global fine-grained feature maps and local coarse-grained feature maps; The differential similarity graph attention module is used to reconstruct the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism to obtain a reconstructed global fine-grained feature map; A multi-layer feature refinement module, which is used to downsample the region of interest from the reconstructed global fine-grained feature map and fuse it with the local coarse-grained feature map to obtain foreground features and boundary features; An instance mask output module is used to extract deep features of foreground features and boundary features, and generate a first instance mask and a second instance mask respectively; Perform a two-fold upsampling operation on the first instance mask, and replace the boundary portion of the two-fold upsampling operation result with the corresponding portion of the second instance mask to obtain a final output instance mask; An instance mask optimization module is used to optimize the output instance mask using a total mask loss function to obtain underwater image instance segmentation results; wherein the total mask loss function includes a boundary region weight loss function; In the differential similarity graph attention module, the process of reconstructing the highest resolution global fine-grained feature map based on the differential similarity graph attention mechanism includes: The highest resolution global fine-grained feature map is used as the input feature map and feature extraction is performed to obtain the intermediate feature map; Each pixel in the intermediate feature map is considered as a graph node. The edge between each two nodes is determined by the Euclidean distance between the nodes. Information is aggregated through the edges to update the features of each graph node. The updated intermediate feature map is reshaped and upsampled by deconvolution operations to obtain the residual information flow of the input image and add it to the input features to finally obtain the reconstructed global fine-grained feature map.

8. The underwater image instance segmentation system according to claim 7, wherein: In the multi-layer feature refinement module, the process of obtaining foreground features and boundary features includes: Perform convolution operations on the global fine-grained feature map and the local coarse-grained feature map to generate initial instance features; Utilizing the global fine-grained feature map, the initial instance features are iteratively refined using a multi-layer feature refinement step; In each iterative stage, local fine-grained features of corresponding size are extracted from the reconstructed global fine-grained feature map, and the local fine-grained features are used to refine the features of the previous stage. Convolution and two-fold upsampling operations to generate the output features of the corresponding stage; Perform the multi-layer feature refinement step twice, outputting two features of different resolutions, namely foreground features and boundary features, which are used for foreground prediction and boundary prediction of the instance respectively; or In the instance mask output module, in the process of generating the first instance mask and the second instance mask, a Laplacian operator is used to generate a boundary from the binary mask; or In the instance mask optimization module, the total mask loss function is the sum of the boundary region weight loss function and the binary cross entropy loss function; or In the multi-level feature extraction module, the backbone network is combined with the feature pyramid network to perform multi-level feature extraction on underwater images.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the underwater image instance segmentation method according to any one of claims 1 to 6 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the underwater image instance segmentation method according to any one of claims 1 to 6 are implemented.

Images