Underwater fish part segmentation method, system and equipment and storage medium

Abstract

The invention discloses an underwater fish part segmentation method, system and device and a storage medium, and the method comprises the steps: obtaining an underwater fish picture, and extracting a first number of image features from the underwater fish picture through a backbone model; performing coding query on the second number of image features to correspondingly obtain a second number of coding features; determining a fourth number of fusion features according to the third number of image features and the second number of coding features; the sum of the second number and the third number is the first number; the fourth number of fusion features respectively and correspondingly generate a fourth number of prediction maps; and fusing the fourth number of prediction maps to generate a fish part segmentation map. Compared with the traditional manual measurement method, the morphological phenotype measurement based on the computer vision technology has the characteristics of low cost, high efficiency, strong generalization and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fish part segmentation, and in particular to an underwater fish part segmentation method, system, device and storage medium. BACKGROUND

[0002] Fish morphological phenotype information is an important data basis for monitoring aquaculture fish, which can be used to evaluate fish growth and health conditions. Using deep learning technology to automatically obtain fish morphological phenotype provides assistance for intelligent decision-making of precise fish breeding, which helps to improve the efficiency and management level of fish aquaculture. Aquaculture fish has become an important source of global food, playing an indispensable role in ensuring food security and meeting the food needs of millions of people around the world. In order to ensure the quality and efficiency of aquaculture, monitoring of cultured fish, accurate understanding of fish growth and health conditions, has become an important foundation work in aquaculture fishery. Fish morphological phenotype information includes fish body appearance, size and part morphology, which is an important basis for monitoring fish culture. Through image segmentation-based methods, the fish body contour and parts are accurately captured to obtain fish shape, size and appearance structure information to measure fish morphological phenotype, which helps to evaluate fish growth and health conditions and improve production efficiency and breeding quality. Early fish morphological phenotype measurement mainly uses manual measurement methods, such as using measuring tools and naked eye observation to obtain phenotype data. However, this measurement method requires manpower and resources, and is easy to cause harm to fish. At the same time, due to the diversity of underwater fish species and the similarity between their parts, the part morphology difference and part cognitive ambiguity make it difficult to obtain morphological phenotype information. The underwater fish has part morphology diversity and structure semantic ambiguity, which leads to inaccurate part recognition and poor segmentation effect. SUMMARY

[0003] Therefore, it is necessary to propose an underwater fish part segmentation method, device, computer equipment and storage medium to solve the above problems.

[0004] An underwater fish part segmentation method, the method comprising: obtaining an underwater fish picture, extracting a first number of image features from the underwater fish picture through a backbone model; encoding and querying a second number of the image features to obtain a second number of encoded features; determining a fourth number of fusion features according to a third number of the image features and the second number of encoded features; the sum of the second number and the third number is the first number; the fourth number of fusion features correspond to generate a fourth number of prediction maps respectively; The fourth number of prediction maps are fused to generate a fish part segmentation map.

[0005] In one embodiment, the underwater fish part segmentation method further comprises: determining a number of correct target pixel points, a number of correct background pixel points, a number of false target pixel points, and a number of false background pixel points in the fish part segmentation map; determining a class average pixel accuracy and an average intersection over union according to the number of target pixel points, the number of correct background pixel points, the number of false target pixel points, and the number of false background pixel points; determining an accuracy of the fish part segmentation map through the class average pixel accuracy and the average intersection over union.

[0006] In one embodiment, the encoding query of the second number of image features to obtain a second number of encoded features comprises: obtaining a query matrix, a key matrix, and a value matrix of each image feature in the second number of image features; performing attention operation on the query matrix, the key matrix, and the value matrix to obtain a plurality of attention scores of each image feature; splicing a plurality of attention scores to obtain a multi-head attention score of each image feature; inputting the multi-head attention score after residual connection and layer normalization processing to a feedforward fully connected network layer, and then performing residual connection and layer normalization processing to obtain an encoded feature.

[0007] In one embodiment, the multi-head attention score is obtained by: wherein Q is a query matrix; K is a key matrix; V is a value matrix; is an image feature; is a weight of the query matrix; is a weight of the key matrix; is a weight of the value matrix; is a dimension of the key matrix; is a linear transformation operation; is an attention score; is a multi-head attention score.

[0008] In one embodiment, the determining the fourth number of fusion features according to the third number of image features and the second number of encoding features is implemented by the following expression: wherein, , , and are local features; , , and are global features; , , and are fusion features; and are image features; , and are encoding features; Conv is a 3x3 convolution; Cat represents channel concatenation; Upsample represents up-sampling; FFC is a 3x3 fast Fourier convolution; CA is channel attention; and FCA is feature local guidance operation.

[0009] In one embodiment, the class average pixel accuracy and the average intersection over union are respectively implemented by the following expressions: wherein, class average pixel accuracy; is average intersection over union; k represents the first number, k is 5; TP is the number of correct target pixels; TN is the number of correct background pixels; FP is the number of false target pixels; and FN is the number of false background pixels.

[0010] An underwater fish part segmentation system, the system comprises: An acquisition module is configured to acquire underwater fish pictures, and extract a first number of image features from the underwater fish pictures by a backbone model; A query module is configured to encode and query a second number of the image features to obtain a second number of encoded features; A determination module is configured to determine a fourth number of fusion features according to a third number of the image features and the second number of encoded features; the sum of the second number and the third number is the first number; A generation module is configured to generate a fourth number of prediction maps respectively corresponding to the fourth number of fusion features; A fusion module is configured to fuse the fourth number of prediction maps to generate a fish part segmentation map.

[0011] A computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the following steps: An acquisition module is configured to acquire underwater fish pictures, and extract a first number of image features from the underwater fish pictures by a backbone model; A query module is configured to encode and query a second number of the image features to obtain a second number of encoded features; A determination module is configured to determine a fourth number of fusion features according to a third number of the image features and the second number of encoded features; the sum of the second number and the third number is the first number; A generation module is configured to generate a fourth number of prediction maps respectively corresponding to the fourth number of fusion features; A fusion module is configured to fuse the fourth number of prediction maps to generate a fish part segmentation map.

[0012] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to make the processor execute the following steps: An acquisition module is configured to acquire underwater fish pictures, and extract a first number of image features from the underwater fish pictures by a backbone model; A query module is configured to encode and query a second number of the image features to obtain a second number of encoded features; A determination module is configured to determine a fourth number of fusion features according to a third number of the image features and the second number of encoded features; the sum of the second number and the third number is the first number; A generation module is configured to generate a fourth number of prediction maps respectively corresponding to the fourth number of fusion features; A fusion module is configured to fuse the fourth number of prediction maps to generate a fish part segmentation map.

[0013] The application obtains an underwater fish picture, extracts a first number of image features from the underwater fish picture through a backbone model, encodes and queries a second number of the image features to obtain a second number of encoded features, determines a fourth number of fusion features according to a third number of the image features and the second number of the encoded features, the sum of the second number and the third number is the first number, the fourth number of fusion features correspond to generate a fourth number of prediction maps respectively, and the fourth number of prediction maps are fused to generate a fish part segmentation map. Compared with a traditional manual measurement method, a morphological phenotype measurement based on a computer vision technology has the characteristics of low cost, high efficiency, strong generalization, etc. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Among them: Figure 1 It is an application environment diagram of the underwater fish part segmentation method in an embodiment; Figure 2 It is a flowchart of the underwater fish part segmentation method in an embodiment; Figure 3 It is a structural block diagram of the underwater fish part segmentation system in an embodiment; Figure 4 It is a flowchart of obtaining the fish part segmentation map in an embodiment; Figure 5 It is a flowchart of obtaining the fusion features in an embodiment; Figure 6 It is a fish part segmentation map displayed by different models in an embodiment; Figure 7 It is a structural block diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0017] Fish morphological phenotype information is an important data basis for monitoring aquaculture fish, which can be used to evaluate fish growth and health status. Using deep learning technology to automatically obtain fish morphological phenotype provides assistance for intelligent decision-making of precise fish farming, which helps to improve the efficiency and management level of fish aquaculture. Fish farming has become an important source of global food, playing an indispensable role in ensuring food security and meeting the food needs of millions of people around the world. In order to ensure the quality and efficiency of aquaculture, monitoring farmed fish, accurately understanding fish growth and health status, has become an important foundation work in aquaculture. Fish morphological phenotype information includes fish body appearance, size and appearance structure, which is an important basis for monitoring fish farming. Through image segmentation-based methods, the fish body contour and parts can be accurately captured to obtain fish shape, size and appearance structure information to measure fish morphological phenotype, which helps to evaluate fish growth and health status, improve production efficiency and farming quality. Early fish morphological phenotype measurement mainly uses manual measurement methods, such as using measuring tools and naked eye observation to obtain phenotype data. However, this measurement method requires manpower and resources, and can easily cause harm to fish. At the same time, due to the diversity of underwater fish species and the similarity between their parts, the difference in part morphology and part recognition ambiguity make it difficult to obtain morphological phenotype information. In order to solve the above technical problems, the underwater fish part segmentation method is provided. Figure 1 An application environment diagram of an underwater fish part segmentation method in an embodiment is shown in FIG. 1. Referring to Figure 1 The underwater fish part segmentation method is applied to an underwater fish part segmentation system. The underwater fish part segmentation system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network, and the terminal 110 can be a desktop terminal or a mobile terminal, and the mobile terminal can be at least one of a mobile phone, a tablet computer, a notebook computer, etc. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The terminal 110 is used to obtain an underwater fish picture, extract a first number of image features from the underwater fish picture through a backbone model, encode and query a second number of the image features to obtain a second number of encoding features, and the server 120 is used to determine a fourth number of fusion features according to a third number of the image features and the second number of the encoding features. The sum of the second number and the third number is the first number. The fourth number of fusion features respectively correspond to generate a fourth number of prediction maps. The fourth number of prediction maps are fused to generate a fish part segmentation map.

[0018] As Figure 2As shown, in one embodiment, an underwater fish part segmentation method is provided. The method can be applied to a terminal or a server, and the present embodiment is exemplified by application to a terminal. The underwater fish part segmentation method specifically comprises the following steps: S10: Obtain an underwater fish picture, and extract a first number of image features from the underwater fish picture through a backbone model; S20: Encode and query the second number of image features to obtain a second number of encoded features; S30: Determine a fourth number of fusion features according to the third number of image features and the second number of encoded features; the sum of the second number and the third number is the first number; S40: The fourth number of fusion features respectively correspond to generate a fourth number of prediction maps; S50: Fuse the fourth number of prediction maps to generate a fish part segmentation map.

[0019] The present application obtains an underwater fish picture, extracts a first number of image features from the underwater fish picture through a backbone model, encodes and queries the second number of image features to obtain a second number of encoded features, determines a fourth number of fusion features according to the third number of image features and the second number of encoded features, the sum of the second number and the third number is the first number, the fourth number of fusion features respectively correspond to generate a fourth number of prediction maps, and fuse the fourth number of prediction maps to generate a fish part segmentation map. Compared with the traditional manual measurement method, the morphological phenotype measurement based on computer vision technology has the characteristics of low cost, high efficiency, strong generalization, etc.

[0020] In one embodiment, the underwater fish part segmentation method further comprises: S60: Determine the number of correct target pixel points, the number of correct background pixel points, the number of false target pixel points, and the number of false background pixel points in the fish part segmentation map; S70: Determine the class average pixel accuracy and the average intersection over union according to the number of target pixel points, the number of correct background pixel points, the number of false target pixel points, and the number of false background pixel points; S80: Determine the accuracy of the fish part segmentation map through the class average pixel accuracy and the average intersection over union.

[0021] In one embodiment, the encoding and querying of the second number of image features in step S20 comprises: S201: Obtain a query matrix, a key matrix, and a value matrix of each image feature in the second number of image features; S202: Perform attention operation on the query matrix, the key matrix, and the value matrix to obtain a plurality of attention scores of each image feature; S203: Concatenate the plurality of attention scores to obtain a multi-head attention score of each image feature; S204: Input the multi-head attention score into a feedforward fully connected network layer after residual connection and layer normalization processing, and then perform residual connection and layer normalization processing to obtain an encoded feature.

[0022] Different from fish target segmentation, fish part segmentation needs to pay attention to the overall target of the fish and accurately analyze each part inside the fish body. Different parts of the fish may have high similarity in shape and texture, increasing the difficulty of distinguishing each part. For example, the fish fin and the fish tail may have similarities in color and shape, and the colors and textures of the head, fin, tail, and trunk are similar, which are easily confused and lead to cognitive errors. From the perspective of human cognition of things, people often combine parts and the whole of the target to construct specific knowledge associations, which helps to form a more in-depth understanding of the target. Fish target segmentation needs to obtain more global features to correctly identify the target, while fish part segmentation needs to extract detailed local information based on the identification of the target to obtain accurate segmentation. Linking the global and local together can find the appearance differences between different parts of a single target and avoid ambiguity of similar object categories, and has generalization when facing different categories of fish. In order to solve the problems of underwater fish part segmentation, an underwater fish part segmentation method is proposed. As shown in Figure 4 The method is divided into four parts. First, five features f1-f5 of different scales are extracted from different layers of the backbone. Second, a local detail extraction module is designed to explore detailed information of the part by querying the shallow feature map to supplement more detailed information about the target. Third, a global-to-local hybrid residual structure (L-G Mixed residual module) is designed to recover feature information from coarse to fine in stages. The module processes global and local dual-branch information respectively, and then effectively fuses the two to optimize the target globally and locally in stages. Finally, the feature maps processed by each L-G module in the hybrid residual structure are converted into prediction maps that meet the number of categories. Multiple prediction maps of different levels are combined to generate the final underwater fish part segmentation result.

[0023] The backbone outputs five scale image features, which are not all passed to the local detail extraction module. Instead, the shallow features of the first three layers of the backbone are used. These shallow image features close to the input layer have the characteristics of high resolution and rich local details. They are more sensitive to features such as edges and textures, but also contain a lot of interference information and have a small receptive field. Through the semantic query module, the shallow image features f1, f2 and f3 are input to the local detail extraction module. Figure 5 In the query encoder, it is first processed into different query matrices, key matrices, and value matrices. The calculation process is as follows: (1) (2) (3) (4) (5) Among them, Q is the query matrix; K is the key matrix; V is the value matrix; is the image feature; is the weight of the query matrix; is the weight of the bond matrix; is the weight of the value matrix; is the dimension of the bond matrix; is a linear transformation operation; Score for attention; Score for long-head attention.

[0024] Fish target segmentation mainly separates the whole fish from the background, while part segmentation goes deep into the fish body and makes detailed divisions of its various components. In previous work, most of the part segmentation was performed on fish of the same category, while the constructed dataset contains a variety of fish with diverse shapes and postures. To perform part segmentation on different types of fish, it is not enough to simply identify the appearance features of the parts, and it is necessary to form the ability to understand the context of fish targets and parts. Therefore, in the task of semantic segmentation of fish parts, it is very important to pay attention to both global and local understanding. Taking into account the problems existing in part segmentation, a global to local hybrid residual structure (LG module) is proposed. In this structure, the deep features f4 and f5 in the backbone are combined with the shallow features of the part detail information enhanced by the local detail extraction module. 、 and The input is input into the global-to-local mixed residual structure. The input features are recovered from coarse to fine and from small scale to large scale, and residual connection is introduced to improve the performance and stability of the model. Through the processing of multiple L-G modules, global information and local information are processed by double branches, finally fused, and dual perception of fish targets and parts is promoted.

[0025] In one embodiment, the determination of the fourth number of fusion features according to the third number of image features and the second number of encoded features in step S30 is implemented by the following expression: (6) (7) (8) (9) (10) (11) (12) (13) (14) (15 (16) (17) wherein, , , and are local features; , , and are global features; , , and are fusion features; and are image features; , and are encoded features; Conv is a 3x3 convolution; Cat represents channel splicing; Upsample represents up-sampling; FFC is a 3x3 fast Fourier convolution; CA is channel attention; FCA is a local guiding operation on features.

[0026] Local features can provide rich details such as texture, edge, etc., while global features are responsible for providing overall structure, integrating semantic context information, eliminating local ambiguity, promoting global understanding, and enhancing model robustness. In fish part segmentation, relying only on local features may make mistakes due to the lack of overall understanding of the fish target. Traditional convolution can only see the information within its receptive field. Although the receptive field can be expanded by stacking layers, this way is inefficient in capturing global relationships. In order to effectively extract global features, frequency domain learning is introduced. Based on the spectral convolution theorem, the point-wise update of the spectral domain signal affects all inputs in the spatial domain, and the interference information in the frequency domain is usually easier to separate from the effective information, reducing the loss of effective information, which enables it to efficiently capture parts of the global effective representation. At the same time, the convolution operation in space is equivalent to multiplication in the frequency domain, which is more convenient to operate. The global branch is mainly composed of Fast Fourier Convolution (FFConv) used to capture global information, which converts spatial data into spectral domain through two-dimensional discrete Fourier transform, and does convolution in the frequency domain to extract global information beneficial to the task. The initial input feature processing method is consistent with the local branch, and the small-scale feature map is up-sampled and channel-spliced with the large-scale feature map, where three frequency domain convolutions are placed, so that the features are extracted in the frequency domain multiple times. The last frequency domain convolution is channel scaling, but it is still processed using FFConv with a convolution kernel of 3, effectively extracting global representation, strengthening global context understanding, and improving part segmentation accuracy. The calculation of the global branch is as follows: In one embodiment, the class average pixel accuracy and the average intersection over union in step S70 are respectively realized by the following expressions: (18) (19) wherein, the class average pixel accuracy; the average intersection over union; k represents the first number, k is taken as 5; TP is the number of correct target pixel points; TN is the number of correct background pixel points; FP is the number of false target pixel points; FN is the number of false background pixel points.

[0027] The application also provides an underwater fish part segmentation system, as shown in Figure 3 The system comprises: An acquisition module 10 is configured to acquire an underwater fish image, and extract a first number of image features from the underwater fish image through a backbone model; A query module 20 is configured to encode and query a second number of the image features to obtain a second number of encoded features; A determination module 30 is configured to determine a fourth number of fusion features according to the third number of image features and the second number of encoding features; the sum of the second number and the third number is the first number; A generation module 40 is configured to correspondingly generate a fourth number of prediction maps according to the fourth number of fusion features; A fusion module 50 is configured to fuse the fourth number of prediction maps to generate a fish part segmentation map.

[0028] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to cause the processor to perform the following steps: S10: obtaining an underwater fish picture, and extracting a first number of image features from the underwater fish picture through a backbone model; S20: encoding and querying a second number of the image features to obtain a second number of encoding features; S30: determining a fourth number of fusion features according to a third number of the image features and the second number of encoding features; the sum of the second number and the third number is the first number; S40: correspondingly generating a fourth number of prediction maps according to the fourth number of fusion features; S50: fusing the fourth number of prediction maps to generate a fish part segmentation map.

[0029] In one embodiment, a computer readable storage medium is provided, storing a computer program, the computer program being executed by a processor to cause the processor to perform the following steps: S10: obtaining an underwater fish picture, and extracting a first number of image features from the underwater fish picture through a backbone model; S20: encoding and querying a second number of the image features to obtain a second number of encoding features; S30: determining a fourth number of fusion features according to a third number of the image features and the second number of encoding features; the sum of the second number and the third number is the first number; S40: correspondingly generating a fourth number of prediction maps according to the fourth number of fusion features; S50: fusing the fourth number of prediction maps to generate a fish part segmentation map.

[0030] 1. Loss function The method uses two loss functions commonly used in semantic segmentation tasks, Cross-Entropy Loss and Dice Loss.

[0031] Cross-Entropy Loss checks each pixel one by one and compares the predicted result (probability distribution vector) of each pixel category with the label vector to measure the gap between the final prediction result and the true value of the label. The calculation formula for the binary classification is as follows: (20) in, Represents the true value of the label of pixel i, represents the prediction result of pixel i. In this study, the number of fish part segmentation categories is set to be greater than 2, which is a multi-classification task. The final calculated Cross-Entropy Loss formula is shown below.

[0032] (twenty one) M represents the number of categories, represents the true label value of pixel i of category c, Indicates the prediction result that pixel i belongs to category c.

[0033] Dice Loss is a loss function based on the Dice coefficient. It is a measure of sample similarity, especially for dealing with foreground-background imbalance. It reduces the influence of background pixels by calculating the intersection ratio between the prediction and the true label.

[0034] In the pixel analysis of the dataset, the proportion of the torso class far exceeds that of other classes. DiceLoss is introduced here to enhance the segmentation ability of small structures. The calculation formula is shown below.

[0035] (twenty two) Where TP stands for True Positive, which is the number of pixels correctly predicted as belonging to the target class. FP stands for False Positive, which is the number of pixels incorrectly predicted as belonging to the non-target class. FN stands for False Negative, which is the number of pixels incorrectly predicted as belonging to the target class. The final loss is the sum of Cross-Entropy Loss and Dice Loss, calculated as follows.

[0036] (twenty three) Cross-Entropy Loss and Dice Loss jointly supervise the training and help iterative optimization of features.

[0037] 2. Experiment and Results In this section, in order to evaluate the effectiveness of the proposed model in the underwater fish part segmentation task, the following experimental verification is designed. First, an ablation experiment is conducted to verify the effectiveness of the key parts in the model, i.e. the local detail extraction module and the global-to-local mixed residual structure. Then, compared with the current advanced semantic segmentation method on the constructed underwater fish part segmentation dataset, the superiority of the model in semantic segmentation of underwater fish parts is verified. Finally, by conducting experiments on different configurations of the global branch, local branch and fusion module in the L-G module, the effectiveness of the interaction of the extracted global information flow and local information flow for segmentation is verified.

[0038] 2.1 Experimental environment and training details The experiment uses GPU to accelerate training, and the experimental environment configuration of this study is as follows: Python 3.8 interpreter, open source deep learning framework Pytorch version 1.8.0, cuda version 11.1, integrated development based on Pycharm platform, data video memory 11G, memory 32GB, operating system Windows 10, using CPU model Intel(R) Core(TM) i7-9700 (3.00GHz) processor, GPU model NVIDIA Geforce RTX 3090 with 24GB of video memory.

[0039] In this study, resnet50 is used as the basic backbone network as a feature extractor. In order to unify the model processing, the resolution size of the input original picture is set to 512x512. The training round is set to 150 epochs, the batch_size is set to 2, Adam is used as the model optimizer, the momentum parameter is set to 0.9, the maximum learning rate of the model is set to 0.0001, the minimum learning rate is set to 0.00001, and the learning rate in the training stage uses the cos decline method.

[0040] 2.2 Ablation experiment In order to verify the functionality and effectiveness of the proposed local detail extraction module and global-to-local mixed residual structure design in PASNet, an ablation experiment is designed on the constructed underwater fish part segmentation dataset. The original resnet50 network used in Backbone is used to extract the features of the input image. The features output by the last layer of resnet50 are restored to the original input resolution through the bilinear interpolation method, and the segmentation result is directly output through channel scaling. On the basis of Backbone, the designed local detail extraction module and global-to-local mixed residual structure containing global and local information interaction are added separately for ablation experiment. Finally, the proposed model PASNet is obtained by integrating, and the experimental results are shown in Table 1.

[0041] Table 1 Model ablation experiment results The experimental results show that compared with the simple Backbone, the proposed local detail extraction module and the global-to-local hybrid residual structure have different degrees of performance improvement for the model, among which the design of the hybrid residual structure has a more obvious effect, and the mIoU is improved by 6.05% and 3.79% compared with Backbone and Backbone plus local detail extraction module respectively, which shows that the appropriate interaction of the positioning ability of local features and the context connection of global features is beneficial to underwater fish part segmentation, and the addition of residual connection improves the model generalization and enhances the overall segmentation ability. The improvement brought by the simultaneous addition of local detail extraction and hybrid residual structure is not as large as the addition of local detail extraction module alone, because the local detail extraction module processes the shallow feature map generated by Backbone which contains more fine-grained information, and extracts the detailed information related to fish parts, which has some repetition with the role of the local branch in the hybrid residual structure. But the Quries interaction in the local detail extraction module is more targeted than the convolution in the local branch for extracting local detail features, and the two complement each other, and the performance improvement brought by adding local detail extraction on top of the hybrid residual structure also illustrates this point. As mentioned above, from the perspective of human cognition of things, by perceiving the subtle details and complex shapes of the target, the target is divided into different parts, so as to understand its overall structure. Organically combining global and local to form knowledge connection, no longer relying on single feature for discrimination, which brings good effect on underwater fish part segmentation task. By adding the designed local detail extraction module and global-local hybrid residual structure to the original Backbone, effective extraction and fusion of global information and local detail information are completed, which improves the overall performance of the model and provides a more accurate and reliable tool for related research.

[0042] 2.3 Comparative test To verify the effectiveness of the designed model for underwater fish part segmentation, further comparisons were made with different segmentation models Unet, Deeplabv3+, Hrnet, P2T, Mask2former and FDLNet on the underwater fish part dataset. The specific results are shown in Table 2. Unet connects the features extracted by the encoder with the corresponding layers in the decoder, using different levels of information to improve segmentation accuracy. However, due to the limitations of the receptive field, Unet's ability to capture features is limited, and it cannot balance target positioning and context information acquisition. The results of processing the scale and morphology changes of different fish parts are not ideal. Deeplabv3+ retains the encoder-decoder structure that gradually recovers spatial information, introduces pyramid pooling with empty convolution, and expands the receptive field to obtain context information, improving the model's ability to handle multi-scale targets. Although it optimizes global semantic information, it still lacks detailed feature extraction and is not sensitive to fish parts. Hrnet introduces low-resolution convolution streams from high-resolution convolution streams to facilitate dense prediction tasks such as human pose estimation, semantic segmentation, and target detection. It connects multiple convolution streams with different resolutions in parallel, maintains high-resolution information flow during transmission, and repeatedly exchanges information to enrich the semantics of different resolution information. However, the excessive number of connected subnets leads to data redundancy and local information loss, which cannot provide accurate detailed information in the part segmentation task and lacks local feedback of global semantic information, failing to capture fish part features well. P2T (Pyramid Pooling Transformer) is a backbone network based on Transformer evolution, which introduces pyramid pooling into the multi-head self-attention module to enhance context extraction capabilities and extract better semantic features. The experimental results also show that P2T has understanding ability in part segmentation tasks, with improved part segmentation indicators compared to CNN networks. Mask2former, as a general image segmentation model, performs attention operations within the predicted mask region, localizes attention, and retains the Object Query query mechanism and Loss settings of Maskformer, achieving experimental results closest to the model proposed in this study, demonstrating good performance. However, in its multiple decoder structure, there are inconsistencies between adjacent layers in the prediction results, leading to differences in target understanding and affecting segmentation effectiveness. FDLNet improves performance through frequency domain learning, which is consistent with the design idea of the global branch in this study. FDLNet learns global features in the frequency domain, captures global interactions, and builds global structure representation paths. It proposes a decomposition stereo attention to realize multi-level feature fusion. Global features are the basis for context understanding in fish part segmentation, but lack the complementary role of local features, resulting in less significant effects. Experimental results also demonstrate that single global features cannot achieve good segmentation results. To verify this point, further experiments were conducted to compare the global and local branches.Compared with other advanced models, PASNet achieves the highest mIoU for all part classes except the background class. The local detail extraction module supplements the detailed features of the target part, reduces interference, and enhances the positioning ability of the part. At the same time, the designed global-to-local hybrid residual structure fuses global and local features, and introduces a residual structure in the transmission process to improve the model's cognitive ability and generalization for fish structures. Compared with the advanced segmentation model Mask2former, the mIoU of PASNet is improved by 0.74%, with the most significant improvement in the tail and head, which are 1.94% and 1.40% respectively, indicating the effectiveness of the proposed model in underwater fish part segmentation.

[0043] Table 2 Comparison test results with advanced segmentation models By comparing the visualization results of different models, it can be seen that the effect of PASNet is better. The visualization results of different models are shown in Figure 6 Compared with other models, the parts segmented by PASNet are more accurate, and there is no part confusion such as the mixed fin in the torso part in Mask2former. The boundaries between parts are clear, and the segmentation effect of the fish body is also relatively accurate. Due to texture, color and other problems, the segmentation of fins is more difficult than other parts, and the fin has the lowest segmentation index among all parts. However, PASNet performs well in fin segmentation. From the visualization results, it can be seen that compared with other models, the identification of fins is more accurate, and the segmentation is smooth and neat. The closest to the true value, even if there are adjacent situations between fins, they can also be well segmented.

[0044] 3.4 Feature branch comparison experiment To more significantly verify the effect of the global-to-local hybrid residual structure on fish part segmentation, the global branch, local branch and fusion module in the hybrid residual structure are tested separately. The hybrid residual structure can effectively preserve global semantic information and local detail information and fuse them in the channel and pixel dimensions, maximizing the model's understanding of the target and part, and improving the perception of fish target structures. The feature branch comparison experiment adopts three different strategies, namely adding only the global branch, adding only the local branch, and the complete hybrid residual structure. Through three different strategies, the influence of different features in underwater fish segmentation tasks is verified.

[0045] Table 3 Comparison of different feature branches The experimental results are shown in Table 3. In the single branch comparison, the global branch with frequency domain learning performs better. In the global branch, the feature is operated in the frequency domain using fast Fourier convolution, which can pay more attention to global semantic features. This is important in fish part segmentation because of the limitations of convolution itself. Even if the receptive field is increased through stacking, it cannot well improve the ability to use global information. And processing image features in the frequency domain can supplement the effective information that is not focused on by the spatial domain. These two points make the effect of the global branch more significant. The effect brought by the local branch is not as good as the global branch, but it also effectively focuses on the details of the fish target part. Detail features are beneficial to positioning the part and have a preliminary understanding of the part. Finally, the global and local branches are fused to obtain global information and local detail information, and to combine low-level detail features and high-level semantic features, to improve the model's perception of fish parts, to understand the whole and the part together, and not to rely on a single information, to ensure the accuracy of the segmentation results. Therefore, the fusion of global and local branches can better form the knowledge cognition of fish structure and enhance the segmentation performance and generalization of the model for different types of fish.

[0046] 4. Conclusion To further analyze fish morphological phenotypes, in addition to the overall fish target, obtaining information about its parts can enhance target analysis. Due to the large variety of fish species and their morphological specificity, fish parts differ significantly from one another, while parts within the fish exhibit similarities. This leads to part ambiguity, inaccurate part identification, and poor segmentation. To address this issue, a global-local perception-based underwater fish part segmentation network (PASNet) was proposed. This model aims to combine global semantics with local detail features to generate a holistic understanding of fish, form knowledge connections, and improve part segmentation accuracy. To extract detailed features of underwater fish parts, a local detail extraction module was designed. This module decomposes the input shallow features into queries, which interact within the query encoder to implement detailed feature queries and capture local features of fish parts. To enhance fish part perception, a global-local hybrid residual structure was proposed. This structure is guided by a global branch in the frequency domain and a local branch in the convolutional domain. Finally, the structures are fused at the channel and pixel levels to achieve a fusion of global semantics and local detail, enabling the model to perceive fish parts and improving segmentation accuracy. Experimental results using PASNet on the proposed fish parts dataset demonstrate that the proposed method performs well in underwater fish part segmentation tasks, significantly improving the accuracy of fish part segmentation. This provides an intelligent and effective approach for acquiring fish morphological phenotypes and analyzing fish structure, providing important indicators for assessing fish production and health, helping to improve aquaculture efficiency and assist in aquaculture decision-making. It is worth noting that this research still has room for improvement. Limited by the dataset, the number of fish categories and individual data currently collected is limited. In real-world environments, fish with more difficult-to-identify parts and more complex situations exist. Further exploration and research is needed to design a model that can uniformly segment fish parts.

[0047] Figure 7 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device can be a terminal or a server. Figure 7 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the underwater fish part segmentation method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the underwater fish part segmentation method. Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0048] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiments can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0049] The technical features of the above embodiments can be combined in any way. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0050] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for segmenting underwater fish parts, characterized in that: The method comprises: Obtain an underwater fish picture, and extract a first number of image features from the underwater fish picture using a backbone model; Performing encoding query on the second number of the image features to obtain a second number of encoding features; Determining a fourth number of fusion features based on the third number of image features and the second number of encoding features, wherein the sum of the second number and the third number is the first number; The fourth number of fused features respectively generate a fourth number of prediction maps; The fourth number of prediction images are fused to generate a fish part segmentation image.

2. The underwater fish part segmentation method according to claim 1, characterized in that: Also includes: Determine the correct target pixel number, the correct background pixel number, the incorrect target pixel number, and the incorrect background pixel number in the fish part segmentation map; Determine the average pixel accuracy rate and the average intersection-over-union ratio of the category according to the number of target pixels, the number of correct background pixels, the number of erroneous target pixels, and the number of erroneous background pixels; The accuracy of the fish part segmentation map is determined by the category average pixel accuracy and the average intersection-over-union ratio.

3. The underwater fish part segmentation method according to claim 2, characterized in that: The second number of coded features obtained by performing coding query on the second number of image features includes: Obtaining a query matrix, a key matrix, and a value matrix for each of the second number of image features; Performing an attention operation on the query matrix, the key matrix, and the value matrix to obtain a plurality of attention scores for each image feature; Concatenate multiple attention scores to obtain a multi-head attention score for each image feature; The multi-head attention score is input into the feedforward fully connected network layer after residual connection and layer normalization processing, and then residual connection and layer normalization processing are performed to obtain the encoding feature.

4. The underwater fish part segmentation method according to claim 3, characterized in that: The multi-head attention score is obtained by the following expression: Among them, Q is the query matrix; K is the key matrix; V is the value matrix; is the image feature; is the weight of the query matrix; is the weight of the bond matrix; is the weight of the value matrix; is the dimension of the bond matrix; is a linear transformation operation; Score for attention; Score for long-head attention.

5. The underwater fish part segmentation method according to claim 1, characterized in that: The determination of the fourth number of fusion features according to the third number of image features and the second number of coding features is achieved by the following expression: in, 、 、 and All are local features; 、 、 and All are global features; 、 、 and All are fusion features; and is the image feature; 、 and All are encoding features; Conv is 3×3 convolution; Cat means channel splicing; Upsample means upsampling; FFC is 3×3 fast Fourier convolution; CA is channel attention; FCA is local guidance operation for features.

6. The underwater fish part segmentation method according to claim 2, characterized in that: The category average pixel accuracy and the average intersection-over-union ratio are respectively realized by the following expressions: in, Category average pixel accuracy; is the average intersection-over-union ratio; k represents the first number, and k is taken as 5; TP is the number of correct target pixels; TN is the number of correct background pixels; FP is the number of incorrect target pixels; and FN is the number of incorrect background pixels.

7. An underwater fish part segmentation system, characterized in that: The system comprises: an acquisition module, configured to acquire an underwater fish picture and extract a first number of image features from the underwater fish picture using a backbone model; A query module, configured to perform a coding query on a second number of the image features to obtain a second number of corresponding coding features; a determination module, configured to determine a fourth number of fusion features based on a third number of the image features and the second number of the coding features, wherein the sum of the second number and the third number is the first number; A generation module, configured to generate a fourth number of prediction maps corresponding to the fourth number of fusion features; A fusion module is used to fuse the fourth number of prediction images to generate a fish part segmentation image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the following method: Obtain an underwater fish picture, and extract a first number of image features from the underwater fish picture using a backbone model; Performing encoding query on the second number of the image features to obtain a second number of encoding features; Determining a fourth number of fusion features based on the third number of image features and the second number of encoding features, wherein the sum of the second number and the third number is the first number; The fourth number of fused features respectively generate a fourth number of prediction maps; The fourth number of prediction images are fused to generate a fish part segmentation image.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor performs the steps of the following method: Obtain an underwater fish picture, and extract a first number of image features from the underwater fish picture using a backbone model; Performing encoding query on the second number of the image features to obtain a second number of encoding features; Determining a fourth number of fusion features based on the third number of image features and the second number of encoding features, wherein the sum of the second number and the third number is the first number; The fourth number of fused features respectively generate a fourth number of prediction maps; The fourth number of prediction images are fused to generate a fish part segmentation image.

Images