Fish body length estimation method and system based on global guidance semantic segmentation network

Abstract

The invention provides a fish body length estimation method and system based on a global guidance semantic segmentation network, relates to the technical field of computer vision and artificial intelligence, and aims to solve the problems that in a scene that a fish body in a monocular image and complex background interference coexist, and in a multi-fish-species mixed scene, the model segmentation generalization ability in the prior art is insufficient, and the fish body length estimation accuracy is poor. And high-precision and robust mask segmentation is difficult to realize. The method comprises the following steps: acquiring a fish body monocular image, and performing distortion correction on the fish body monocular image; inputting the corrected fish body monocular image into a trained global guide semantic segmentation network for semantic segmentation to obtain a semantic segmentation mask graph of the fish body and the calibration plate, and constructing a mapping relationship between the physical size and the pixel size of the calibration plate; and performing ellipse fitting on the semantic segmentation mask graph of the fish body, calculating the pixel length of the fish body, and converting the pixel length of the fish body into the actual length of the fish body. The method solves the problems in the prior art, provides the generalization ability of the network model, and achieves the precise estimation of the length of the fish body.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence technology, and in particular relates to a fish body length estimation method and system based on a global guided semantic segmentation network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Fish length is a key indicator for measuring fish growth, health, and profitability, and has broad applications in aquaculture and fishery management. Quality grading based on fish length effectively selects fish that meet market demand, thereby improving product competitiveness. Regularly monitoring changes in fish length allows for accurate assessment of fish growth, enabling timely adjustments to breeding strategies, optimized feed ratios, and improved feeding efficiency. Fish length can also be used to determine the optimal time to sell, helping to improve resource utilization and promote sustainable fishery development. Therefore, accurate fish length measurement is crucial for refined fishery management.

[0004] However, in the aquaculture and processing industries, traditional fish length measurement relies primarily on manual labor. This is not only time-consuming and labor-intensive, but also prone to human error, making it difficult to guarantee measurement accuracy and reducing the reliability of length measurements. In contrast, deep learning-based fish length estimation methods, with their high efficiency and high accuracy, provide intelligent and precise solutions for the fishery industry.

[0005] The inventors discovered that the background environments of fish images captured by monocular cameras are often highly complex and diverse, such as aquaculture processing plants, breeding areas, and fishing scenes. These background elements often closely resemble the fish themselves in color, texture, and even shape, which increases the difficulty of image segmentation, affects the fine segmentation of the fish mask, and seriously affects the accuracy of subsequent length estimation. Furthermore, the current aquaculture industry involves a wide variety of fish species, and different fish species exhibit significant differences in morphology, color distribution, and scale reflectivity. This results in insufficient generalization of the model's segmentation capabilities when faced with diverse fish species, making it difficult to achieve high-precision and robust mask segmentation. Summary of the Invention

[0006] To overcome the shortcomings of the aforementioned prior art, the present invention provides a fish length estimation method and system based on a global guided semantic segmentation network. This method addresses the challenges of fish coexisting with complex background interference and mixed fish species in monocular images. By constructing a Transformer-based global guided semantic segmentation network, the system can segment fish masks more precisely in complex background environments. Furthermore, the system achieves accurate fish length estimation using an ellipse fitting method, which has important applications in promoting intelligent aquaculture and processing.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: A first aspect of the present invention provides a fish body length estimation method based on a global guided semantic segmentation network, comprising: Acquire a monocular image of a fish body, wherein the monocular image of the fish body includes the fish body and a calibration plate; Performing distortion correction on the monocular image of the fish body to obtain a corrected monocular image of the fish body; The corrected fish monocular image is input into the global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask of the fish body and the calibration plate; Based on the semantic segmentation mask of the calibration plate, a mapping relationship between the physical size of the calibration plate and the pixel size is constructed; Perform ellipse fitting on the fish body semantic segmentation mask image and calculate the pixel length of the fish body; Based on the mapping relationship between the physical size of the calibration plate and the pixel size, the pixel length of the fish body is converted into the actual length of the fish body.

[0008] As an implementation method, the corrected monocular fish image is input into a global guided semantic segmentation network for semantic segmentation. The global guided semantic segmentation network includes a deep convolutional neural network module, a residual Transformer module, a dilated spatial pyramid pooling module, and multiple Transformer global guided modules. The specific process is as follows: Input the corrected fish monocular image into the deep convolutional neural network module to extract features at different scales to obtain the first feature map, the second feature map, the third feature map, the fourth feature map, and the fifth feature map; The fourth feature map is input into the residual Transformer module for feature extraction and fusion to obtain the fused feature map; The fused feature map is input into the dilated spatial pyramid pooling module for feature extraction to obtain multi-scale features; Based on multi-scale features, the first feature map, second feature map, third feature map and fourth feature map are respectively input into the corresponding Transformer global guidance module for feature guidance and recalibration to obtain the semantic segmentation mask map of the fish body and calibration plate.

[0009] As an implementation method, the fourth feature map is input into the residual transformer module for feature extraction, wherein the residual transformer module includes a feature map blocking module, a linear mapping module, a position encoding module, a transformer encoding module and a reshaping module. The specific process is as follows: Converting the fourth feature map into a vectorized feature image block through a feature map block module; Perform linear mapping on the vectorized feature image blocks through a linear mapping module to obtain mapped feature image blocks; The mapped feature image blocks are spatially encoded through the position encoding module to obtain serialized features; The serialized features are input into the Transformer encoding module for feature extraction and enhancement to obtain the enhanced feature map; The enhanced feature map is reshaped by the reshaping module to obtain a reshaped feature map; The reshaped feature map is added element-by-element to the fifth feature map to obtain the fused feature map.

[0010] As an implementation method, the fused feature map is input into the dilated spatial pyramid pooling module for feature extraction to obtain multi-scale features. The dilated spatial pyramid pooling module includes five branch modules in parallel. The specific process is as follows: The fused feature map is input into the first branch module, and features are extracted through the convolution layer to obtain the first branch feature map; The fused feature maps are input into the second, third and fourth branch modules respectively, and feature extraction is performed using corresponding dilated convolutional layers with different dilation rates to obtain the second branch feature map, the third branch feature map and the fourth branch feature map; The fused feature map is input into the fifth branch module, the feature map is compressed through the global average pooling layer, and the spatial resolution of the compressed feature image is restored through the convolution layer and upsampling module to obtain the fifth branch feature map; Splicing the first branch feature map, the second branch feature map, the third branch feature map, the fourth branch feature map, and the fifth branch feature map to obtain a spliced ​​feature map; The convolution operation is performed on the spliced ​​feature map through the convolution layer to obtain multi-scale features.

[0011] As an implementation method, based on multi-scale features, the first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the corresponding Transformer global guidance module for feature guidance and recalibration. The specific process is as follows: Performing an upsampling operation on the multi-scale features to obtain a first reconstructed feature map; Inputting the first reconstructed feature map and the fourth feature map into the corresponding fourth Transformer global guidance module for fusion and correction to obtain a first fused and corrected feature map; Perform feature splicing on the first fused corrected feature map and the first reconstructed feature map to obtain a first spliced ​​feature map; Performing an upsampling operation on the first concatenated feature map to obtain a second reconstructed feature map; Input the second reconstructed feature map and the third feature map into the corresponding third Transformer global guidance module for fusion and correction to obtain a second fused and corrected feature map; Perform feature splicing on the second fused corrected feature map and the second reconstructed feature map to obtain a second spliced ​​feature map; Performing an upsampling operation on the second concatenated feature map to obtain a third reconstructed feature map; Input the third reconstructed feature map and the second feature map into the corresponding second Transformer global guidance module for fusion and correction to obtain a third fused and corrected feature map; Perform feature splicing on the third fused corrected feature map and the third reconstructed feature map to obtain a third spliced ​​feature map; Performing an upsampling operation on the third concatenated feature map to obtain a fourth reconstructed feature map; Inputting the fourth reconstructed feature map and the first feature map into the corresponding first Transformer global guidance module for fusion and correction to obtain a fourth fused and corrected feature map; Perform feature splicing on the fourth fused corrected feature map and the fourth reconstructed feature map to obtain a fourth spliced ​​feature map; The fourth concatenated feature map is convolved through a convolution layer to obtain a semantic segmentation mask map of the fish body and the calibration plate.

[0012] As an implementation method, the first reconstructed feature map and the fourth feature map are input into the corresponding fourth Transformer global guidance module for fusion and correction. The specific process is as follows: Perform convolution and maximum pooling operations on the fourth feature map to obtain a pooled fourth feature map; The first reconstructed feature map is added to the fourth feature map after pooling element by element, and an activation function is used to perform feature enhancement to obtain the first enhanced fusion feature; The first enhanced fusion feature is subjected to global maximum pooling and global average pooling respectively, and the two pooled features are spliced ​​along the channel to obtain the first enhanced global feature map; Perform convolution and upsampling operations on the first enhanced global feature map to obtain a first restored feature map; An activation function is used to generate a first weight map of the first restored feature map, and the first weight map is spatially multiplied element-by-element with the fourth feature map to obtain a first weighted correction feature map; The first weighted corrected feature map is added to the fourth feature map element by element to obtain a first fused corrected feature map.

[0013] As an implementation method, the pixel length of the fish body is converted to the actual length of the fish body using the formula: ; in, Indicates the pixel length of the fish body, specifically the long side of the circumscribed rectangle of the fish body outline; represents the pixel perimeter of the minimum circumscribed rectangle of the calibration plate, Indicates the actual physical circumference of the calibration plate.

[0014] A second aspect of the present invention provides a fish length estimation system based on a global guided semantic segmentation network, comprising: An image acquisition module is used to acquire a monocular image of the fish body, wherein the monocular image of the fish body includes the fish body and the calibration plate; and perform distortion correction on the monocular image of the fish body to obtain a corrected monocular image of the fish body; The mask image generation module is used to input the corrected fish body monocular image into the global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask image of the fish body and the calibration plate; The length estimation module is used to construct a mapping relationship between the physical size and pixel size of the calibration plate based on the semantic segmentation mask of the calibration plate; perform ellipse fitting on the semantic segmentation mask of the fish body to calculate the pixel length of the fish body; and convert the pixel length of the fish body into the actual length of the fish body based on the mapping relationship between the physical size and pixel size of the calibration plate.

[0015] The third aspect provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0016] A fourth aspect provides a computer-readable storage medium having a computer program stored thereon, which performs the steps of the above method when executed by a processor.

[0017] One or more of the above technical solutions have the following beneficial effects: In this embodiment, in order to solve the problem of coexistence of fish bodies and complex background interference and mixed scenes of multiple fish species in monocular images, a Transformer-based global guided semantic segmentation network is constructed to segment more refined fish body masks in complex background environments, and accurate fish body length estimation is achieved based on the ellipse fitting method. Specifically, in the Transformer global guided semantic segmentation network, a deep convolutional neural network module is used to extract multi-scale feature maps of the fish body. The residual Transformer module extracts global context information while modeling the global interaction between local features through the attention mechanism, thereby retaining local features and ensuring the efficiency and accuracy of feature expression. The multi-scale context information is captured by a variety of different dilated convolution operations of the dilated spatial pyramid pooling module, which can simultaneously capture small-scale local details and large-scale global features, which is of great significance for the fine segmentation of fish body boundaries. Multiple Transformer global guided modules and upsampling operations are used to effectively integrate cross-regional information, thereby improving the segmentation accuracy of fish body edges and detail areas, thereby improving the network's adaptability to the morphology of different fish species and enhancing the network's generalization ability. Finally, by performing ellipse fitting on the segmented fish body contour, the proportional relationship between the fish body and the calibration plate pixels is established, and combined with the physical size of the calibration plate, accurate estimation of the fish body length is achieved.

[0018] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 Schematic diagram of the framework of the fish body length estimation method based on the global guided semantic segmentation network of the first embodiment; Figure 2 This is a schematic diagram of the residual Transformer module framework of the first embodiment; Figure 3 This is a schematic diagram of the ASPP module framework of the first embodiment; Figure 4 This is a schematic diagram of the Transformer global guidance module of the first embodiment; Figure 5 Schematic diagram of monocular image acquisition in the first embodiment; Among them, (a) is the monocular image data acquisition environment, and (b) is a schematic diagram of the monocular camera layout; Figure 6This is an example of image acquisition under different lighting and background conditions of the first embodiment; Figure 7 This is a diagram showing the segmentation results of the fish body and the calibration plate in the first embodiment; Among them, (a) is the real image of the fish body and the calibration plate, and (b) is the semantic segmentation mask image of the fish body and the calibration plate; Figure 8 Schematic diagram of the image before and after the fish body mask rotation transformation of the first embodiment; Among them, (a) is the original fish body mask image, (b) is the fish body mask image after rotation transformation; Figure 9 This is the visualization result of the semantic segmentation of the fish body in this embodiment; Figure 10 The relative error distribution of fish body length estimation for different fish species in the first embodiment; Figure 11 The relative error distribution of fish length estimation in different environments of the first embodiment; Figure 12 Statistical distribution of fish length estimation results using different methods in this first embodiment. DETAILED DESCRIPTION

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0022] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0023] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0024] Example 1 This embodiment discloses a fish body length estimation method based on a global guided semantic segmentation network.

[0025] To more clearly illustrate this embodiment, the fish body length estimation method based on the global guided semantic segmentation network includes: S1. Acquire a monocular image of a fish body, wherein the monocular image of the fish body includes the fish body and a calibration plate; S2. performing distortion correction on the monocular image of the fish body to obtain a corrected monocular image of the fish body; The corrected fish monocular image is input into the global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask of the fish body and the calibration plate; S3. Based on the semantic segmentation mask of the calibration plate, a mapping relationship between the physical size and pixel size of the calibration plate is constructed; Perform ellipse fitting on the fish body semantic segmentation mask image and calculate the pixel length of the fish body; Based on the mapping relationship between the physical size of the calibration plate and the pixel size, the pixel length of the fish body is converted into the actual length of the fish body.

[0026] like Figure 1 As shown, in step S1, a monocular image of the fish body is obtained, wherein the monocular image of the fish body includes the fish body and a calibration plate; and the monocular image of the fish body is subjected to distortion correction to obtain a corrected monocular image of the fish body.

[0027] In this embodiment, (1) monocular camera calibration and correction.

[0028] A Logitech C920E full HD 1080P camera is used as a monocular image acquisition device. The camera has a resolution of 1920×1080 and supports 30 frames per second video shooting, meeting the needs of high-precision fish image acquisition.

[0029] First, the monocular camera needs to be distorted to ensure that the dimensional information in the image accurately maps to the real world. The camera imaging process produces geometric distortion due to the optical properties of the lens, primarily radial and tangential distortion. Radial distortion refers to the radial offset of image pixels relative to the center of distortion, while tangential distortion is the positional deviation caused by the lens not being completely parallel to the image plane. Therefore, accurate distortion correction is a crucial step in ensuring the geometric authenticity of the image and the accuracy of the estimation.

[0030] In the study of fish length estimation, a mapping relationship between actual physical dimensions and image pixel dimensions is established based on a calibration plate, enabling the conversion of fish length from pixels to physical dimensions, thereby accurately estimating fish length. To ensure that the captured images have accurate scale information, this paper requires camera calibration to eliminate geometric distortion caused by the lens. The specific process is as follows: 1) Secure the monocular camera and capture 22 images using a 12x9 checkerboard calibration pattern with a square grid length of 30 mm. Ensure that the calibration pattern is intact and clearly visible in each image.

[0031] 2) Use the Camera Calibrator toolbox in Matlab 2020a to calibrate the camera. After loading all captured images, execute the calibration process to obtain the camera's intrinsic parameters and distortion coefficients.

[0032] The distortion coefficient is obtained according to the calibration results, and the image is corrected for distortion to ensure the accuracy of size mapping. The distortion correction formula is: (1) in, is the coordinate of the pixel in the image before distortion correction, are the new coordinates after correction; k1, k2, k3 are the distortion coefficients of radial distortion, which are used to describe the degree of offset of image pixels due to radial distortion; r is the distance between the pixel point in the image and the principal point; p1 and p2 are the distortion coefficients of tangential distortion.

[0033] (2) Data collection scene setting.

[0034] The experimental fish included crucian carp (Carassius auratus), tilapia (Oreochromis niloticus) and sea bass (Lateolabrax japonicus), with four of each species. All experimental fish were purchased from supermarkets in Jinan, Shandong Province, and randomly selected by supermarket employees to ensure their health. The monocular image data was collected indoors at the Qianfoshan Campus of Shandong University in Jinan, Shandong Province, in November 2024. The data collection environment is as follows: Figure 5 (a) As shown. During the experiment, the fish that were not photographed were placed in a glass water tank, and the fish that had been photographed were placed in a blue water bucket. Both were supplied with oxygen through an oxygen pump to ensure their health. The monocular camera was fixed on a bracket, about 620mm above the ground, and the camera's field of view was parallel to the ground, as shown in Figure 2. Figure 5 (b) The camera transmits captured image data to a computer via a USB port. During filming, the fish to be photographed is placed within the camera's field of view, with a white plastic sheet placed beneath it to enhance image contrast. A 9×7 checkerboard calibration board with a 20 mm square grid is placed next to the fish as a physical reference to facilitate subsequent fish length estimation.

[0035] Considering the impact of lighting changes and complex background on fish segmentation, we set various indoor lighting settings during image data acquisition and placed different pictures on white plastic boards to simulate complex background environments. A total of four image data were collected under different lighting and background environments: normal lighting, dim lighting, shadow environment, and complex background. The collected image data are as follows: Figure 6 shown.

[0036] (3) Construction of fish body length and semantic segmentation dataset.

[0037] 1) Length estimation dataset Before image acquisition, the full length of each fish was manually measured using a steel ruler with an accuracy of 1 mm. The true lengths of four crucian carp, four seabass, and four tilapia were manually measured. These measurements served as the ground truth labels for the fish lengths. The relevant data are shown in Table 1.

[0038] Table 1 Real data of body length of three fish

[0039] 2) Fish body semantic segmentation dataset After data collection, LabelMe software was used to annotate the complete fish outline and calibration plate outline in 450 randomly selected images of each fish species. During the annotation process, the fish outline did not include the dorsal fin, anal fin, and pelvic fin.

[0040] To expand the fish semantic segmentation dataset, this paper collected images of fish in complex backgrounds from the internet. A total of 150 images each of crucian carp, sea bass, and tilapia were collected. These images do not include calibration plates and can only be used for training and testing the fish semantic segmentation network; they cannot be used for fish length estimation. This additional data helps improve the generalization ability of the semantic segmentation model in complex backgrounds. The composition of the data used for semantic segmentation is shown in Table 2.

[0041] Table 2 Composition of fish body semantic segmentation data

[0042] like Figure 1 As shown, in step S2, the corrected fish body monocular image is input into the global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask map of the fish body and the calibration plate.

[0043] S2-1. Build a global guided semantic segmentation network.

[0044] In this embodiment, the global guided semantic segmentation network TGGNet is improved based on the U-Net architecture.

[0045] like Figure 1 As shown in the figure, the global guided semantic segmentation network TGGNet includes a deep convolutional neural network module, a residual Transformer module, a void space pyramid pooling module and multiple Transformer global guidance modules.

[0046] The residual Transformer module includes a feature map partitioning module, a linear mapping module, a position encoding module, a Transformer encoding module, and a reshaping module. The atrous spatial pyramid pooling module (ASPP) consists of five parallel branch modules.

[0047] The fish semantic segmentation network TGGNet was deployed on a server equipped with four NVIDIA GeForce RTX 3090 GPUs. The operating system was Ubuntu 20.04.4 LTS, the code was run in Python 3.8, the deep learning framework was PyTorch 1.11.0, and CUDA version 11.3. The dataset contained 1,800 images, including 1,080 for training and 720 for testing. All input images were resized to 512×512. During network training, the batch size was set to 8, and the maximum number of epochs was set to 150. The Adam optimizer was used, with an initial learning rate of 0.0001. Cosine annealing was used for learning rate decay, ultimately reducing the learning rate to 0.000001 to ensure model stability and improved accuracy in later training stages.

[0048] S2-2. Train the global guided semantic segmentation network.

[0049] The rectified monocular fish images in the training set are input into the global guided semantic segmentation network for semantic segmentation.

[0050] The global guided semantic segmentation network uses VGG16 as the feature extraction skeleton to generate 5 feature maps of different scales In the feature map The Transformer module is then introduced to model the global interaction between local features through the attention mechanism, so that the dependency between sequence features can be effectively captured. The feature map after the Transformer transformation is connected through residual connections and The fusion is performed to extract global context information while retaining local features, ensuring the efficiency and accuracy of feature expression. Subsequently, the ASPP module uses a variety of different hole convolution operations to capture multi-scale context information, further enhancing the network's ability to model multi-scale information. This design can simultaneously capture small-scale local details and large-scale global features, which is of great significance for fine segmentation of fish body boundaries. In addition, multi-scale context modeling improves the network's adaptability to the morphology of different fish species and enhances the network's generalization ability. Finally, TGGNet uses the global features extracted by Transformer to guide and recalibrate multi-scale features, and further improves the feature expression ability through the Transformer global guidance module TGGM, highlighting larger target areas such as the fish body and calibration plate, and improving the accuracy of the segmentation boundary.

[0051] The specific process is: S2-2-1. Input the corrected monocular image of the fish body into the deep convolutional neural network module to extract features at different scales to obtain a first feature map, a second feature map, a third feature map, a fourth feature map, and a fifth feature map.

[0052] In this embodiment, the deep convolutional neural network module uses VGG16 as the feature extraction skeleton to generate five feature maps of different scales: the first feature map, the second feature map, the third feature map, the fourth feature map, and the fifth feature map. .

[0053] S2-2-2. Input the fourth feature map into the residual Transformer module for feature extraction and fusion to obtain a fused feature map.

[0054] Given the large proportion of the fish body in the image, the residual Transformer module was introduced, leveraging its powerful ability to capture global information and modeling the global dependencies of the extracted local features. The residual Transformer module uses a self-attention mechanism to exploit the correlations and interactions between local features, capturing global information about the overall shape of the fish. The introduction of global information helps enhance the network's understanding of the overall structure of the fish, improves the segmentation process's ability to focus on the target area of ​​the fish, and reduces interference from complex background or other noise. This global morphological information is crucial for fine-grained segmentation of fish boundaries and improves the network's adaptability to morphological changes across different fish species.

[0055] like Figure 2 As shown in Figure 2, the specific process of feature extraction and fusion in the residual Transformer module is as follows: (1) The fourth feature map is converted into a vectorized feature image block through the feature map block module.

[0056] In this embodiment, before the fourth feature map is input into the Transformer encoding module, it is first reconstructed into a one-dimensional feature sequence to meet the Transformer's structural requirements for the input sequence.

[0057] Specifically, input the fourth feature map The size is , divide it into N non-overlapping 2D image patches of size P×P, where , thus, the fourth feature map of the input is converted into a flattened 2D sequence of length N, that is, a vectorized feature image block After converting the feature map into a Patch sequence, Transformer is able to model the relationship between different regions globally through the self-attention mechanism.

[0058] The above steps yield a feature map that preserves local spatial structure information, ensuring that each image block retains the spatial context within the local area. This structure enables the Transformer to capture long-range dependencies in the image while maintaining regional awareness. This provides a foundation for improving global modeling and feature representation capabilities, effectively integrating local and global features and enhancing the model's ability to identify and represent key fish structures.

[0059] (2) The vectorized feature image blocks are linearly mapped through the linear mapping module to obtain the mapped feature image blocks.

[0060] In this embodiment, the original image block is flattened to form a low-dimensional original vector, which is difficult to be directly used for high-level semantic modeling. Therefore, it is necessary to vectorize the feature image block. Mapped to an embedding space with a dimension of D, the linear mapping module uses a fully connected layer, which can be projected into a high-dimensional embedding space through the linear mapping operation (i.e., the fully connected layer).

[0061] The above steps enable feature dimension conversion and embedding, extracting richer and more abstract feature representations and providing more discriminative input features for the Transformer encoding module. This also enhances feature expression capabilities. Specifically, by introducing trainable parameters into the linear mapping process, it can optimize and learn the most appropriate feature transformation method based on the task, thereby improving the semantic expression of each image block and helping subsequent modules more accurately capture key areas and boundary features of the fish.

[0062] (3) The mapped feature image blocks are spatially encoded through the position encoding module to obtain serialized features.

[0063] In this example, the fish body structure has obvious spatial distribution characteristics, such as the positions of the head, torso and tail have a stable relative layout. However, the Transformer is essentially a sequence-unbiased structure and cannot explicitly distinguish the spatial position information between different image blocks. Therefore, it is necessary to introduce position encoding for the image blocks to help the network learn the spatial distribution of image semantic information during training. , as shown below: (2) in, represents the feature vector of each image block, ; Represents a linear mapping matrix ; represents the position code, ; Represents the features of the serialized image.

[0064] Through the above steps, spatial position information is injected to compensate for the shortcomings of the Transformer structure. That is, by adding a corresponding position encoding vector to each image block, the network can explicitly perceive the spatial position of each image block in the original image during processing, thereby enhancing the model's spatial modeling capabilities. It improves the spatial consistency of semantic understanding. Position encoding enables the model to learn the semantic distribution patterns in these spaces when extracting features, thereby more accurately identifying key structures and improving the accuracy and robustness of fish length estimation. It promotes global context relationship modeling, allowing the Transformer to subsequently model the relationship between any two image blocks in the sequence through the self-attention mechanism, not only considering their semantic features but also integrating their spatial position information, giving the model stronger global perception capabilities when processing the overall structure of the fish.

[0065] (4) The serialized features are input into the Transformer encoding module for feature extraction and enhancement to obtain the enhanced feature map.

[0066] like Figure 2 As shown, the Transformer module consists of It consists of a stacked multi-head self-attention mechanism (MSA) and a multi-layer perceptron (MLP).

[0067] The core of the residual Transformer module is the self-attention mechanism. This mechanism achieves efficient feature representation for sequential data by modeling global dependencies between input features. The Transformer's self-attention mechanism dynamically learns global correlations between positions in the sequence. Unlike traditional convolutional neural networks that rely solely on a local receptive field, this mechanism can simultaneously capture connections between distant parts of fish images (such as the head, tail, torso, and edges). The multi-head self-attention mechanism learns different attention representations in parallel across multiple subspaces, capturing diverse relationships between image patches and effectively enhancing feature diversity and robustness.

[0068] The concepts of query (Q), key (K), and value (V) are introduced in the self-attention calculation process. The attention weight of the feature is obtained by calculating the similarity score between the query and the key, and then the weighted sum of the values ​​is performed to generate the attention output. The formula is: (3) in, represents the correlation weight between each feature vector and the global feature, are the dimensions of Q and K, Represents the scaling factor used to reduce The magnitude of the gradient is used to prevent the gradient from disappearing and exploding during training.

[0069] The MLP module in the Transformer encoder performs further nonlinear transformations on the self-attention output, enabling the effective expression of complex structural relationships in high-dimensional feature space. This is achieved by stacking multiple encoder layers in the above structure.

[0070] Among them, the encoder The output of the layer can be expressed as: (4) (5) Among them, MSA represents the multi-head attention mechanism, MLP is the multi-layer perceptron, and LN represents the layer normalization operation (Layernormalization, LN). is the output of the previous layer. LN normalizes the input features in the channel dimension to keep their mean and variance stable, alleviate the internal covariate shift problem, improve the training stability of the network, and accelerate convergence.

[0071] Through the above steps, the global dependencies between fish features are modeled, enabling more accurate structural modeling and contour understanding, which helps improve the accuracy of fish length estimation. Multi-head attention enhances the diversity of feature expression. The multi-head mechanism can comprehensively judge from different angles, improving the ability to locate key areas (such as the head and tail). The MLP subnetwork enhances the nonlinear expression of features. This step enhances the discriminative ability of fish image sequences and provides richer high-level semantic features for the subsequent length estimation module. Support for module stacking for layer-by-layer feature enhancement can achieve layer-by-layer enhancement of the input sequence, gradually increasing the depth of feature semantic expression and further strengthening the model's adaptability to complex fish morphology.

[0072] (5) The enhanced feature map is reshaped by the reshaping module to obtain the reshaped feature map.

[0073] In the initial stage of network training, the convolution operation is mainly used for local feature extraction, while the Transformer is mainly used for global feature modeling. In the initial stage, the Transformer is relatively weak in capturing fine-grained local information. If the Transformer is directly used to replace the convolution operation, the network performance may fluctuate. Therefore, when training the feature map After the Transformer transformation, the tensor shape is adjusted by reshaping operation so that the feature map With the original feature map The sizes are consistent, that is, the reshaped feature map is obtained.

[0074] (6) Add the reshaped feature map and the fifth feature map element by element to obtain the fused feature map.

[0075] The reshaped feature map With the fifth characteristic diagram Add element by element to get the fused feature map .

[0076] After the above steps, the fusion process fully combines the advantages of Transformer in capturing global information and the convolution operation's ability to extract local features, thereby effectively improving the network's feature expression capabilities.

[0077] S2-2-3. Input the fused feature map into the dilated spatial pyramid pooling module for feature extraction to obtain multi-scale features.

[0078] The goal of fish semantic segmentation is to accurately separate the fish from the background. Fish contours are typically complex, and their boundaries contain rich details. Traditional convolutional models are limited by a fixed receptive field, making it difficult to effectively model long-range dependencies and also limiting their ability to extract fine boundary features.

[0079] To alleviate these issues, the guided global semantic segmentation network (TGGNet) introduces the ASPP module. This module expands the network's receptive field by introducing multi-scale dilated convolutions, enabling it to extract both local details and global features at different scales, effectively reducing detail loss due to insufficient receptive field. In some fish images, the background often contains complex textures, lighting variations, and environmental interference. This makes accurately segmenting the fish from the background more challenging. The ASPP module utilizes multiple parallel branches, each employing dilated convolutions with different dilation rates, enabling the network to simultaneously capture fine local features and global contextual information. This multi-scale feature fusion helps improve the network's understanding of global and local semantic relationships. This mechanism enhances the distinguishability of fish from background and improves segmentation accuracy. Furthermore, different fish species vary significantly in morphology, size, and texture, requiring the network to possess strong generalization capabilities to adapt to these variations. The ASPP module uses multi-scale context modeling to not only effectively extract the local structural information of the fish body, but also enhances the overall perception of the fish body shape, enabling the network to adapt to the morphological changes of different fish species, thereby improving the robustness and generalization ability of the segmentation model.

[0080] like Figure 3 As shown, in this embodiment, the ASPP module consists of five branches, which respectively extract feature information of different scales. The specific process is as follows: (1) The fused feature map is input into the first branch module, and feature extraction is performed through the convolution layer to obtain the first branch feature map.

[0081] In this embodiment, the first branch uses 1×1 convolution, which is mainly used to extract local fine features to obtain the first branch feature map.

[0082] (2) The fused feature maps are input into the second, third and fourth branch modules respectively, and the corresponding dilated convolutional layers with different dilation rates are used for feature extraction to obtain the second branch feature map, the third branch feature map and the fourth branch feature map.

[0083] In this embodiment, the second, third, and fourth branches all use 3×3 dilated convolutions, and the dilation rates are set to 6, 12, and 18, respectively. By introducing dilated convolutions, the network can effectively expand the receptive field without increasing the amount of computation, thereby capturing a wider range of contextual information.

[0084] (3) The fused feature map is input into the fifth branch module, the feature map is compressed through the global average pooling layer, and the spatial resolution of the compressed feature image is restored through the convolution layer and upsampling module to obtain the fifth branch feature map.

[0085] In this embodiment, the fifth branch uses adaptive global average pooling to compress the fused feature map into a global context representation, and uses a 1×1 convolution to further process the feature. The spatial resolution of the feature map is then restored by upsampling to obtain the fifth branch feature map.

[0086] By utilizing global context to enhance structural perception capabilities, the global average pooling module introduced in the fifth branch extracts contextual information of the entire image, enhances the model's ability to recognize the overall morphology of the fish, and further improves the performance of fish boundary recognition and internal consistency segmentation.

[0087] (4) Splicing the first branch feature map, the second branch feature map, the third branch feature map, the fourth branch feature map, and the fifth branch feature map to obtain a spliced ​​feature map; In this embodiment, the feature maps extracted by all branches are integrated through a splicing operation, so that information from different receptive fields complement each other, thereby improving the model's ability to model multi-scale features.

[0088] (5) The convolution operation is performed on the spliced ​​feature map through the convolution layer to obtain multi-scale features.

[0089] In this embodiment, the spliced ​​feature map undergoes a 1×1 convolution to enhance information interaction between channels, further fuse multi-scale features, and compress the channel dimension to generate the final feature map, i.e., multi-scale features.

[0090] After the above steps, the multi-scale feature extraction capability is enhanced and the segmentation accuracy is improved: the ability to recognize the edge contours and overall shape of the fish body is improved, and the problem of loss of detail information caused by the limited receptive field of traditional convolution is alleviated, thereby improving the semantic segmentation accuracy.

[0091] Improving the robustness of fish segmentation in complex backgrounds: Faced with interference factors such as lighting changes and complex background textures in underwater images, the ASPP module effectively enhances the ability to semantically distinguish between fish and background through multi-scale context modeling, enabling the network to stably segment fish areas even in complex environments.

[0092] Adapting to morphological changes of different fish species and improving model generalization capabilities: Since different fish species vary in body shape, texture, and size, the multi-scale features extracted by the ASPP module help the network learn discriminative semantic representations of fish bodies, thereby giving the model strong adaptability and generalization capabilities, and maintaining high segmentation performance in a variety of fish image data.

[0093] Effectively integrate information at all scales and improve the richness of feature expression: Through multi-branch feature splicing and subsequent 1×1 convolution fusion, semantic information at different scales is uniformly integrated, enabling the model to have strong semantic understanding capabilities while maintaining spatial details, helping to achieve more refined and continuous segmentation results.

[0094] S2-2-4. Based on multi-scale features, the first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the corresponding Transformer global guidance module for feature guidance and recalibration to obtain the semantic segmentation mask map of the fish body and the calibration plate.

[0095] In the semantic segmentation network of the U-Net structure, in order to make full use of feature information of different scales, the skip connection mechanism is usually introduced to convert the feature map extracted by the deep convolutional neural network module VGG16 into And the reconstructed feature map obtained by upsampling The main function of the jump connection is to effectively combine the local detail features extracted by the encoder with the high-level semantic information generated by the decoder while preserving the spatial detail information. However, the feature map of the deep convolutional neural network module VGG16 Although it contains rich spatial details and local features, its ability to express the overall target shape and global semantics is weak due to the limited receptive field. This excessive granularity of local features may lead to a lack of global consistency of the fish body edges, which can easily lead to blurred or discontinuous fish body edge segmentation.

[0096] In contrast, the reconstructed feature map obtained by upsampling The Transformer's global modeling effectively integrates cross-regional information, demonstrating significant superiority in capturing the semantics of fish contours and shapes. This global modeling capability not only enhances the network's overall perception of the fish's shape but also effectively compensates for the global information lost during the encoding phase, thereby improving the segmentation accuracy of fish edges and detailed areas.

[0097] This embodiment introduces a Transformer Global Guidance Module (TGGM) for feature guidance and recalibration of feature maps from the encoding stage, as shown in Figure 4. The Transformer Global Guidance Module (TGGM) not only preserves the spatial detail information of features from the encoding stage but also optimizes the encoded features by leveraging the global perception capabilities of features from the decoding stage. This module uses the global semantic information of feature maps from the decoding stage as guidance and, combined with a cross-resolution feature interaction mechanism, performs weighted adjustments on the encoded feature maps to enhance the detailed representation of fish contours and shapes. This module dynamically adjusts the weight distribution of encoded features based on the global semantic information from the decoding stage, enabling the synergy between local details and global semantic information, thereby improving the network's performance in fish semantic segmentation tasks.

[0098] like Figure 1 、 Figure 4 As shown, in this embodiment, the specific process is: (1) Perform upsampling on the multi-scale features to obtain the first reconstructed feature map.

[0099] Reconstructing feature maps , the first reconstructed feature map is input to TGGM module 4 .

[0100] (2) Inputting the first reconstructed feature map and the fourth feature map into the corresponding fourth Transformer global guidance module for fusion and correction to obtain a first fused and corrected feature map; The input of Transformer Global Guided Module (TGGM) consists of two feature maps, namely the feature map extracted by the deep convolutional neural network module VGG16 and the upsampled reconstructed feature map The specific process is as follows: 1) Perform convolution and maximum pooling operations on the fourth feature map to obtain the pooled fourth feature map.

[0101] Compared with the shallow feature map extracted by the deep convolutional neural network module VGG16 , the deep feature map after upsampling and reconstruction It has stronger global information modeling capabilities and richer semantic feature expression capabilities. Therefore, in order to achieve the same resolution as Align, right Perform 1×1 convolution and maximum pooling operations to convert it into , the formula is: (6) in, Represents a 1×1 convolution operation, which is used to expand the channel dimension; Represents the maximum pooling operation, which is used to reduce the spatial resolution.

[0102] Specifically, the matching of the two input feature maps of the fourth Transformer global guidance module, TGGM module 4, in the channel dimension has been completed in advance, so the 1×1 convolution operation is not performed on the Channel dimension expansion is performed, and resolution consistency is guaranteed only through alignment operation. In TGGM module 4, according to formula (6), the fourth feature map , get the fourth feature map after pooling .

[0103] The operating logic of the other three modules is exactly the same as that of TGGM module 4.

[0104] 2) The first reconstructed feature map is added to the fourth feature map after pooling element by element, and an activation function is used to enhance the feature to obtain the first enhanced fusion feature.

[0105] The reconstructed feature map And the feature map after pooling Add element by element to enhance the global semantic information of local features, and apply the ReLU activation function (Rectified Linear Unit, ReLU) to the fused features. The formula is: (7) in, Represents the ReLU activation function, which can enhance the nonlinear expression ability of features.

[0106] Specifically, in the TGGM module 4, the first reconstructed feature map , the fourth feature map after pooling , according to formula (7), the first enhanced fusion feature is obtained .

[0107] 3) Perform global maximum pooling and global average pooling on the fused features after the first enhancement, and concatenate the two pooled features along the channel to obtain the global feature map after the first enhancement.

[0108] right Perform global context modeling, performing global maximum pooling and global average pooling respectively. Maximum pooling focuses on extracting significant features and highlighting the spatial contour information of the fish body; while global average pooling focuses on capturing global background information, effectively supplementing the semantic features of the environment, and providing a smoother and more balanced description for the global representation of features. The two context features after pooling are integrated by splicing along the channel dimension to form an enhanced global feature map. , the formula is: (8) Among them, Max represents the global maximum pooling, Avg represents the global average pooling, and || represents splicing along the channel dimension.

[0109] Specifically, in TGGM module 4, the first enhanced fusion feature , according to formula (8), the first enhanced global feature map is obtained .

[0110] 4) Perform convolution and upsampling operations on the first enhanced global feature map to obtain the first restored feature map.

[0111] After completing the feature splicing, the enhanced global feature map Perform a 7×7 convolution operation. The main purpose of this convolution operation is to smooth the global features after splicing and integrate local information, and expand the receptive field to capture richer spatial relationships and contextual dependencies in the region. Subsequently, the scale of the feature map is restored by upsampling, that is, the restored feature map is obtained, so that it is consistent with the input feature map. The same resolution is maintained to ensure the spatial alignment of context features. Specifically, in TGGM module 4, the first restored feature map is obtained.

[0112] 5) An activation function is used to generate a first weight map of the first restored feature map, and the first weight map is spatially multiplied element-by-element with the fourth feature map to obtain a first weighted correction feature map.

[0113] First, after restoring to the original resolution, the Sigmoid activation function is used to generate the weight map , used to perform weighted correction on features, the formula is: (9) in, represents a 7×7 convolution, Upsample represents an upsampling operation, and Sig represents a Sigmoid activation function.

[0114] Specifically, in TGGM module 4, the first enhanced global feature map , according to formula (9), we get the first weight graph .

[0115] Then, With the generated weight Perform spatial element-by-element multiplication to achieve weighted correction of features in the spatial dimension. Specifically, in the TGGM module 4, a first weighted correction feature map is obtained.

[0116] 6) Add the first weighted corrected feature map and the fourth feature map element by element to obtain a first fused corrected feature map.

[0117] The weighted features and Perform element-wise addition to obtain the final corrected output features , the formula is: (10) in, Represents spatial element-wise multiplication.

[0118] Specifically, in TGGM module 4, the fourth feature map , the first weight graph , according to formula (10), the final corrected output feature is obtained, that is, the first fusion corrected feature map .

[0119] (3) Perform feature splicing on the first fused corrected feature map and the first reconstructed feature map to obtain a first spliced ​​feature map.

[0120] The first fusion correction feature map With the first reconstructed feature map Perform feature splicing to obtain the first spliced ​​feature map.

[0121] (4) Perform an upsampling operation on the first concatenated feature map to obtain a second reconstructed feature map.

[0122] The second reconstructed feature map is input to the third Transformer global guidance module, TGGM module 3 .

[0123] (5) The second reconstructed feature map and the third feature map are input into the corresponding third Transformer global guidance module for fusion and correction to obtain a second fused and corrected feature map.

[0124] According to formulas (6) to (10) in steps 1) to 6) of step (2), the second fusion correction feature map is obtained .

[0125] (6) Perform feature splicing on the second fused corrected feature map and the second reconstructed feature map to obtain a second spliced ​​feature map.

[0126] The second fusion correction feature map With the second reconstructed feature map Perform feature splicing to obtain the first spliced ​​feature map.

[0127] (7) Perform an upsampling operation on the second concatenated feature map to obtain a third reconstructed feature map.

[0128] The third reconstructed feature map is input to the second Transformer global guidance module, TGGM module 2 .

[0129] (8) The third reconstructed feature map and the second feature map are input into the corresponding second Transformer global guidance module for fusion and correction to obtain a third fused and corrected feature map.

[0130] According to formulas (6) to (10) in steps 1) to 6) of step (2), the third fusion correction feature map is obtained .

[0131] (9) Perform feature splicing on the third fused corrected feature map and the third reconstructed feature map to obtain a third spliced ​​feature map.

[0132] The third fusion correction feature map and the third reconstructed feature map Perform feature splicing to obtain the first spliced ​​feature map.

[0133] (10) An upsampling operation is performed on the third concatenated feature map to obtain a fourth reconstructed feature map.

[0134] The fourth reconstructed feature map is input to the first Transformer global guidance module, TGGM module 1 .

[0135] (11) The fourth reconstructed feature map and the first feature map are input into the corresponding first Transformer global guidance module for fusion and correction to obtain a fourth fused and corrected feature map.

[0136] According to formulas (6) to (10) in steps 1) to 6) of step (2), the fourth fusion correction feature map is obtained .

[0137] (12) Perform feature splicing on the fourth fused corrected feature map and the fourth reconstructed feature map to obtain a fourth spliced ​​feature map.

[0138] The fourth fusion correction feature map and the fourth reconstructed feature map Perform feature splicing to obtain the first spliced ​​feature map.

[0139] (13) The fourth concatenated feature map is convolved through a convolution layer to obtain a semantic segmentation mask map of the fish body and the calibration plate.

[0140] The global guided semantic segmentation network (TGGNet) is used to perform fine mask segmentation on the fish body and calibration plate in the distortion-corrected image, as shown in the following example: Figure 7 shown.

[0141] Through steps (2) (5) (8) (11), the alignment and complementarity of deep and shallow features are strengthened. The TGGM module achieves effective alignment and complementarity of low-level fine edge information and high-level semantic information by performing multi-layer fusion and guided correction of shallow features (such as the fourth feature map) and upsampled deep reconstruction features (such as the first reconstruction feature map), thereby enhancing the modeling ability of fish body edge contours and structural details.

[0142] In steps (2), (5), (8), and (11), global context information is introduced to achieve accurate correction. That is, the TGGM module uses global maximum pooling and global average pooling mechanisms to construct a global context feature map and use it to guide the feature correction process, as shown in step (2) (3). This process enhances the network's ability to recognize the overall morphology and spatial structure of the fish body, effectively avoids local missegmentation, and improves the overall mask consistency.

[0143] In steps (2) (5) (8) (11), by generating a spatial weight map and performing weighted correction on the features, specifically as step 5) in step (2), the perceptual guidance of the salient area is achieved, and the key target area is weightedly enhanced. The TGGM module can adaptively highlight the saliency of the fish body area while suppressing background interference, achieving spatial guidance and emphasis on the key target area, and improving the network's ability to focus on the target area.

[0144] In steps (1)(2)(4)(5)(7)(8)(10)(11), the multi-scale fusion mechanism enhances the generalization and robustness of the model. That is, in the process of upsampling and fusing the feature maps reconstructed at different scales layer by layer, the TGGM module not only achieves consistent matching of spatial resolution, but also effectively integrates the semantic features at different scales, making the model more generalizable and adaptable to various fish morphologies and complex background changes.

[0145] Through the layer-by-layer guided correction and fusion of the TGGM module, the final spliced ​​feature map is more complete, with clearer boundaries and richer semantic expression, providing high-quality feature support for the subsequent convolutional output of the fish body and calibration plate mask, significantly improving the overall semantic segmentation performance.

[0146] After the global guided semantic segmentation network is trained according to step S2-2 using a training dataset of 1080 images, the global guided semantic segmentation network is tested using a test dataset of 720 images.

[0147] Based on the trained global guided semantic segmentation network, accurate semantic segmentation masks of the fish body and calibration plate are obtained.

[0148] like Figure 1 As shown, in step S3, based on the semantic segmentation mask image of the calibration plate, a mapping relationship between the physical size and the pixel size of the calibration plate is constructed; the semantic segmentation mask image of the fish body is subjected to ellipse fitting to calculate the pixel length of the fish body; based on the mapping relationship between the physical size and the pixel size of the calibration plate, the pixel length of the fish body is converted into the actual length of the fish body.

[0149] S3-1. Based on the semantic segmentation mask of the calibration plate, a mapping relationship between the physical size of the calibration plate and the pixel size is constructed.

[0150] In this embodiment, since the fish body and the calibration plate in the image are located on the same horizontal plane, the segmentation results can be used to establish a proportional relationship between the pixels of the fish body and the calibration plate. Combined with the physical size information of the calibration plate, a mapping relationship between the pixel length of the fish body and the actual physical size can be further constructed.

[0151] Based on the calibration plate contour obtained by segmentation, the minimum bounding rectangle fitting method is used to extract its bounding rectangle features and calculate its pixel perimeter in the image. The actual physical circumference of the calibration plate =2×(h+w)×d, where d is the side length of the grid, h and w are the number of grids in the height and width directions of the calibration plate. Combined with the pixel perimeter of the minimum circumscribed rectangle of the calibration plate and physical perimeter The correspondence between them is used to establish a mapping relationship between pixels and actual physical sizes.

[0152] S3-2. Perform ellipse fitting on the fish body semantic segmentation mask image and calculate the pixel length of the fish body.

[0153] In current aquaculture research, some studies often use ellipse fitting methods to process fish body contours for subsequent analysis and calculations. This is because fish body contours are mostly slender, with a distinct aspect ratio, which facilitates mathematical modeling and calculations. Therefore, ellipse fitting methods are used to process fish body contours.

[0154] In the specific implementation process, the cv2.fitEllipse() function provided by OpenCV is used to fit the contour points based on the least squares method. The specific process is: (1) Rotate the fish body mask image.

[0155] Based on the center coordinates and the inclination of the major axis relative to the horizontal line obtained after fitting the ellipse, the fish mask is rotated to the horizontal direction through affine transformation, so that the major axis of the ellipse is parallel to the horizontal direction of the image, thereby standardizing the direction of the fish mask in the image and improving the accuracy of subsequent calculations. The effect of the fish mask before and after the transformation is as follows: Figure 8 As shown, Figure 8 The green box in (b) represents the circumscribed rectangle of the fish body outline.

[0156] (2) After the rotation transformation is completed, the long side of the circumscribed rectangle of the fish body outline is used as the pixel length of the fish body .

[0157] After the above steps, the fish posture is standardized by introducing the ellipse fitting and image affine transformation mechanism, thus providing a more accurate and robust geometric basis for subsequent length measurement.

[0158] Traditional fish contouring often suffers from inconsistent orientation due to factors such as shooting angle and posture changes, affecting measurement accuracy. This method uses the least squares method to fit an elliptical model to accurately obtain the major axis direction and center point coordinates of the fish contour. Combined with an affine transformation, the fish's orientation is aligned with the image's horizontal direction, regularizing the fish contour and effectively improving the accuracy and consistency of length extraction.

[0159] S3-3. Based on the mapping relationship between the physical size of the calibration plate and the pixel size, the pixel length of the fish body is converted into the actual length of the fish body.

[0160] Combined with the pixel-physical size mapping relationship of the calibration plate, the real length of the fish can be calculated , the formula is: (11) After the above steps, the pixel value of the long side of the circumscribed rectangle of the fish body outline is used as the estimated value of the fish body length. Combined with the pixel-physical size mapping relationship of the calibration plate, the pixel length can be reliably converted into the actual physical length, ensuring that the measurement results have physical meaning and comparability.

[0161] The method in this embodiment improves the normalization of fish body orientation, facilitating subsequent unified processing; enhances the accuracy and stability of fish body length calculation; can adapt to a variety of fish body shapes and shooting scenes, and is highly robust; has good portability and engineering application value, and is suitable for deployment in actual aquaculture monitoring systems.

[0162] S4. Index Evaluation and Performance Experiments S4-1. Indicator evaluation of the global guided semantic segmentation network and fish body length estimation.

[0163] (1) Evaluation indicators of fish body semantic segmentation experiment.

[0164] The performance of the fish body semantic segmentation model is evaluated using indicators such as Mean Intersection over Union (mIoU), Dice Similarity Coefficient, Precision, Recall, and Accuracy. Among them, mIoU is mainly used to measure the degree of overlap between the predicted segmentation result and the true label. The higher the value, the better the model segmentation effect; the Dice similarity coefficient is particularly important in measuring the consistency between the segmentation result and the true target area. It is particularly suitable for datasets with uneven category distribution and can more sensitively reflect the performance of the model in target area segmentation; the precision rate is used to measure the proportion of correct predictions among the results predicted by the model as positive samples, thereby evaluating the model's misjudgment of non-target areas. The recall rate indicates the proportion of positive samples correctly predicted by the model to all true positive samples, and is used to measure the model's recognition ability of the target area; the accuracy rate is used to evaluate the accuracy of the model's overall prediction, reflecting its classification performance on all samples. The specific calculation formula is as follows: (12) (13) (14) (15) (16) (17) Where N represents the total number of segmentation categories, i represents the category index, TP (True Positive), TN (True Negative), FN (False Negative), and FP (False Positive) represent the number of true positive, true negative, false negative, and false positive pixels predicted as the category in the segmentation results, respectively.

[0165] (2) Evaluation indicators of fish length estimation experiment The mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE) are used to evaluate the performance of the fish length estimation method to comprehensively measure its accuracy and stability. The specific calculation formula is as follows: (18) (19) (20) Among them, N represents the number of predicted fish bodies, is the fish body index, is the true length, For estimated length.

[0166] S4-2. Fish body semantic segmentation ablation experiment and comparative experiment (1) Ablation experiment Ablation experiments were conducted to verify the impact of each TGGNet module on model performance. Using U-Net as the baseline model, the experiments sequentially added various TGGNet modules to this model and compared segmentation performance under different configurations. The experimental results are shown in Tables 3 and 4. These results show that the addition of each module significantly improved all evaluation metrics of the model, demonstrating that these modules play a significant role in feature extraction, semantic understanding, and segmentation accuracy, effectively improving the overall performance of TGGNet.

[0167] Table 3 Ablation experiment results: overall evaluation indicators

[0168] As shown in Table 3, the U-Net baseline model achieves a mean Intersection Over Union (MIoU) of 92.67% and a Dice coefficient of 96.11%. Precision, recall, and accuracy are 95.52%, 96.73%, and 99.11%, respectively, demonstrating that the U-Net model possesses high basic performance in the fish semantic segmentation task. However, in the presence of strong background interference or diverse fish morphologies, the model may suffer from blurred boundaries or misclassification. Introducing the residual Transformer module into the baseline model improves the mIoU by 0.71% and the Dice coefficient by 0.4%. Other metrics also improve, demonstrating that the residual Transformer module enhances the model's feature modeling capabilities through a global self-attention mechanism, leading to improved segmentation accuracy. Further, adding the ASPP module to the residual Transformer module increases the mIoU to 93.71% and the Dice coefficient to 96.59%. This demonstrates that the ASPP module effectively aggregates multi-scale features, improving the model's adaptability to fish of varying sizes. When the TGGM module is used instead of the ASPP module, the model performance is further improved, with mIoU reaching 94.54% and the Dice coefficient increasing to 97.15%. This shows that TGGM optimizes the feature expression capability through cross-scale feature interaction and global guidance mechanism, enabling the model to more accurately capture the edge details of the fish body. Finally, a complete TGGNet is constructed, and the ASPP module is further introduced on the basis of residual Transformer and TGGM. The mIoU is increased to 94.89%, the Dice coefficient reaches 97.38%, the precision, recall and accuracy reach 97.32%, 97.49% and 99.41% respectively, achieving the best performance under the data set and evaluation indicators set in this experiment. This shows that the residual Transformer module, ASPP module and TGGM module can work together to improve the feature modeling ability and segmentation accuracy of the model. Table 4 Ablation experiment results: IoU of each category

[0169] Table 4 shows the IoU results for various categories in the ablation experiment, further validating the improvement achieved by each module on the fish semantic segmentation task. The baseline model performs well on the background and calibration plate categories, with IoUs of 99.18% and 99.09%, respectively. However, its performance on the fish category is relatively poor, with IoUs of 88.14%, 91.29%, and 85.66% for crucian carp, sea bass, and tilapia, respectively. Due to the large morphological variations and complex surface textures of fish, U-Net has limitations in feature extraction, which in turn affects the fine-grained segmentation of fish. Introducing the residual Transformer module into the baseline model significantly improves the IoU for tilapia by 3.01%, while the IoU improvements for crucian carp and sea bass are relatively small, indicating that the residual Transformer module still has certain limitations in improving some categories. Adding the ASPP module significantly improved the IoU for seabass, from 91.39% to 93.79%, but the IoU for crucian carp and tilapia decreased to 88.02% and 88.20%, respectively. The ASPP module may introduce some redundant features during the multi-scale information aggregation process, weakening the feature representation of small-scale objects. Replacing the ASPP module with the TGGM module further improved the model's segmentation performance for fish, reaching IoUs of 89.73% for crucian carp, 93.73% for seabass, and 90.71% for tilapia. This demonstrates that the TGGM module, through its cross-scale feature interaction mechanism, optimizes the fusion of low-level spatial information and high-level semantic features, thereby improving the segmentation accuracy of fish contours. Ultimately, TGGNet, combining the residual Transformer module, the ASPP module, and the TGGM module, achieved the best segmentation results for all categories. TGGNet can make good use of multi-scale information and multi-level features, and through the synergy between modules, it can further refine the fish body boundary segmentation effect while maintaining high-level semantic information.

[0170] (2) Comparative experiment To verify the advancedness of the proposed network, it is compared with several mainstream semantic segmentation algorithms, including FCN, Deeplabv3+, PSPNet, U-Net (baseline model), TransUNet, TransUNet+ and HRNet (for semantic segmentation). The experimental results are shown in Tables 5 and 6.

[0171] Table 5 Comparative experimental results: overall evaluation indicators

[0172] Table 5 shows that the segmentation performance of FCN, Deeplabv3+, and PSPNet is relatively low. Although Deeplabv3+ and PSPNet utilize multi-scale feature extraction techniques such as spatial pyramid pooling, they still lack information interaction between high- and low-level features. In contrast, TransUNet and TransUNet+, based on the Transformer framework, demonstrate advantages in global information modeling, but still have some shortcomings. TransUNet's overall performance is slightly lower than U-Net. Although TransUNet possesses global modeling capabilities based on the Transformer framework, it is limited in preserving low-level feature details, which affects the segmentation accuracy of small object areas. In addition, the lack of an effective interactive fusion mechanism between high-level global features and low-level local features further restricts the network's performance. While TransUNet+ performs slightly better than U-Net, it is still lower than TGGNet overall. HRNet excels in preserving and fusing multi-scale features, however, HRNet still has certain limitations.

[0173] Table 6 Comparative experimental results: IoU of each category

[0174] The comparative experimental results in Table 6 show that all models achieve high IoUs for the background and calibration plate categories. TGGNet achieves the best performance on these two categories, reaching 99.46% and 99.16%, respectively, surpassing the other models. This is due to TGGNet's ability to effectively model global feature correlations and focus on the overall feature expression of the fish body, further improving the model's segmentation capabilities. The performance differences among the models are even more pronounced for the fish category. FCN's IoU is lower than that of the other models for all fish categories. In contrast, Deeplabv3+ and PSPNet achieve slightly better IoUs, but still underperform on the fish category. U-Net's performance is relatively balanced; TransUNet's performance is slightly lower than U-Net, while TransUNet+ performs slightly better than U-Net. HRNet performs well on all fish categories, but its IoU for crucian carp and sea bass is lower than that of TGGNet. TGGNet achieves IoUs of 90.33% and 94.19% for crucian carp and sea bass, respectively, significantly exceeding the other models. This shows that TGGNet has more advantages in global modeling of multi-scale features.

[0175] like Figure 9 The visualization results of fish body semantic segmentation are shown. Compared with other network models, the proposed TGGNet achieves the best performance in segmentation accuracy and robustness.

[0176] S4-3. Fish body length estimation experiment (1) Analysis of the applicability of the fish body length estimation method based on ellipse fitting to different fish species In the process of estimating fish body length, the differences in body structure, surface texture, and edge features of different fish species may affect the accuracy of the estimation method. This section aims to evaluate the fish body length estimation method based on ellipse fitting, analyze its applicability to different fish species, and explore whether there are significant differences. A total of 80 length estimates were conducted for each fish species, and the experimental results are shown in Tables 7 and Figure 10 shown.

[0177] Table 7 Statistics of fish length estimation errors for different fish species

[0178] From Table 7 and Figure 10 As shown, a one-way analysis of variance (ANOVA) of the estimation errors across different fish species revealed a p-value of 0.41, exceeding the significance level of 0.05. This indicates that the ellipse-fitting-based fish length estimation method exhibits no significant differences in estimation errors across different fish species. This method demonstrates relatively stable performance across different fish species, demonstrating good applicability.

[0179] (2) Analysis of the impact of environmental changes on fish length estimation To evaluate the applicability of the fish length estimation method based on ellipse fitting in different environments, four shooting environments were designed for experiments, including normal lighting, dim lighting, shadow environment, and complex background. The fish length estimation errors under different environmental conditions were compared and analyzed to verify the robustness of the method. In each environment, fish length estimation was performed 60 times. The experimental results are shown in Table 8 and Figure 11 shown.

[0180] Table 8 Statistics of fish length estimation errors in different environments

[0181] As shown in Table 8 and Figure 11 As shown in the figure, a one-way ANOVA was conducted on the estimation errors in different environments. The results showed a p-value of 0.60, which is greater than the significance level of 0.05. This indicates that the estimation error of the fish length estimation method based on ellipse fitting does not differ significantly across different shooting environments. Whether in normal lighting, dim light, shadowed environments, or complex backgrounds, the estimation error does not show significant fluctuations across different environments. This result further verifies the robustness and reliability of TGGNet and the fish length estimation method based on ellipse fitting under different environmental conditions.

[0182] (3) Comparative experiment On the data used in this experiment, the fish body length estimation method based on image refinement and the fish body length estimation method based on minimum bounding rectangle fitting were reproduced, and compared with the fish body length estimation method in this paper. The results are shown in Table 9 and Figure 12 shown.

[0183] Table 9 Fish body length estimation error results of different methods

[0184] Compared to the other two methods, the image refinement-based fish length estimation method produces an overall larger estimate. The minimum bounding rectangle-based method has a slightly higher overall error than the ellipse-fitting-based method. Compared to minimum bounding rectangle fitting, the ellipse-fitting method more accurately fits the principal axes and shape of the fish, thereby better capturing the fish's geometric characteristics. In particular, the ellipse-fitting method can effectively reduce errors caused by posture changes when the fish is tilted or has an irregular shape. Therefore, the ellipse-fitting-based fish length estimation method demonstrates higher accuracy in processing fish contours.

[0185] The method in this embodiment is applied to the estimation of the length of the fish body in the image to be detected, specifically: The fish image to be detected is input into the trained global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask of the fish body and the calibration plate; Perform ellipse fitting on the fish body semantic segmentation mask image and calculate the pixel length of the fish body; Based on the mapping relationship between the physical size of the calibration plate and the pixel size, the pixel length of the fish body is converted into the actual length of the fish body, that is, the estimated value of the fish body length in the measured fish body image is obtained.

[0186] Example 2 The purpose of this embodiment is to provide a fish length estimation system based on a global guided semantic segmentation network, including: An image acquisition module is used to acquire a monocular image of the fish body, wherein the monocular image of the fish body includes the fish body and the calibration plate; and perform distortion correction on the monocular image of the fish body to obtain a corrected monocular image of the fish body; The mask image generation module is used to input the corrected fish body monocular image into the global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask image of the fish body and the calibration plate; The length estimation module is used to construct a mapping relationship between the physical size and pixel size of the calibration plate based on the semantic segmentation mask of the calibration plate; perform ellipse fitting on the semantic segmentation mask of the fish body to calculate the pixel length of the fish body; and convert the pixel length of the fish body into the actual length of the fish body based on the mapping relationship between the physical size and pixel size of the calibration plate.

[0187] Based on the provided fish body length estimation system based on the global guided semantic segmentation network, the method steps in Example 1 are implemented.

[0188] Example 3 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method in the first embodiment are implemented.

[0189] Example 4 This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method in the first embodiment are performed.

[0190] The steps involved in the apparatus of the above embodiment correspond to those of the method embodiment 1. For detailed implementation, please refer to the relevant description of embodiment 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any method of the present invention.

[0191] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0192] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A fish body length estimation method based on a global guided semantic segmentation network, characterized by: include: Acquire a monocular image of a fish to be tested, wherein the monocular image of the fish to be tested includes the fish body and a calibration plate; Performing distortion correction on the monocular image of the fish body to obtain a corrected monocular image of the fish body; The corrected fish monocular image is input into the trained global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask of the fish body and the calibration plate; Based on the semantic segmentation mask image of the calibration plate, a mapping relationship between the physical size and the pixel size of the calibration plate is constructed; Performing ellipse fitting on the fish body semantic segmentation mask image to calculate the pixel length of the fish body; Based on the mapping relationship between the physical size of the calibration plate and the pixel size, the pixel length of the fish body is converted into the actual length of the fish body.

2. The method for estimating fish body length based on a global guided semantic segmentation network according to claim 1, wherein: The corrected fish monocular image is input into the trained global guided semantic segmentation network for semantic segmentation. The trained global guided semantic segmentation network includes a deep convolutional neural network module, a residual Transformer module, a dilated spatial pyramid pooling module, and multiple Transformer global guided modules. The specific process is as follows: Input the corrected fish monocular image into the deep convolutional neural network module to extract features at different scales to obtain the first feature map, the second feature map, the third feature map, the fourth feature map, and the fifth feature map; Inputting the fourth feature map into the residual Transformer module for feature extraction and fusion to obtain a fused feature map; The fused feature map is input into the dilated spatial pyramid pooling module for feature extraction to obtain multi-scale features; Based on multi-scale features, the first feature map, the second feature map, the third feature map and the fourth feature map are respectively input into the corresponding Transformer global guidance module for feature guidance and recalibration to obtain the semantic segmentation mask map of the fish body and the calibration plate.

3. The method for estimating fish body length based on a global guided semantic segmentation network as claimed in claim 2, wherein: The fourth feature map is input into the residual transformer module for feature extraction, wherein the residual transformer module includes a feature map block module, a linear mapping module, a position encoding module, a transformer encoding module and a reshaping module. The specific process is as follows: Converting the fourth feature map into a vectorized feature image block by a feature map block module; Performing linear mapping on the vectorized feature image block through a linear mapping module to obtain a mapped feature image block; Performing spatial encoding on the mapped feature image blocks through a position encoding module to obtain serialized features; Inputting the serialized features into the Transformer encoding module for feature extraction and enhancement to obtain an enhanced feature map; Reshaping the enhanced feature map by a reshaping module to obtain a reshaped feature map; The reshaped feature map and the fifth feature map are added element by element to obtain a fused feature map.

4. The method for estimating fish body length based on a global guided semantic segmentation network as claimed in claim 2, wherein: The fused feature map is input into the dilated spatial pyramid pooling module for feature extraction to obtain multi-scale features. The dilated spatial pyramid pooling module includes five parallel branch modules. The specific process is as follows: The fused feature map is input into the first branch module, and features are extracted through the convolution layer to obtain the first branch feature map; The fused feature maps are input into the second, third and fourth branch modules respectively, and feature extraction is performed using corresponding dilated convolutional layers with different dilation rates to obtain the second branch feature map, the third branch feature map and the fourth branch feature map; The fused feature map is input into the fifth branch module, the feature map is compressed through the global average pooling layer, and the spatial resolution of the compressed feature image is restored through the convolution layer and upsampling module to obtain the fifth branch feature map; Splicing the first branch feature map, the second branch feature map, the third branch feature map, the fourth branch feature map, and the fifth branch feature map to obtain a spliced ​​feature map; A convolution operation is performed on the spliced ​​feature map through a convolution layer to obtain multi-scale features.

5. The method for estimating fish body length based on a global guided semantic segmentation network as claimed in claim 2, wherein: Based on the multi-scale features, the first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the corresponding Transformer global guidance module for feature guidance and recalibration. The specific process is as follows: Performing an upsampling operation on the multi-scale features to obtain a first reconstructed feature map; Inputting the first reconstructed feature map and the fourth feature map into the corresponding fourth Transformer global guidance module for fusion and correction to obtain a first fused and corrected feature map; Perform feature splicing on the first fused corrected feature map and the first reconstructed feature map to obtain a first spliced ​​feature map; Performing an upsampling operation on the first concatenated feature map to obtain a second reconstructed feature map; Inputting the second reconstructed feature map and the third feature map into the corresponding third Transformer global guidance module for fusion and correction to obtain a second fused and corrected feature map; Perform feature splicing on the second fused corrected feature map and the second reconstructed feature map to obtain a second spliced ​​feature map; Performing an upsampling operation on the second concatenated feature map to obtain a third reconstructed feature map; Inputting the third reconstructed feature map and the second feature map into the corresponding second Transformer global guidance module for fusion and correction to obtain a third fused and corrected feature map; Performing feature splicing on the third fused corrected feature map and the third reconstructed feature map to obtain a third spliced ​​feature map; Performing an upsampling operation on the third spliced ​​feature map to obtain a fourth reconstructed feature map; Inputting the fourth reconstructed feature map and the first feature map into the corresponding first Transformer global guidance module for fusion and correction to obtain a fourth fused and corrected feature map; Performing feature splicing on the fourth fused corrected feature map and the fourth reconstructed feature map to obtain a fourth spliced ​​feature map; A convolution operation is performed on the fourth spliced ​​feature map through a convolution layer to obtain a semantic segmentation mask map of the fish body and the calibration plate.

6. The method for estimating fish body length based on a global guided semantic segmentation network according to claim 5, wherein: The first reconstructed feature map and the fourth feature map are input into the corresponding fourth Transformer global guidance module for fusion and correction. The specific process is as follows: Performing convolution and maximum pooling operations on the fourth feature map to obtain a pooled fourth feature map; Adding the first reconstructed feature map and the pooled fourth feature map element by element, and performing feature enhancement using an activation function to obtain a first enhanced fusion feature; Performing global maximum pooling and global average pooling on the first enhanced fusion features, and concatenating the two pooled features along the channel to obtain a first enhanced global feature map; Performing convolution and upsampling operations on the first enhanced global feature map to obtain a first restored feature map; Generate a first weight map using an activation function, and perform spatial element-wise multiplication of the first weight map with the fourth feature map to obtain a first weighted correction feature map; The first weighted corrected feature map and the fourth feature map are added element by element to obtain a first fused corrected feature map.

7. The method for estimating fish body length based on a global guided semantic segmentation network according to claim 1, wherein: The formula for converting the pixel length of the fish body into the actual length of the fish body is: ； in, Indicates the pixel length of the fish body, specifically the long side of the circumscribed rectangle of the fish body outline; represents the pixel perimeter of the minimum circumscribed rectangle of the calibration plate, Indicates the actual physical circumference of the calibration plate.

8. Fish body length estimation system based on global guided semantic segmentation network, characterized by: include: An image acquisition module is configured to acquire a monocular image of a fish body, wherein the monocular image of the fish body includes the fish body and a calibration plate; and perform distortion correction on the monocular image of the fish body to obtain a corrected monocular image of the fish body; The mask image generation module is used to input the corrected fish body monocular image into the global guided semantic segmentation network for semantic segmentation to obtain the semantic segmentation mask image of the fish body and the calibration plate; The length estimation module is used to construct a mapping relationship between the physical size and pixel size of the calibration plate based on the semantic segmentation mask map of the calibration plate; perform ellipse fitting on the semantic segmentation mask map of the fish body to calculate the pixel length of the fish body; and convert the pixel length of the fish body into the actual length of the fish body based on the mapping relationship between the physical size and pixel size of the calibration plate.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Images