Underwater fish school semantic measurement method and system and electronic equipment

Abstract

The invention discloses an underwater fish school semantic measurement method and system and electronic equipment, and the method comprises the following steps: carrying out the enhancement of an underwater fish school image through an image enhancement network, and obtaining an enhanced image; according to the image enhancement network, a GhostConv module is introduced behind an encoder and a decoder of a U-shape-Transform model, and a GhostSkip jump connection module is introduced between the encoder and the decoder; performing target detection on the enhanced image by using a target detection network to obtain a target detection result; according to the target detection network, an RF-SKAB mechanism is introduced into C3K2 of a YOLO model to form a C3K2-RFSKAB module, an SODNet feature fusion module is introduced into a Neck part, and SPDConv is adopted to replace downsampling operation; generating a saliency map which highlights the fish body region based on the target detection result by using a saliency detection network, and obtaining a semantic segmentation result based on the saliency map by using an improved SAM model; the improved SAM model is formed by inserting a lightweight multi-scale adapter and a detail enhancement module behind an encoder layer in the SAM model.

Description

Technical Field

[0001] This invention relates to the field of semantic segmentation technology, and in particular to a method, system and electronic device for semantic measurement of underwater fish schools. Background Technology

[0002] In applications such as underwater fishery resource surveys, ecological monitoring, and intelligent aquaculture, vision-based fish school analysis technology is gradually becoming an important tool. However, existing methods still face a series of technical bottlenecks in practical applications, mainly in the following three aspects:

[0003] In underwater image enhancement: The underwater imaging environment is complex, and light is easily absorbed and scattered in water, leading to widespread image degradation such as blurring, color distortion, low contrast, and uneven illumination, severely impacting the input quality for subsequent visual tasks. Traditional enhancement methods, mostly based on physical models or histogram adjustments, often struggle to adapt to different water quality and lighting conditions, easily introducing noise or losing details. While deep learning-based methods (such as the U-Net architecture) have made progress in image restoration, they typically involve large numbers of model parameters, computational complexity, and lack lightweight and targeted designs for underwater fish scenarios, making it difficult to achieve real-time and reliable image enhancement in resource-constrained underwater equipment.

[0004] In underwater fish target identification: existing fish detection methods mostly follow general target detection frameworks (such as YOLO, Faster R-CNN, etc.). However, underwater fish targets are characterized by large scale differences, similar morphology, and dense distribution, especially when small targets coexist with complex backgrounds (such as algae and sediment), leading to prominent false negatives and false negatives. Furthermore, poor underwater image quality further increases the difficulty of feature extraction and target differentiation. Current methods often do not fully consider multi-scale feature fusion mechanisms and background interference suppression strategies, resulting in insufficient adaptability of the models to the underwater environment. The detection accuracy and robustness are insufficient to meet the accurate statistical needs for fish population and distribution information in actual fisheries surveys and ecological monitoring.

[0005] In the semantic segmentation of underwater fish schools: semantic-level segmentation of fish schools can not only locate the fish's position but also further identify their morphological structures (such as fins and tails), thereby enabling quantitative analysis of individual size, body proportions, and behavioral postures. This type of information is crucial for assessing fish growth status, health levels, and population structure, providing scientific guidance for intelligent aquaculture management, feeding strategy optimization, and sustainable resource utilization. However, existing semantic segmentation models perform poorly in underwater scenarios: on the one hand, poor underwater image quality and blurred target edges easily lead to inaccurate segmentation boundaries, structural breaks, or discontinuities; on the other hand, fish exhibit diverse morphologies and varied postures, making it difficult for models to adapt to their high intraclass variability. Furthermore, mainstream methods typically rely on large amounts of finely labeled data and have high computational costs, making them unsuitable for dynamic, real-time underwater monitoring systems. Although large-scale visual models demonstrate strong segmentation potential, they usually require manual prompting for activation, have low automation levels, and are not optimized for the morphological characteristics and multi-scale features of underwater fish, resulting in insufficient segmentation integrity of fine structures such as tails and fins, limiting the effectiveness of subsequent semantic measurements and growth analysis. Summary of the Invention

[0006] The purpose of this invention is to solve the problems in the prior art.

[0007] The technical solution adopted by this invention to solve its technical problem is: to provide a semantic measurement method for underwater fish schools, comprising the following steps:

[0008] The image enhancement step involves using an image enhancement network to enhance the underwater fish image, resulting in an enhanced image. The image enhancement network is based on the U-shape-Transformer model, with a GhostConv module introduced after the encoder and decoder, and a GhostSkip skip connection module introduced between the encoder and decoder.

[0009] The target detection step involves using a target detection network to detect targets in the enhanced image and obtaining the target detection results. The target detection network is based on the YOLO model, introduces the RF-SKAB mechanism into C3K2 to form the C3K2-RFSKAB module, introduces the SODNet feature fusion module in the Neck part, and replaces the downsampling operation with SPDConv.

[0010] The semantic segmentation step involves using a saliency detection network to generate a saliency map highlighting the fish body region based on the target detection results, and then using an improved SAM model to obtain the semantic segmentation result based on the saliency map. The improved SAM model consists of inserting a lightweight multi-scale adapter and a detail enhancement module after the encoder layer in the SAM model.

[0011] Preferably, the GhostConv module includes two stages: main path convolution and cheap operation convolution. The main path convolution receives the input feature map and uses 1×1 convolution combined with batch normalization and ReLU activation function to generate a basic feature map. The cheap operation convolution performs depthwise separable convolution transformation on the basic features to generate supplementary features, which are defined as Ghost features. The basic features and Ghost features are concatenated along the channel dimension to obtain the output feature map of the GhostConv module.

[0012] Preferably, the GhostSkip skip connection module performs the following operations on the features between the encoder and decoder:

[0013] Bilinear interpolation upsampling is performed on the high-level features of the encoder to achieve spatial size alignment. Then, channel compression and reconstruction are performed through the Ghost module to solve the problem of mismatch in the number of feature channels between different levels, and the processed encoder features are obtained.

[0014] The processed encoder features are fused with the corresponding layer features of the decoder to promote efficient interaction between shallow detail information and deep semantic information.

[0015] Preferably, the structure of the C3K2-RFSKAB module is as follows:

[0016] When the C3K2 output is False, it includes sequentially connected Conv convolutions, Split segmentation, two R-Bottlenecks, Concat concatenation, and Conv convolutions. The R-Bottleneck consists of a Conv convolution, an adaptive receptive field convolution RFSKABConv, and a skip connection across layers. The RFSKABConv layer includes three parallel processing branches and a fusion output layer: a global channel attention branch, a grouped feature transformation branch, a local spatial attention branch, and a Conv output convolution.

[0017] When the C3K2 output is True, it includes sequentially connected Conv convolutions, Split segmentation, two C3K-RFSKABs, Concat concatenation, and Conv convolutions. Among them, C3K-RFSKABs include an input Conv convolution, a segmentation layer, two cascaded R-Bottleneck sub-modules, a concatenation layer, and an output Conv convolution. Its workflow follows the standard "convolution-segmentation-dual-bottleneck processing-fusion-convolution" paradigm. The input is split after the initial convolution, and the branches are processed for features through two R-Bottlenecks. Subsequently, the branch features are merged and finally output through the convolutional layer.

[0018] Preferably, the workflow of the RFSKABConv includes:

[0019] In the global channel attention branch, the features are first compressed into channel descriptors by global average pooling, and then passed through a lightweight gating network consisting of a linear fully connected layer, a ReLU activation function, a fully connected layer and a sigmoid activation function to finally generate a weight vector representing the global importance of each channel.

[0020] In the group feature transformation branch, the input features are efficiently interacted and transformed through group convolution, then normalized and ReLU activated, and then a "shape adjustment" operation is performed to restore or reorganize the channel and spatial structure of the feature map.

[0021] Meanwhile, the local spatial attention branch performs AvgPool average pooling and MaxPool max pooling on the input features respectively, and concatenates the outputs of the two in the channel dimension; then a spatial attention weight map is learned and generated through a Conv convolutional layer and a Sigmoid activation function. This weight map is used to "reweight" the output of the group feature transformation branch to highlight key spatial region information.

[0022] The outputs of the three paths are integrated: the channel weight vector output by the global channel attention branch is multiplied channel by channel with the grouped features after spatial reweighting, thereby achieving collaborative calibration in channel and spatial dimensions. The calibrated features are finally integrated and dimensionally transformed by a Conv output convolutional layer to form the final output of RFSKABConv.

[0023] Preferably, the structure or workflow of the SODNet feature fusion module is as follows:

[0024] First, the input features are processed by a 1×1 convolution for channel compression and basic integration;

[0025] Subsequently, a feature segmentation strategy was adopted to divide it into two parts: only 25% of the features entered the subsequent multi-branch processing module, while the remaining 75% of the features were directly retained as a shortcut path, thereby reducing the amount of computation while preserving the original backbone information.

[0026] Next, the features entering the multi-branch processing stage are processed by three parallel paths: the small target branch uses 1×1 depthwise separable convolution to capture detailed textures; the large target branch extracts long-range spatial context through multi-scale strip convolution; and the global branch enhances global representation and key feature selection capabilities through FSAM frequency domain attention mechanism and DCAM spatial domain attention mechanism in turn.

[0027] Finally, the outputs of the three branches are concatenated with the previously retained 75% of the original features, and then fused and adjusted through a 1×1 convolution to form the final output.

[0028] Preferably, the SPDConv module is used to replace the traditional downsampling operation. The workflow of the SPDConv module includes two stages: lossless reconstruction in the spatial dimension and re-encoding of features in the channel dimension.

[0029] In the first stage, the module performs a space-to-depth transformation on the input feature map. The space-to-depth transformation first divides the feature map into a set of regular, non-overlapping sub-regions in space, and then, through a rearrangement operation, completely transfers all spatial location information in each sub-region to the channel dimension.

[0030] In the second stage, a standard convolutional layer is used to process the reconstructed feature map.

[0031] Preferably, the semantic segmentation step includes the following steps:

[0032] A saliency map is generated by using a saliency detection network to highlight the fish body region, and key points are automatically sampled or bounding boxes are generated based on prior knowledge of fish morphology.

[0033] A lightweight multi-scale adapter is introduced after the frozen SAM image encoder to extract and fuse information from different receptive fields in parallel through multi-scale pooling or small convolution kernels.

[0034] A detail enhancement module is introduced after the lightweight multi-scale adapter. It adopts a dual-branch structure: the main branch processes high-level semantic features, and the auxiliary branch extracts edge information from the original image or shallow features. By fusing the two, the segmentation effect of fine structures such as fins and tails is enhanced.

[0035] A connectivity prediction head is introduced after the SAM image decoder to predict the connectivity between pixels, and a connectivity loss function is introduced to guide the model to generate a segmentation mask with more continuous boundaries and a more reasonable structure.

[0036] The present invention also provides an underwater fish school semantic measurement system, comprising:

[0037] The image enhancement module uses an image enhancement network to enhance underwater fish images to obtain enhanced images. The image enhancement network is based on the U-shape-Transformer model, and introduces a GhostConv module after the encoder and decoder, and a GhostSkip skip connection module between the encoder and decoder.

[0038] The target detection module uses a target detection network to perform target detection on the enhanced image and obtain the target detection result. The target detection network uses the YOLO model as the basis, introduces the RF-SKAB mechanism into C3K2 to form the C3K2-RFSKAB module, introduces the SODNet feature fusion module in the Neck part, and uses SPDConv to replace the downsampling operation.

[0039] The semantic segmentation module uses a saliency detection network to generate a saliency map of the prominent fish body region based on the target detection results, and uses an improved SAM model to obtain the semantic segmentation result based on the saliency map; the improved SAM model is composed of a lightweight multi-scale adapter and a detail enhancement module inserted after the encoder layer of the SAM model.

[0040] The present invention also provides an electronic device, comprising:

[0041] One or more processors;

[0042] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to perform any of the methods described above.

[0043] This invention provides an end-to-end underwater fish swarm semantic measurement method, system, and electronic device. Through a set of collaborative technical modules, it effectively improves the clarity of underwater images, achieves accurate localization of multi-scale fish swarm targets, and completes detailed and topologically correct fish semantic segmentation. Simultaneously, it ensures the entire system has low parameter count and computational complexity, facilitating deployment in practical underwater monitoring equipment. It possesses the following significant advantages:

[0044] (1) By connecting the Ghost module introduced in the image enhancement stage with GhostSkip, the underwater image quality is effectively improved while the model is lightweight, making it more suitable for deployment on edge devices with limited computing resources. In the target detection stage, through the synergistic effect of the RF-SKAB attention mechanism and the SODNet multi-branch fusion design, the model can adaptively focus on the target and effectively cope with scale changes, especially the detection recall rate of small fish groups is greatly improved;

[0045] (2) In the semantic segmentation stage, through a series of innovations such as automatic prompt generation, multi-scale adaptation, detail enhancement and connectivity prediction, the segmentation accuracy and topological integrity of the fine structure of underwater fish have been significantly improved on the basis of inheriting the powerful segmentation capabilities of the large model, and finally a set of efficient, accurate and practical overall solutions for semantic measurement of underwater fish groups have been formed.

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the steps of an underwater fish school semantic measurement method according to an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of the image enhancement network according to an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the target detection network according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the C3K2-RFSKAB module according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of the structure of RFSKABConv according to an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of the SODNet feature fusion module according to an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of the Omnikernel branch in the SODNet feature fusion module of this embodiment of the invention;

[0054] Figure 8 This is a schematic diagram of the operation flow of the SPDConv module according to an embodiment of the present invention;

[0055] Figure 9 This is a detailed flowchart of the semantic segmentation steps of an underwater fish school semantic measurement method according to an embodiment of the present invention;

[0056] Figure 10 This is a schematic diagram of the structure of an underwater fish school semantic measurement system according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0058] See Figure 1 As shown, the core technical solution of this invention consists of a coherent end-to-end three-stage process, including the following steps:

[0059] S101, Image enhancement step: The underwater fish image is enhanced using an image enhancement network to obtain an enhanced image; The image enhancement network is based on the U-shape-Transformer model, and a GhostConv module is introduced after the encoder and decoder, and a GhostSkip skip connection module is introduced between the encoder and decoder.

[0060] S102, Target Detection Step: Target detection is performed on the enhanced image using a target detection network to obtain the target detection result. The target detection network uses the YOLO model as its basis, introduces the RF-SKAB mechanism in C3K2 to form the C3K2-RFSKAB module, introduces the SODNet feature fusion module in the Neck part, and replaces the downsampling operation with SPDConv.

[0061] S103, Semantic segmentation step: A saliency detection network is used to generate a saliency map of the prominent fish body region based on the target detection result, and an improved SAM model is used to obtain the semantic segmentation result based on the saliency map; the improved SAM model is composed of a lightweight multi-scale adapter and a detail enhancement module inserted after the encoder layer in the SAM model.

[0062] In the image enhancement stage, the model employs an improved model based on the U-Shape-Transformer architecture, such as... Figure 2As shown in the diagram, this architecture, while retaining the original Transformer structure, introduces two types of lightweight functional modules: the GhostConv module and the GhostSkip skip connection module. Together, they constitute the model's feature extraction and cross-layer information fusion mechanism. The core idea of ​​the GhostConv module is to achieve lightweight feature generation through feature decoupling. Its structure includes two stages: main path convolution and low-cost operation convolution. The main path uses 1×1 convolution combined with batch normalization and ReLU activation function to generate a small number of basic feature maps, undertaking the core feature extraction task. The low-cost operation convolution performs depthwise separable convolution transformation on the basic features, generating multiple sets of supplementary features, called "Ghost features," with extremely low computational cost. Finally, the basic features and Ghost features are concatenated along the channel dimension to form a complete output feature map. This design transforms the redundant computation of traditional convolutional layers into linear transformations, significantly reducing parameter scale and computational complexity while maintaining feature expressiveness. The GhostSkip skip connection module is used to optimize feature transfer between the encoder and decoder. This module first performs bilinear interpolation upsampling on the high-level features of the encoder to achieve spatial size alignment. Then, the Ghost module performs channel compression and reconstruction to address the mismatch in the number of feature channels between different levels. Finally, the processed encoder features are fused with the corresponding layer features of the decoder, promoting efficient interaction between shallow detail information and deep semantic information. This mechanism, while retaining the multi-scale advantages of the U-shaped network, further improves feature reuse efficiency and reconstruction consistency.

[0063] In the object detection stage, the model follows the classic Backbone, Neck, and Head structure, such as... Figure 3 As shown. To enhance the model's ability to focus on underwater targets, this scheme introduces an RF-SKAB attention module in the Neck section, resulting in the C3K2-RFSKAB module as shown. Figure 4 As shown, this module achieves this through... Figure 5 The parallel processing space and channel attention path are shown, and the results are dynamically fused using learnable parameters to adaptively adjust the receptive field, enhance the features of the target region, and suppress interference from complex backgrounds.

[0064] The C3K2-RFSKAB module has the following two structures:

[0065] When the C3K2 output is False, the process includes sequentially connected Conv convolutions, Split segmentation, two R-Bottlenecks, Concat concatenation, and Conv convolutions. The R-Bottleneck consists of a Conv convolution, an adaptive receptive field convolution RFSKABConv, and a skip connection across layers. This module deepens the network while maximizing computational efficiency and training stability, aiming to enhance the model's feature selection and representation capabilities. RFSKABConv comprises three parallel processing branches and a fusion output layer: a global channel attention branch, a grouped feature transformation branch, a local spatial attention branch, and a Conv output convolution. In the global channel attention branch, features are first compressed into channel descriptors using Global Average Pooling, then passed through a lightweight gating network consisting of a Linear fully connected layer, a ReLU activation function, a fully connected layer, and a Sigmoid activation function, ultimately generating a weight vector representing the global importance of each channel. In the grouped feature transformation branch, the input features undergo efficient feature interaction and transformation through Group Convolution, followed by normalization and ReLU activation, and then a "shape adjustment" operation to restore or reconstruct the channel and spatial structure of the feature map. Simultaneously, the local spatial attention branch performs AvgPool average pooling and MaxPool max pooling on the input features, concatenating their outputs along the channel dimension. Subsequently, a Conv convolutional layer and a Sigmoid activation function are used to learn and generate a spatial attention weight map, which is used to "reweight" the output of the grouped feature transformation branch to highlight key spatial region information. Finally, the module integrates the outputs of the three paths: the channel weight vector output from the global channel attention branch is multiplied channel-by-channel with the spatially reweighted grouped features, achieving collaborative calibration along the channel and spatial dimensions. The calibrated features are finally processed by a Conv output convolutional layer for information integration and dimensional transformation, forming the module's final output.

[0066] When the C3K2 output is True, it consists of sequentially connected Conv convolutions, Split segmentation, two C3K-RFSKABs, Concat concatenation, and Conv convolutions. Each C3K-RFSKAB comprises an input Conv convolution, a segmentation layer, two cascaded R-Bottleneck submodules, a concatenation layer, and an output Conv convolution. Its workflow follows the standard "convolution-segmentation-dual-bottleneck processing-fusion-convolution" paradigm. The input is split after the initial convolution, and each branch undergoes feature processing through two R-Bottlenecks. Subsequently, the branch features are merged and finally output through the convolutional layer. This submodule can be repeatedly stacked (N times) to efficiently build depth within the network.

[0067] To address the issue of underwater fish schools varying in size, especially the tendency to miss small targets, this invention employs an SODNet feature fusion module. This module segments the input features proportionally. Figure 6 Multi-branch calculation is performed only on a small subset of features, such as Figure 7 The method includes a small target branch that uses 1x1 depthwise separable convolution to capture local details, a large target branch that uses multi-scale strip convolution to capture long-range dependencies, and a global branch that combines frequency domain and spatial domain attention. Finally, the outputs of all branches are recombined with the original features, which significantly improves the recall rate of small targets with only a few additional parameters.

[0068] The structure or workflow of the SODNet feature fusion module is as follows: First, the input features are compressed and integrated using a 1×1 convolution. Then, a feature segmentation strategy is employed, dividing them into two parts: only 25% of the features enter the subsequent multi-branch processing module, while the remaining 75% are directly retained as a shortcut, thus reducing computational cost while preserving the original backbone information. Next, the features entering the multi-branch processing stage are processed by three parallel paths: the small target branch uses a 1×1 depthwise separable convolution to capture detailed textures; the large target branch extracts long-range spatial context through multi-scale strip convolutions; and the global branch enhances global representation and key feature selection capabilities through FSAM frequency domain attention and DCAM spatial domain attention mechanisms. Finally, the outputs of the above three branches are concatenated with the previously retained 75% of the original features, and then fused and adjusted using a 1×1 convolution to form the final output. This process, through partial feature depth processing, multi-scale information complementarity, and residual connections, significantly enhances the model's perception and fusion capabilities for small underwater targets while maintaining lightweight design.

[0069] Furthermore, this solution uses the SPDConv module to replace the traditional downsampling operation. The SPDConv structure is as follows: Figure 8By segmenting the feature map into sub-blocks and recombining them from space to depth, lossless downsampling is achieved, better preserving high-frequency details of small targets. The SPDConv module replaces traditional downsampling operations. The core design of the SPDConv module lies in decoupling and optimizing the inherent information compression and feature encoding operations in traditional downsampling to achieve high-fidelity feature map resolution reduction. Its workflow can be summarized in two clear stages: lossless spatial dimension reconstruction and channel dimension feature re-encoding. In the first stage, the module performs a space-to-depth transformation on the input feature map. This transformation first divides the feature map into a set of regular, non-overlapping sub-regions in space, and then, through a rearrangement operation, completely transfers all spatial location information in each sub-region to the channel dimension. This process directly reduces the spatial size of the feature map and correspondingly increases the channel dimension without any pooling or stepping, thus mathematically ensuring the integrity of spatial information and effectively avoiding the loss of high-frequency details. In the second stage, the module uses a standard convolutional layer to process the recombined feature map. The role of this convolutional layer is to fuse, filter, and compress the surge in channel information, mapping it to the target output channel number, and in the process, learning and extracting higher-level abstract features. By separating pure spatial structure transformation from learnable feature extraction, the SPDConv module preserves the geometric structure and details of the input data to the maximum extent while performing downsampling, significantly enhancing the model's robustness in representing subtle deformations and unconventional poses of the target.

[0070] In the semantic segmentation stage, this scheme makes targeted improvements to the powerful SAM model to achieve end-to-end fish segmentation. The structure is as follows: Figure 9As shown, firstly, an automatic prompt generator is designed. It utilizes a saliency detection network to generate saliency maps highlighting fish body regions and automatically samples key points or generates bounding boxes based on prior knowledge of fish morphology (such as elongated structures), thus eliminating reliance on manual prompts. Secondly, to enhance the SAM original image encoder's ability to perceive multi-scale features of underwater fish, this scheme inserts a lightweight multi-scale adapter after the frozen encoder layer. This adapter extracts and fuses information from different receptive fields in parallel using multi-scale pooling or small convolutional kernels, improving multi-scale feature capture capabilities with minimal parameter cost. Subsequently, this scheme introduces a detail enhancement module with a dual-branch structure. The main branch processes high-level semantic features, while the auxiliary branch extracts edge information from the original image or shallow features. By fusing the two, the segmentation effect of fine structures such as fins and tails is enhanced. Finally, in order to improve the continuity of the segmentation results from the topological level and prevent broken tails or fins, this scheme adds an additional connectivity prediction head. This head does not directly predict the pixel category, but predicts the connectivity relationship between pixels, and guides the model to generate a segmentation mask with more continuous boundaries and a more reasonable structure by introducing a connectivity loss function.

[0071] See Figure 10 The diagram shown is a structural schematic of an underwater fish school semantic measurement system according to an embodiment of the present invention, comprising:

[0072] Image enhancement module 1001 uses an image enhancement network to enhance underwater fish images to obtain enhanced images; the image enhancement network is based on the U-shape-Transformer model, introduces a GhostConv module after the encoder and decoder, and introduces a GhostSkip jump connection module between the encoder and decoder.

[0073] The target detection module 1002 uses a target detection network to perform target detection on the enhanced image and obtain the target detection result. The target detection network uses the YOLO model as the basis, introduces the RF-SKAB mechanism in C3K2 to form the C3K2-RFSKAB module, introduces the SODNet feature fusion module in the Neck part, and uses SPDConv to replace the downsampling operation.

[0074] The semantic segmentation module 1003 uses a saliency detection network to generate a saliency map of the prominent fish body region based on the target detection result, and uses an improved SAM model to obtain the semantic segmentation result based on the saliency map; the improved SAM model is composed of a lightweight multi-scale adapter and a detail enhancement module inserted after the encoder layer in the SAM model.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantic measurement method for underwater fish schools, characterized in that, Includes the following steps: The image enhancement step involves using an image enhancement network to enhance the underwater fish image, resulting in an enhanced image. The image enhancement network is based on the U-shape-Transformer model, with a GhostConv module introduced after the encoder and decoder, and a GhostSkip skip connection module introduced between the encoder and decoder. The target detection step involves using a target detection network to detect targets in the enhanced image and obtaining the target detection results. The target detection network is based on the YOLO model, introduces the RF-SKAB mechanism into C3K2 to form the C3K2-RFSKAB module, introduces the SODNet feature fusion module in the Neck part, and replaces the downsampling operation with SPDConv. The semantic segmentation step involves using a saliency detection network to generate a saliency map highlighting the fish body region based on the target detection results, and then using an improved SAM model to obtain the semantic segmentation result based on the saliency map. The improved SAM model consists of inserting a lightweight multi-scale adapter and a detail enhancement module after the encoder layer in the SAM model.

2. The underwater fish school semantic measurement method according to claim 1, characterized in that, The GhostConv module consists of two stages: main path convolution and cheap operation convolution. The main path convolution receives the input feature map and uses 1×1 convolution combined with batch normalization and ReLU activation function to generate a basic feature map. The cheap operation convolution performs depthwise separable convolution transformation on the basic features to generate supplementary features, which are defined as Ghost features. The basic features and Ghost features are concatenated along the channel dimension to obtain the output feature map of the GhostConv module.

3. The underwater fish school semantic measurement method according to claim 1, characterized in that, The GhostSkip skip connection module performs the following operations on the features between the encoder and decoder: Bilinear interpolation upsampling is performed on the high-level features of the encoder to achieve spatial size alignment. Then, channel compression and reconstruction are performed through the Ghost module to solve the problem of mismatch in the number of feature channels between different levels, and the processed encoder features are obtained. The processed encoder features are fused with the corresponding layer features of the decoder to promote efficient interaction between shallow detail information and deep semantic information.

4. The underwater fish school semantic measurement method according to claim 1, characterized in that, The structure of the C3K2-RFSKAB module is as follows: When the C3K2 output is False, it includes sequentially connected Conv convolutions, Split segmentation, two R-Bottlenecks, Concat concatenation, and Conv convolutions. The R-Bottleneck consists of a Conv convolution, an adaptive receptive field convolution RFSKABConv, and a skip connection across layers. The RFSKABConv layer includes three parallel processing branches and a fusion output layer: a global channel attention branch, a grouped feature transformation branch, a local spatial attention branch, and a Conv output convolution. When the C3K2 output is True, it includes sequentially connected Conv convolutions, Split segmentation, two C3K-RFSKABs, Concat concatenation, and Conv convolutions. Among them, C3K-RFSKABs include an input Conv convolution, a segmentation layer, two cascaded R-Bottleneck sub-modules, a concatenation layer, and an output Conv convolution. Its workflow follows the standard "convolution-segmentation-dual-bottleneck processing-fusion-convolution" paradigm. The input is split after the initial convolution, and the branches are processed for features through two R-Bottlenecks. Subsequently, the branch features are merged and finally output through the convolutional layer.

5. The underwater fish school semantic measurement method according to claim 4, characterized in that, The workflow of RFSKABConv includes: In the global channel attention branch, the features are first compressed into channel descriptors by global average pooling, and then passed through a lightweight gating network consisting of a linear fully connected layer, a ReLU activation function, a fully connected layer and a sigmoid activation function to finally generate a weight vector representing the global importance of each channel. In the group feature transformation branch, the input features are efficiently interacted and transformed through group convolution, then normalized and ReLU activated, and then a "shape adjustment" operation is performed to restore or reorganize the channel and spatial structure of the feature map. Meanwhile, the local spatial attention branch performs AvgPool average pooling and MaxPool max pooling on the input features respectively, and concatenates the outputs of the two in the channel dimension; then a spatial attention weight map is learned and generated through a Conv convolutional layer and a Sigmoid activation function. This weight map is used to "reweight" the output of the group feature transformation branch to highlight key spatial region information. The outputs of the three paths are integrated: the channel weight vector output by the global channel attention branch is multiplied channel by channel with the grouped features after spatial reweighting, thereby achieving collaborative calibration in channel and spatial dimensions. The calibrated features are finally integrated and dimensionally transformed by a Conv output convolutional layer to form the final output of RFSKABConv.

6. The underwater fish school semantic measurement method according to claim 1, characterized in that, The structure or workflow of the SODNet feature fusion module: First, the input features are processed by a 1×1 convolution for channel compression and basic integration; Subsequently, a feature segmentation strategy was adopted to divide it into two parts: only 25% of the features entered the subsequent multi-branch processing module, while the remaining 75% of the features were directly retained as a shortcut path, thereby reducing the amount of computation while preserving the original backbone information. Next, the features entering the multi-branch processing stage are processed by three parallel paths: the small target branch uses 1×1 depthwise separable convolution to capture detailed textures; the large target branch extracts long-range spatial context through multi-scale strip convolution; and the global branch enhances global representation and key feature selection capabilities through FSAM frequency domain attention mechanism and DCAM spatial domain attention mechanism in turn. Finally, the outputs of the three branches are concatenated with the previously retained 75% of the original features, and then fused and adjusted through a 1×1 convolution to form the final output.

7. The underwater fish school semantic measurement method according to claim 1, characterized in that, The SPDConv module is used to replace the traditional downsampling operation. The workflow of the SPDConv module includes two stages: lossless reconstruction in the spatial dimension and re-encoding of features in the channel dimension. In the first stage, the module performs a space-to-depth transformation on the input feature map; The space-to-depth transformation first divides the feature map into a set of regular, non-overlapping sub-regions in space, and then, through a rearrangement operation, completely transfers all spatial location information in each sub-region to the channel dimension. In the second stage, a standard convolutional layer is used to process the reconstructed feature map.

8. The underwater fish school semantic measurement method according to claim 1, characterized in that, The semantic segmentation step includes the following steps: A saliency map is generated by using a saliency detection network to highlight the fish body region, and key points are automatically sampled or bounding boxes are generated based on prior knowledge of fish morphology. A lightweight multi-scale adapter is introduced after the frozen SAM image encoder to extract and fuse information from different receptive fields in parallel through multi-scale pooling or small convolution kernels. A detail enhancement module is introduced after the lightweight multi-scale adapter. It adopts a dual-branch structure: the main branch processes high-level semantic features, and the auxiliary branch extracts edge information from the original image or shallow features. By fusing the two, the segmentation effect of fine structures such as fins and tails is enhanced. A connectivity prediction head is introduced after the SAM image decoder to predict the connectivity between pixels, and a connectivity loss function is introduced to guide the model to generate a segmentation mask with more continuous boundaries and a more reasonable structure.

9. An underwater fish swarm semantic measurement system, characterized in that, include: The image enhancement module uses an image enhancement network to enhance underwater fish images, resulting in enhanced images. The image enhancement network uses the U-shape-Transformer model as its foundation, introduces the GhostConv module after the encoder and decoder, and introduces the GhostSkip skip connection module between the encoder and decoder. The target detection module uses a target detection network to perform target detection on the enhanced image and obtain the target detection result. The target detection network uses the YOLO model as the basis, introduces the RF-SKAB mechanism into C3K2 to form the C3K2-RFSKAB module, introduces the SODNet feature fusion module in the Neck part, and uses SPDConv to replace the downsampling operation. The semantic segmentation module uses a saliency detection network to generate a saliency map of the prominent fish body region based on the target detection results, and uses an improved SAM model to obtain the semantic segmentation result based on the saliency map; the improved SAM model is composed of a lightweight multi-scale adapter and a detail enhancement module inserted after the encoder layer of the SAM model.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to perform the method as described in any one of claims 1-6.

Images