Improved u-net-based segmentation measurement method and system for macrobrachium rosenbergii

Abstract

This invention discloses a segmentation and measurement method and system for giant freshwater prawns based on an improved U-Net, belonging to the field of intelligent monitoring technology for aquaculture. The method includes: constructing an enhanced semantic segmentation model RVA-UNet, which adopts an encoder-decoder symmetric structure; training and segmentation: using the constructed RVA-UNet model to perform semantic segmentation on images of giant freshwater prawns under test, outputting segmentation masks for each part of the prawn body and automated measurement, significantly improving the segmentation accuracy of giant freshwater prawns under complex backgrounds and poses; proposing a skeletonization repair and projection compensation mechanism to solve the measurement error problem caused by the breakage and fragmentation of slender structures; achieving fully automated conversion from image input to biological length output; avoiding stress and errors caused by manual measurement; and being suitable for large-scale, intelligent aquaculture scenarios.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology for aquaculture, and in particular to a segmentation measurement method and system for giant freshwater prawns based on an improved U-Net. Background Technology

[0002] The giant freshwater prawn (Macrobrachium rosenbergii) is one of the world's largest-scale farmed and most economically valuable freshwater shrimp species. Accurate monitoring of its growth indicators, such as body length and weight, is crucial for achieving precision aquaculture. Traditional manual measurement methods suffer from low accuracy, high labor intensity, and are prone to causing stress to the shrimp, making them unsuitable for the intelligent management needs of modern aquaculture. In recent years, machine vision technology has been gradually applied to aquatic animal measurement; however, existing methods largely rely on traditional image processing or basic convolutional neural networks, which have the following shortcomings: poor adaptability to the scale differences between the shrimp's slender limbs (such as pincers and walking legs) and broad trunk; low segmentation accuracy under complex postures, overlapping limbs, and background interference; and difficulty in achieving high-precision automated conversion from image to biological length. Therefore, there is an urgent need for a method and system capable of high-precision segmentation of the giant freshwater prawn and achieving non-contact automated measurement. Summary of the Invention

[0003] To overcome the shortcomings of the prior art, the purpose of this invention is to provide a method and system for segmenting and measuring giant freshwater prawns based on an improved U-Net.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: This invention provides a segmentation measurement method for giant freshwater prawns based on an improved U-Net, comprising: The enhanced semantic segmentation model RVA-UNet is constructed. This model adopts an encoder-decoder symmetric structure, specifically including: The encoder incorporates a residual convolution module, Res-CNN, which alleviates gradient vanishing through identity mapping and enhances the response to slender limb features. A visual transformer module (ViT) is embedded in the deepest coding unit to construct long-distance spatial dependencies using a self-attention mechanism and identify limb associations under complex postures. The bottleneck layer integrates the Spatial Convolution Pooling Pyramid Module (ASPP), which captures morphological features of different sizes through multi-scale dilated convolution. A spatial attention upsampling module is introduced into the decoding path to suppress background noise through an additive gated attention mechanism; Training and segmentation: The constructed RVA-UNet model is used to perform semantic segmentation on the test images of giant freshwater prawns, and the segmentation masks of each part of the giant freshwater prawn body are output. Automated measurement: The output segmentation mask is processed to measure the dimensions of various parts of the giant freshwater prawn, specifically including: Morphological topological refinement is performed on the segmentation mask to extract the skeleton with a single pixel width; A breakpoint reconnection mechanism based on directional coherence is used to connect the fractured skeleton; For extremely fine structures, a fragment projection compensation model is used to estimate the geometric length of the missing parts; For solid structures of the cephalothorax and abdomen, a smooth midline was extracted based on constrained Voronoi diagrams and multinomial regression. The body length, carapace length, abdominal length, and chelicerae length are calculated by path integration over the central axis or skeleton.

[0005] Furthermore, the composite loss function used in the training process of the RVA-UNet model is a weighted combination of weighted cross-entropy loss and Dice loss, wherein the weight of Dice loss is λ=3.

[0006] Furthermore, the automated measurement also includes a geometric topology repair module, used to remove burrs, connect fracture points, and compensate for micro-fragments during the skeletonization process.

[0007] This invention also provides a segmentation and measurement system for giant freshwater prawns based on an improved U-Net, comprising: Build modules: Used to construct the enhanced semantic segmentation model RVA-UNet, which employs an encoder-decoder symmetric structure, specifically including: The encoder incorporates a residual convolution module, Res-CNN, which alleviates gradient vanishing through identity mapping and enhances the response to slender limb features. A visual transformer module (ViT) is embedded in the deepest coding unit to construct long-distance spatial dependencies using a self-attention mechanism and identify limb associations under complex postures. The bottleneck layer integrates the Spatial Convolution Pooling Pyramid Module (ASPP), which captures morphological features of different sizes through multi-scale dilated convolution. A spatial attention upsampling module is introduced into the decoding path to suppress background noise through an additive gated attention mechanism; Training and segmentation module: Used for training and segmentation: The constructed RVA-UNet model is used to perform semantic segmentation on the test images of giant freshwater prawns, and outputs the segmentation masks of each part of the giant freshwater prawn body; Measurement module: Used for automated measurement: Processes the output segmentation mask and completes the measurement of various parts of the giant freshwater prawn's body, specifically including: Morphological topological refinement is performed on the segmentation mask to extract the skeleton with a single pixel width; A breakpoint reconnection mechanism based on directional coherence is used to connect the fractured skeleton; For extremely fine structures, a fragment projection compensation model is used to estimate the geometric length of the missing parts; For solid structures of the cephalothorax and abdomen, a smooth midline was extracted based on constrained Voronoi diagrams and multinomial regression. The body length, carapace length, abdominal length, and chelicerae length are calculated by path integration over the central axis or skeleton.

[0008] Compared with the prior art, the technical solution disclosed in this invention has the following beneficial effects: 1. By leveraging the synergistic effects of Res-CNN, ViT, ASPP, and attention mechanisms, the segmentation accuracy of Macrobrachium rosenbergii under complex backgrounds and poses was significantly improved, achieving a Dice coefficient of 97.7% and an MIoU of 96.7%. 2. A skeleton-based repair and projection compensation mechanism was proposed to solve the measurement error problem caused by the fracture and fragmentation of slender structures; 3. It achieves fully automated conversion from image input to biological length output, avoiding stress and errors caused by manual measurement, and is suitable for large-scale, intelligent aquaculture scenarios. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the RVA-UNet network structure provided in an embodiment of the present invention; Figure 2 This is a structural diagram of the Res-CNN residual convolution module in an embodiment of the present invention; Figure 3 This is a structural diagram of the ViT deep global modeling module in an embodiment of the present invention; Figure 4 This is a structural diagram of the ASPP module in an embodiment of the present invention; Figure 5 This is a structural diagram of the spatial attention upsampling module in an embodiment of the present invention; Figure 6 This is a schematic diagram of the breakpoint reconnection mechanism based on directional coherence in an embodiment of the present invention; Figure 7 This is a schematic diagram of fragment projection compensation for extremely fine structures in an embodiment of the present invention; Figure 8 This is a diagram showing the results of centerline extraction based on constrained Voronoi diagrams in an embodiment of the present invention. Figure 9In this embodiment, a is the segmentation result diagram, and b is the automatic measurement output result diagram in this embodiment. Figure 10 This is a schematic diagram of the process for segmentation and measurement of giant freshwater prawns based on an improved U-Net, provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] like Figures 1-10 As shown, embodiments of the present invention provide a method and system for segmenting and measuring giant freshwater prawns based on an improved U-Net, comprising: S1: Construct the enhanced semantic segmentation model RVA-UNet, which adopts an encoder-decoder symmetric structure, specifically including: S11: The encoder introduces a residual convolution module Res-CNN, which alleviates gradient vanishing through identity mapping and enhances the response of slender limb features; S12: Embed the Vision Transformer module ViT in the deepest coding unit to construct long-distance spatial dependencies using a self-attention mechanism and identify limb associations under complex postures; S13: The bottleneck layer integrates the Spatial Convolution Pooling Pyramid Module (ASPP), which captures morphological features of different sizes through multi-scale dilated convolution. S14: A spatial attention upsampling module is introduced into the decoding path to suppress background noise through an additive gating attention mechanism.

[0014] S2: Training and Segmentation: The RVA-UNet model constructed in S1 is used to perform semantic segmentation on the test images of giant freshwater prawns, and the segmentation masks of each part of the giant freshwater prawn body are output. The composite loss function used in the training process of the RVA-UNet model is a weighted combination of weighted cross-entropy loss and Dice loss, where the weight of Dice loss is λ=3, in order to enhance the segmentation sensitivity of slender limbs.

[0015] S3: Automated Measurement: Processes the segmentation mask output from S2 and completes the measurement of the dimensions of various parts of the giant freshwater prawn, specifically including: S31: Perform morphological topological refinement on the segmentation mask to extract the skeleton with a single pixel width; S32: A breakpoint reconnection mechanism based on directional coherence is used to connect the fractured skeleton; S33: Use a fragment projection compensation model for extremely fine structures to estimate the geometric length of the missing parts; S34: For cephalothorax and abdominal solid structures, smooth midline is extracted based on constrained Voronoi diagram and multinomial regression. S35: Calculate the body length, carapace length, abdominal length, and chelicerae length by performing path integration over the central axis or skeleton.

[0016] The automated measurement also includes a geometric topology repair module, used to remove burrs, connect break points, and compensate for small fragments during the skeletonization process.

[0017] Based on the same idea, embodiments of the present invention also provide a segmentation and measurement system for giant freshwater prawns based on an improved U-Net, including: Build modules: Used to construct the enhanced semantic segmentation model RVA-UNet, which employs an encoder-decoder symmetric structure, specifically including: S11: The encoder introduces a residual convolution module Res-CNN, which alleviates gradient vanishing through identity mapping and enhances the response of slender limb features; S12: Embed the Vision Transformer module ViT in the deepest coding unit to construct long-distance spatial dependencies using a self-attention mechanism and identify limb associations under complex postures; S13: The bottleneck layer integrates the Spatial Convolution Pooling Pyramid Module (ASPP), which captures morphological features of different sizes through multi-scale dilated convolution. S14: A spatial attention upsampling module is introduced into the decoding path to suppress background noise through an additive gated attention mechanism; Training and segmentation module: Used for training and segmentation: The constructed RVA-UNet model is used to perform semantic segmentation on the test images of giant freshwater prawns, and outputs the segmentation masks of each part of the giant freshwater prawn body; Measurement module: Used for automated measurement: Processes the output segmentation mask and completes the measurement of various parts of the giant freshwater prawn's body, specifically including: S31: Perform morphological topological refinement on the segmentation mask to extract the skeleton with a single pixel width; S32: A breakpoint reconnection mechanism based on directional coherence is used to connect the fractured skeleton; S33: Use a fragment projection compensation model for extremely fine structures to estimate the geometric length of the missing parts; S34: For cephalothorax and abdominal solid structures, smooth midline is extracted based on constrained Voronoi diagram and multinomial regression. S35: Calculate the body length, carapace length, abdominal length, and chelicerae length by performing path integration over the central axis or skeleton.

[0018] The specific implementation method is as follows: Example 1: RVA-UNet Model Construction like Figure 1 As shown, RVA-UNet uses U-Net as its baseline architecture, and the encoder part adopts the Res-CNN residual convolution module (e.g., Figure 2 As shown in the diagram, each residual module contains two 3×3 convolutional layers, batch normalization, and a ReLU activation function, and element-wise addition of the input and output is achieved through short-circuit connections. When the input and output channels are inconsistent, 1×1 convolutions are used for projection alignment.

[0019] After the fourth layer downsampling, embed the ViT module (such as...). Figure 3 As shown), the feature map is flattened into a sequence, and global context modeling is achieved through a multi-head self-attention mechanism. The bottleneck layer introduces an ASPP module (such as...). Figure 4 As shown in the figure, parallel dilated convolution branches with dilation rates of 6, 12 and 18 are set to capture multi-scale features.

[0020] In the decoder, the upsampled deep features and the encoder's shallow features are processed by a spatial attention module (e.g., ...). Figure 5 (As shown) are fused to generate an attention map to suppress background noise.

[0021] During training, a composite loss function is used, where the Dice loss weight λ can be set according to the actual situation to enhance the segmentation sensitivity of slender limbs.

[0022] Example 2: Automated Measurement Process After binarizing the segmentation results, a parallel thinning operator is used to extract the skeleton. To address the skeleton breakage issue, a breakpoint reconnection mechanism (such as...) is employed. Figure 6 (As shown): Morphological refinement is performed on the segmented binary mask to obtain multiple skeleton segments and their endpoints; for the identified independent skeleton segments, principal component analysis (PCA) is used to calculate their local tangent vectors. This serves as the initial state for subsequent geometric matching. During segmentation of the same category, the skeleton will have multiple skeleton lines, and a heuristic reconnection mechanism based on directional coherence is used. For any two isolated skeleton segments in space... and The algorithm first searches its endpoint set. Let... for The endpoints, for The endpoints. Define the connection vector. The merging logic must simultaneously satisfy both the spatial distance threshold and the angle consistency constraint:

[0023] This formula ensures that the two structural segments are physically connected, where... The distance is the Euclidean distance, which is the straight-line distance (in pixels) between two breakpoints. The spatial distance threshold (initially set to 20 pixels by default) is used as follows: Figure 6 As shown.

[0024]

[0025] Cosine similarity constraint, definition The tangent vector at the endpoints is ,since point to The vector is This formula mathematically represents a very strong consistency in opposite directions. The angle between the distances between endpoints and the tangent vector is calculated; a connection is made when the distance is less than a threshold and the cosine of the angle is less than -0.9.

[0026] For extremely fine structures such as pliers, a fragment projection compensation model (e.g.) is used. Figure 7 As shown): Extract the main support structure, calculate the projection of the fragment centroid onto the main support direction, and compensate for the missing length.

[0027] For the cephalothorax and abdomen, the constrained Voronoi diagram method (e.g.) is used. Figure 8 As shown): Extract the contour point set, construct the Voronoi diagram, retain the internal vertices, sort them by PCA, fit the second-order parametric curve, and finally calculate the length by line integral.

[0028] The final measurement results are output through a visual interface (such as...). Figure 9 (as shown) Figure 9 The 'a' in the diagram shows the segmentation effect of different regions. Figure 9 The b in the image can display the measurement results of different parts in real time, including indicators such as body length, carapace length, abdominal length, and chelicerae length.

[0029] Example 3: Experimental Verification On a dataset containing 170 augmented images, with the training and validation sets split in an 8:2 ratio, the RVA-UNet model achieved a Dice coefficient of 97.7% and an MIoU of 96.7%, significantly outperforming existing models such as U-Net, DeepLabV3+, and PSPNet. Ablation experiments demonstrate that the synergistic effect of Res-CNN, ViT, ASPP, and the attention module is key to its performance improvement.

[0030] RVA-UNet's superior performance is not due to a single improvement, but rather the result of the synergistic effect of its modules: Res-CNN ensures feature depth, ViT and ASPP expand feature breadth, and Attention-up optimizes the purity of feature fusion.

[0031] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0032] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0033] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of the present invention.

[0034] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0035] It should be understood that the qualifying terms "first", "second", "third", "fourth", "fifth" and "sixth" used in the description of the embodiments of the present invention are only used to more clearly illustrate the technical solutions and are not intended to limit the scope of protection of the present invention.

[0036] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A segmentation and measurement method for giant freshwater prawns based on an improved U-Net, characterized in that, include: The enhanced semantic segmentation model RVA-UNet is constructed. This model adopts an encoder-decoder symmetric structure, specifically including: The encoder incorporates a residual convolution module, Res-CNN, which alleviates gradient vanishing through identity mapping and enhances the response to slender limb features. A visual transformer module (ViT) is embedded in the deepest coding unit to construct long-distance spatial dependencies using a self-attention mechanism and identify limb associations under complex postures. The bottleneck layer integrates the Spatial Convolution Pooling Pyramid Module (ASPP), which captures morphological features of different sizes through multi-scale dilated convolution. A spatial attention upsampling module is introduced into the decoding path to suppress background noise through an additive gated attention mechanism; Training and segmentation: The constructed RVA-UNet model is used to perform semantic segmentation on the test images of giant freshwater prawns, and the segmentation masks of each part of the giant freshwater prawn body are output. Automated measurement: The output segmentation mask is processed to measure the dimensions of various parts of the giant freshwater prawn, specifically including: Morphological topological thinning is performed on the segmentation mask to extract the skeleton with a single pixel width; A breakpoint reconnection mechanism based on directional coherence is used to connect the fractured skeleton; For extremely fine structures, a fragment projection compensation model is used to estimate the geometric length of the missing parts; For solid structures of the cephalothorax and abdomen, a smooth midline was extracted based on constrained Voronoi diagrams and multinomial regression. The body length, carapace length, abdominal length, and chelicerae length are calculated by path integration over the central axis or skeleton.

2. The measurement method according to claim 1, characterized in that, The composite loss function used in the training process of the RVA-UNet model is a weighted combination of weighted cross-entropy loss and Dice loss, where the weight of Dice loss is λ=3.

3. The measurement method according to claim 1, characterized in that, The automated measurement also includes a geometric topology repair module, used to remove burrs, connect break points, and compensate for small fragments during the skeletonization process.

4. A segmentation and measurement system for giant freshwater prawns based on an improved U-Net, characterized in that, include: Build modules: Used to build the enhanced semantic segmentation model RVA-UNet, which adopts an encoder-decoder symmetric structure, specifically including: The encoder incorporates a residual convolution module, Res-CNN, which alleviates gradient vanishing through identity mapping and enhances the response to slender limb features. A visual transformer module (ViT) is embedded in the deepest coding unit to construct long-distance spatial dependencies using a self-attention mechanism and identify limb associations under complex postures. The bottleneck layer integrates the Spatial Convolution Pooling Pyramid Module (ASPP), which captures morphological features of different sizes through multi-scale dilated convolution. A spatial attention upsampling module is introduced into the decoding path to suppress background noise through an additive gated attention mechanism; Training and segmentation module: Used for training and segmentation: The constructed RVA-UNet model is used to perform semantic segmentation on the test images of giant freshwater prawns, and outputs the segmentation masks of each part of the giant freshwater prawn body; Measurement module: Used for automated measurement: Processes the output segmentation mask and completes the measurement of various parts of the giant freshwater prawn's body, specifically including: Morphological topological refinement is performed on the segmentation mask to extract the skeleton with a single pixel width; A breakpoint reconnection mechanism based on directional coherence is used to connect the fractured skeleton; For extremely fine structures, a fragment projection compensation model is used to estimate the geometric length of the missing parts; For solid structures of the cephalothorax and abdomen, a smooth midline was extracted based on constrained Voronoi diagrams and multinomial regression. The body length, carapace length, abdominal length, and chelicerae length are calculated by path integration over the central axis or skeleton.

Images