Non-modal segmentation method for shielding perception instance recovery of abalone fries
By constructing a biological topological constraint index system and a CM-RTransUnet model, the problem of segmenting the shading area of abalone seedlings was solved, achieving efficient and accurate non-contact measurement, adapting to different growth stages and special conditions, and improving the level of intelligent management of abalone seedling farming.
Patent Information
- Application Number
- CN202511698660.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-19
- Publication Date
- 2026-02-13
AI Technical Summary
Existing abalone seedling size measurement methods are inefficient and inconsistent. Traditional segmentation models lack the ability to recover the structure of occluded regions, and the segmentation results violate biological growth patterns. The lack of standardized datasets leads to inaccurate supervision signals for model training.
A biological topological constraint index system and morphological map library for abalone seedlings were constructed. A CM-RTransUnet dual-branch nonmodal segmentation model was built, and a cross-attention feature fusion module and a multi-scale dilation residual fusion module were embedded. The model was trained using a standardized occlusion dataset and a biological topological constraint dynamic optimization mechanism was embedded to achieve instance recovery of occluded regions.
It improves the accuracy of structural restoration in occluded areas, adapts to adhesion and occlusion problems in high-density aquaculture scenarios, ensures that the segmentation results conform to biological laws, enhances the model's generalization ability and adaptability, and achieves non-contact precision measurement.
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of intelligent aquaculture, in particular to a non-modal segmentation method for instance recovery of abalone larva shielding perception. BACKGROUND
[0002] The existing abalone larva size measurement mainly relies on manual operation, measuring shell parameters one by one through a caliper, which is not only inefficient and inconsistent in results, but also causes stress reaction and affects survival rate due to turning the larva. With the development of computer vision technology, image-based segmentation methods have been gradually applied in the field of aquaculture, but there are still significant defects:
[0003] Firstly, traditional instance segmentation models (such as MaskR-CNN) lack the ability to recover the structure of the occluded area, making it difficult to adapt to the serious adhesion and occlusion caused by the dense arrangement of abalone larvae;
[0004] Secondly, existing non-modal segmentation methods do not combine the biological growth nature of abalone larvae, and the segmentation results may appear "visually reasonable but biologically distorted" (such as abnormal spiral curvature and inconsistent tentacle number);
[0005] Thirdly, there is a lack of standardized abalone larva occlusion dataset, and manual annotation has strong subjectivity and structural distortion, resulting in inaccurate model training supervision signals and limited generalization ability. These problems seriously restrict the intelligent management level of abalone larva cultivation, and there is an urgent need for a shielding perception instance recovery method that combines visual accuracy and biological authenticity. SUMMARY
[0006] In view of the deficiencies of the prior art, the present application provides a non-modal segmentation method for instance recovery of abalone larva shielding perception, which solves the problems of low efficiency, poor shielding recovery accuracy, segmentation results violating biological growth rules, and insufficient standardization of the dataset of traditional methods.
[0007] To achieve the above purpose, the present application realizes the following technical scheme: a non-modal segmentation method for instance recovery of abalone larva shielding perception, comprising the following steps:
[0008] S1: Constructing an abalone larva biological topological constraint index system and a morphological atlas library
[0009] Collect 5-30 day-old abalone larva samples through high-resolution microscopic imaging equipment, extract the core biological topological features of the shell and soft body, and quantify them into computable constraint indexes to establish a dynamically updated "abalone larva morphological atlas library"; the biological topological constraint indexes include shell constraints and soft body constraints, wherein the shell constraints include spiral curvature, shell line spacing, and shell opening shape, and the soft body constraints include shell attachment area and tentacle distribution;
[0010] S2: Constructing a standardized abalone larva occlusion dataset
[0011] Collecting original images of abalone larvae without occlusion and performing pixel-by-pixel labeling to generate basic image-label pairs; through image enhancement and target synthesis strategies, paste the abalone larva instances in the basic images to different background images to simulate various occlusion scenarios from slight boundary contact to complete covering, generating a standardized occlusion dataset containing visible region masks and complete target masks;
[0012] S3: Building a CM-RTransUnet dual-branch non-modal segmentation model
[0013] Construct a parallel dual-branch encoder with SwinTransformer and ResNet as the core to extract global semantic information and local edge details respectively; embed a cross-attention feature fusion module (CAFF) in the encoding stage to realize deep interaction and complementarity of dual-branch features; introduce a multi-scale dilated residual fusion module (MDRF) in the skip connection to aggregate multi-range context information to repair structure breaks caused by occlusion; use a U-shaped decoding architecture and dual-output segmentation head to output visible region masks and complete target masks in parallel;
[0014] S4: Embedding a biological topological constraint dynamic optimization mechanism
[0015] Start topological consistency verification during model inference, realize dynamic matching of constraint indicators and segmentation results through real-time triggering, periodic triggering, and manual supplementary recording triggering; ensure that the segmentation results conform to the biological growth rules of abalone larvae through three guarantee mechanisms: dynamic update of morphological atlas library, configurable constraint weights, and tracing of completed results;
[0016] S5: Model training and occlusion-aware instance recovery
[0017] Based on the standardized occlusion dataset constructed in S2, use a hybrid guided training strategy to train the CM-RTransUnet model, introduce a topological consistency loss term in the loss function to balance visual accuracy and biological authenticity; input the abalone larva image with occlusion to be processed into the trained model, output the complete target mask conforming to the biological rules through non-modal segmentation and biological topological constraint verification, and realize instance recovery in the occlusion area.
[0018] Preferably, the spiral curvature in step S1 is calculated according to the formula k = \frac{|y''|}{(1+y'^2)^{3 / 2}}, the spiral curvature of 5-day-old abalone larvae is 0.8, and the spiral curvature of 30-day-old abalone larvae is 0.3, which decreases linearly with age; the daily average growth of shell stripe spacing is 0.02 mm, and the deviation of shell stripe spacing of the same larva is ≤0.005 mm; the shell aperture shape is elliptical, and the long axis / short axis ratio is fixed at 1.2:1 with a deviation of ≤0.05.
[0019] Preferably, the software part in step S1 is attached to the shell area of 60% ± 5% of the shell bottom area; the tentacles are evenly distributed at the edge of the shell opening, 8 tentacles, the adjacent tentacles are at an angle of 45° ± 2°, and the length of the tentacles is 1 / 3 ± 0.01 mm of the minor axis of the shell opening.
[0020] Preferably, the number of base images collected in step S2 is 500, the pixel resolution is 4192*2721, the outermost contour of the abalone seedling shell is outlined pixel by pixel through the Labelme labeling tool, and an independent instance ID and a JSON annotation file are generated; the standardized occlusion dataset is synthesized after image enhancement operations such as random horizontal flip, size scaling and position disturbance, and the sample number is 5000, covering different occlusion degrees.
[0021] Preferably, the cross-attention feature fusion module (CAFF) in step S3 is obtained by a bidirectional cross-attention mechanism, respectively generating a query with a Transformer branch feature, a key and a value with a CNN branch feature, and a query with a CNN branch feature, a key and a value with a Transformer branch feature, to obtain two groups of cross-attention outputs, which are fused into final features through channel splicing and 1*1 convolution compression.
[0022] Preferably, the multi-scale dilated residual fusion module (MDRF) in step S3 includes three parallel convolution branches with dilations of 1, 2 and 3 respectively, which capture multi-range context information through convolution with different expansion rates, and output enhanced skip connection features through element-wise addition, ReLU activation and residual connection.
[0023] Preferably, in step S4, the real-time trigger starts synchronously in the occlusion area prediction stage of the non-modal segmentation, and the prediction morphology and the corresponding age constraint index are compared within 10ms, and the deviation exceeds the threshold value to trigger loss correction; the periodic trigger updates the constraint parameters of the morphology atlas library according to every 5-day age cycle; the artificial supplementary recording trigger supports uploading the images and morphology data of the seedlings in special growth state, and updates the constraint rules in real time.
[0024] Preferably, in step S5, the loss function is L_{total}=L_{ce}+L_{dice}+\lambda\timesL_{topo}, wherein L_{ce} is the cross-entropy loss, L_{dice} is the Dice loss, L_{topo} is the topological consistency loss, and \lambda is the constraint weight coefficient, which is calibrated to 0.3 after 1000+ sample iterations.
[0025] The application provides a non-modal segmentation method for abalone seedling occlusion perception instance recovery.
[0026] 1、The present application quantifies the essence of abalone larvae growth into a computable constraint by constructing a biological topological constraint index system and a morphological atlas library, solves the problem that the existing segmentation method result violates the biological law, and ensures that the segmentation result has visual accuracy and biological authenticity.
[0027] 2、The present application strengthens the model's ability to capture global semantics and local details through the collaborative design of the double-branch encoder, the cross-attention feature fusion module and the multi-scale dilated residual fusion module, significantly improves the structure recovery accuracy of the occluded area, and adapts to the serious adhesion and occlusion problem in high-density farming scene.
[0028] 3、The present application provides a distortion-free visible area and complete target mask through the construction of a standardized occlusion dataset, solves the problem of strong subjectivity of manual annotation, provides accurate and stable supervision signal for model training, and improves the generalization ability of the model.
[0029] 4、The present application supports real-time verification, periodic update and manual supplement through the dynamic optimization of the topological constraint embedding mechanism, adapts to different growth stages and special growth states of abalone larvae, enhances the adaptability and reliability of the method in actual farming scene, realizes non-contact measurement, and avoids damage to the larvae by manual operation. DETAILED DESCRIPTION
[0030] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.
[0031] Embodiment one:
[0032] The present application provides a non-modal segmentation method for abalone larvae occlusion perception instance recovery, comprising the following steps:
[0033] S1: Construction of biological topological constraint index system and morphological atlas library
[0034] A high-resolution microscopic imaging device with a resolution of 4096x3072 is used to collect 5-30 day-old Haliotis discus hannai Ino larvae samples under the conditions of 18℃ water temperature and regular feed feeding, 30 samples are collected every day, a total of 750 effective samples are obtained; after pretreatment (denoising, normalization) of the sample images, the shell and soft part topological features are extracted:
[0035] Helical curvature: The shell helical curve equation y(x) is obtained through image edge detection and calculated according to the formula k=\frac{|y''|}{(1+y'^2)^{3 / 2}}. The curvature values at 5 days old (0.8), 10 days old (0.6), 15 days old (0.5), 20 days old (0.4), 25 days old (0.35), and 30 days old (0.3) are recorded to establish a linear decreasing model;
[0036] Shell pattern spacing: By calibrating the pixel-to-actual size (1 pixel = 0.001 mm), the shell pattern spacing of seedlings of different ages was measured to determine the average daily growth of 0.02 mm, and the deviation of the shell pattern spacing of the same seedling was ≤0.005 mm;
[0037] Shell aperture shape: The fitted shell aperture edge is elliptical, and the ratio of the major axis to the minor axis is calculated to ensure that the ratio is between 1.15 and 1.25;
[0038] Soft part constraint: The soft part region is extracted by image segmentation, and the area attached to the shell bottom is calculated to be 55%-65%; the number of tentacles at the edge of the shell opening is counted to be 8, the included angle between adjacent tentacles is measured to be 43°-47°, and the tentacle length is 1 / 3 ± 0.01 mm of the minor axis of the shell opening;
[0039] The above-mentioned characteristic parameters are classified and stored according to age to construct an "Abalone Seedling Morphology Atlas Library" that supports parameter query and update.
[0040] S2: Construction of Standardized Occlusion Dataset
[0041] Basic data collection: 500 original images of abalone seedlings were collected under a uniform white background, with no overlap or occlusion, and a pixel resolution of 4192×2721.
[0042] Manual annotation: Using the Labelme annotation tool, the outermost outline of each seedling's shell is drawn pixel by pixel, generating closed polygons and unique instance IDs, and outputting 500 JSON annotation files;
[0043] Data synthesis: 500 base images are randomly divided into a target image group (200 images) and a background image group (300 images). The target images are enhanced by random horizontal flipping, scaling by 0.8-1.2 times, and rotation by ±10°. The processed target instances are pasted into random positions on the background images to simulate different degrees of occlusion.
[0044] Mask generation: Based on the original annotation file, the visible region mask (modalmask) and the complete target mask (amodalmask) are automatically generated, resulting in 5000 standardized samples with different degrees of occlusion, which are divided into training set, validation set and test set in a 7:2:1 ratio.
[0045] S3: CM-RTransUnet Model Setup
[0046] Dual-branch encoder: The Transformer branch adopts the SwinTransformer architecture, which contains 4 SwinBlocks and captures global semantics through window multi-head self-attention (W-MSA) and shift window attention (SW-MSA); the CNN branch adopts the ResNet50 architecture, which contains 4 residual modules and extracts local edge and texture details.
[0047] Cross-Attention Feature Fusion Module (CAFF): After embedding into each encoder layer, it combines the CNN branch features Ci∈R... C ×H×W With Transformer branch features Ti∈R C×H×W Perform a linear transformation to construct the query (Q), key (K), and value (V). Calculate cf and tf through bidirectional cross attention. After channel concatenation and 1×1 convolution compression, output the fused feature F_i.
[0048] Multi-scale dilated residual fusion module (MDRF): Embedded in the skip connection path, it contains 3 parallel convolutional branches (dilation rate 1, 2, 3). Each branch consists of a 3×3 dilated convolution, a BatchNorm, and a 1×1 convolution. The three features are added element-wise and then activated by ReLU. They are then connected to the input feature residuals to output enhanced features.
[0049] Decoder and segmentation head: A U-shaped decoding architecture is adopted. The transposed convolution upsamples layer by layer, concatenating the features of the encoding stage with the features of the skip connection. Finally, through two parallel convolution branches, the visible region mask and the complete target mask are output respectively. The mask resolution is 4192×2721.
[0050] S4: Deployment of a Bio-topological Constraint Dynamic Optimization Mechanism
[0051] Trigger logic configuration:
[0052] Real-time triggering: During model inference, for every frame of occlusion recovery result output (time ≤ 10ms), the constraint parameters corresponding to the age in the morphology map library are automatically queried, and the deviation between the predicted morphology and the parameters is compared. If the deviation exceeds the threshold (e.g., the spiral curvature deviates from the corresponding value of the age by ±0.1), the topology consistency loss correction is triggered.
[0053] Periodic trigger: Set an update cycle of 5 days to automatically retrieve new seedling sample data from the breeding log and update the constraint parameters of the corresponding age in the atlas library;
[0054] Manual data entry trigger: Provides a visual interface that supports manual uploading of seedling images and annotation data in stress and disease states, manual adjustment of constraint parameters (such as widening the shell pattern spacing deviation to 0.01mm in stress state), and real-time updates of the atlas library;
[0055] Security mechanism configuration:
[0056] The atlas library is dynamically updated: it connects to the aquaculture environment monitoring system to automatically collect seedling samples with different water temperatures (16℃-24℃) and feed types, and adds 500+ samples every month to optimize constraint parameters;
[0057] Constraint weight configuration: The default topology consistency loss weight is 0.3. In the breeding grading scenario, it is adjusted to 0.4 (focusing on shell aperture shape and helical curvature). In the growth monitoring scenario, it is adjusted to 0.3 (focusing on shell groove spacing).
[0058] Results traceability: Record the comparison results of constraint indicators, loss correction magnitude, and final morphological parameters for each segmentation, store them in the database, and support backtracking analysis.
[0059] S5: Model Training and Instance Recovery
[0060] Training parameter settings: Training was conducted on a Windows 11 platform using Python 3.8.10 and PyTorch 1.13 frameworks. The hardware configuration was an RTX 3070 GPU, an i9-10900K CPU, and CUDA 12.2. Batch size was 16, with 200 epochs, an initial learning rate of 0.005, and the AdamW optimizer (weight decay = 3 × 10⁻⁻⁴) was used. 5 );
[0061] Loss function: L_{total}=L_{ce}+L_{dice}+0.3×L_{topo}, where L_{ce} is the cross-entropy loss, L_{dice} is the Dice loss, and L_{topo} is the weighted sum of the deviations of each topology index;
[0062] Model inference: Input the occluded seedling images from the test set into the trained model. The model first extracts features through a dual-branch encoder and fusion module to generate a preliminary complete mask. Then, after topological constraint verification and correction, it outputs the final mask that conforms to biological laws.
[0063] Size calculation: Based on the final mask, parameters such as the major axis, minor axis, and shell texture spacing of the shell opening are extracted. The actual size data of the seedling is obtained through pixel-to-actual size calibration.
[0064] Example 2:
[0065] This invention provides a non-modal segmentation method for recovering occlusion perception instances in abalone larvae, which differs from Embodiment 1 in that:
[0066] In S1, seedling samples of the new abalone variety were collected to supplement the biological topological constraint parameters of different varieties and to construct a multi-variety morphological atlas library.
[0067] In S3, the CNN branch adopts the ResNet101 architecture to enhance the ability to extract local features.
[0068] In S4, the constraint weights support custom configuration, and users can adjust the weights of each indicator according to specific breeding needs (such as disease early warning).
[0069] In S5, a transfer learning strategy is adopted, and the model is initialized based on the pre-trained SwinTransformer and ResNet101 weights, shortening the training cycle to 150 epochs.
[0070] Experimental verification shows that the method of the present invention achieves a non-modal segmentation IoU of 80.62%, Dice of 82.38%, AP@50 of 84.96%, and AR of 73.04% on the test set, with a biological morphology compliance rate of ≥95%. Compared with existing methods, the occlusion recovery accuracy and biological authenticity are significantly improved, enabling non-contact and accurate measurement of abalone seedlings.
[0071] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A non-modal segmentation method for recovering occlusion-sensing instances in abalone larvae, characterized in that, Includes the following steps: S1: Constructing a biological topological constraint index system and morphological atlas library for abalone seedlings Abalone larvae aged 5-30 days were collected using high-resolution microscopic imaging equipment. Core biological topological features of the shell and soft tissue were extracted and quantified into computable constraint indicators to establish a dynamically updated "Abalone Larvae Morphological Atlas Library". The biological topological constraint indicators include shell constraints and soft tissue constraints. Shell constraints include helical curvature, shell groove spacing, and shell aperture shape, while soft tissue constraints include the shell attachment area and tentacle distribution. S2: Construct a standardized abalone larvae occlusion dataset Unobstructed abalone seedling raw images were collected and pixel-by-pixel labeled to generate base image-label pairs. Abalone seedling instances from the base images were pasted onto different background images through image enhancement and target synthesis strategies to simulate various occlusion scenarios from slight boundary contact to complete occlusion, generating a standardized occlusion dataset containing visible area masks and complete target masks. S3: Building a CM-RTransUnet dual-branch nonmodal segmentation model A parallel dual-branch encoder based on SwinTransformer and ResNet is constructed to extract global semantic information and local edge details respectively. A cross-attention feature fusion module (CAFF) is embedded in the encoding stage to achieve deep interaction and complementarity of dual-branch features. A multi-scale dilated residual fusion module (MDRF) is introduced in the skip connections to aggregate multi-range contextual information to repair structural breaks caused by occlusion. A U-shaped decoding architecture and a dual-output segmentation head are adopted to output visible region masks and complete target masks in parallel. S4: Embedded Biotopological Constraint Dynamic Optimization Mechanism During the model inference phase, topological consistency verification is initiated. Through three logics—real-time triggering, periodic triggering, and manual supplementation triggering—dynamic matching between constraint indicators and segmentation results is achieved. Through three major guarantee mechanisms—dynamic updating of the morphological atlas library, configurable constraint weights, and traceability of supplementation results—the segmentation results are ensured to conform to the biological growth patterns of abalone seedlings. S5: Model Training and Occlusion-Aware Instance Recovery Based on the standardized occlusion dataset constructed using S2, a hybrid guided training strategy was adopted to train the CM-RTransUnet model. A topological consistency loss term was introduced into the loss function to balance visual accuracy and biological realism. The occluded abalone seedling images to be processed were input into the trained model. Through non-modal segmentation and biological topological constraint verification, a complete target mask that conforms to biological laws was output to realize instance restoration of the occluded region.
2. The non-modal segmentation method for recovering occlusion perception instances in abalone larvae according to claim 1, characterized in that... In step S1, the spiral curvature is calculated using the formula k=\frac{|y''|}{(1+y'^2)^{3 / 2}}. The spiral curvature of abalone seedlings is 0.8 at 5 days old and 0.3 at 30 days old, decreasing linearly with age. The shell pattern spacing increases by an average of 0.02 mm per day, and the deviation of the shell pattern spacing for the same seedling is ≤0.005 mm. The shell opening shape is elliptical, with a fixed major axis / minor axis ratio of 1.2:1 and a deviation of ≤0.
05.
3. The non-modal segmentation method for recovering occlusion perception instances in abalone larvae according to claim 1, characterized in that: In step S1, the soft part and the area where the shell is attached occupy 60% ± 5% of the bottom area of the shell; the tentacles are evenly distributed in 8s at the edge of the shell opening, with an angle of 45° ± 2° between adjacent tentacles, and the length of the tentacles is 1 / 3 ± 0.01 mm of the short axis of the shell opening.
4. The non-modal segmentation method for recovering occlusion perception instances in abalone seedlings according to claim 1, characterized in that: In step S2, the number of basic images acquired is 500, with a pixel resolution of 4192×2721. The outermost outline of the abalone seedling shell is delineated pixel by pixel using the Labelme annotation tool, generating an independent instance ID and a JSON annotation file. The standardized occlusion dataset is synthesized after image enhancement operations such as random horizontal flipping, size scaling, and position perturbation, with a sample size of 5000 images covering different degrees of occlusion.
5. The non-modal segmentation method for recovering occlusion perception instances in abalone seedlings according to claim 1, characterized in that: In step S3, the cross-attention feature fusion module (CAFF) uses a bidirectional cross-attention mechanism to generate queries using Transformer branch features and keys and values using CNN branch features, and vice versa, to obtain two sets of cross-attention outputs. These outputs are then fused into the final features through channel concatenation and 1×1 convolution compression.
6. The non-modal segmentation method for recovering occlusion perception instances in abalone seedlings according to claim 1, characterized in that: The multi-scale dilated residual fusion module (MDRF) in step S3 contains three parallel convolutional branches with dilation rates of 1, 2, and 3. It captures multi-range contextual information through convolution with different dilation rates, and outputs enhanced skip connection features after element-wise addition, ReLU activation, and residual connection.
7. The non-modal segmentation method for recovering occlusion perception instances in abalone larvae according to claim 1, characterized in that: In step S4, the real-time triggering is initiated synchronously during the prediction stage of the occlusion region in non-modal segmentation. The comparison between the predicted morphology and the corresponding age constraint index is completed within 10ms. When the deviation exceeds the threshold, loss correction is triggered. The periodic triggering updates the morphological map library constraint parameters every 5 days according to the age cycle. The manual recording triggering supports uploading images and morphological data of seedlings in special growth states and updates the constraint rules in real time.
8. The non-modal segmentation method for recovering occlusion perception instances in abalone seedlings according to claim 1, characterized in that: The loss function in step S5 is L_{total}=L_{ce}+L_{dice}+\lambda\timesL_{topo}, where L_{ce} is the cross-entropy loss, L_{dice} is the Dice loss, L_{topo} is the topology consistency loss, and \lambda is the constraint weight coefficient, which is calibrated to 0.3 after more than 1000 sample iterations.