Intelligent automatic generation system and method for coral species semantic sample

Abstract

The invention discloses an intelligent automatic generation system and method for coral species semantic samples, relates to the technical field of computer vision and marine ecological monitoring, and aims to efficiently generate high-precision semantic annotation samples from underwater coral images. The system adopts a hierarchical feature extraction architecture, a common model and a field-specific model CoralSCOP are collaboratively optimized, an SAM series provides a high-precision geometric segmentation region through zero sample learning, and the CoralSCOP realizes multi-granularity label distribution through parallel semantic branches. The method innovatively introduces a two-stage mask optimization mechanism: in the initial stage, a semantic segmentation model is utilized to generate a 50 + category semantic mask; in the post-processing stage, boundary details are extracted through SAM / CoralSCOP, semantic and geometric features are matched through a region-level fusion strategy, and the segmentation precision and efficiency are improved. The system provides a high-quality marking benchmark for coral reef monitoring, the expert dependence is remarkably reduced through the zero-sample and multi-granularity characteristics, and the system is suitable for marine ecological research and AI model training.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and marine ecological monitoring, and in particular to a system and method for intelligently and automatically generating semantic samples of coral species. Background Art

[0002] Coral reef ecosystem monitoring is an important part of marine environmental protection, and coral species identification technology based on computer vision has become one of the key means of ecological assessment. However, the efficiency of obtaining coral semantic samples (i.e., image data with accurate species annotations and pixel-level segmentation masks) seriously restricts the performance improvement of the recognition model. Existing technologies face the following challenges: (1) Underwater imaging quality: underwater light attenuation and scattering effects cause image color distortion (such as significant attenuation of the red light band), and corals have diverse morphologies (such as branches, leaves, and blocks). Their boundaries are blurred with seabed sediments or algae cover, resulting in a decrease in the IoU index of the general segmentation model at the coral-background interface. (2) Traditional tools (such as LabelMe and CVAT) rely on manual point-by-point boundary outlining, which is time-consuming and costly; semi-automatic tools are prone to over-segmentation in complex areas (such as coral pores), increasing the workload of post-correction. (3) Model adaptability: Using only general models (such as SAM) is difficult to fully respond to the unique biological characteristics of corals (such as skeletal growth patterns, symbiotic occlusion, etc.), and lacks optimization for the coral field, affecting segmentation accuracy.

[0003] Existing technologies such as the invention patent application with publication number CN117690033A discloses an image sample expansion method and system that integrates a large AI model and semantic segmentation technology. The method helps to minimize the problems of traditional methods of manually making samples. It integrates the most advanced image segmentation method - SegmentAnythingModel (SAM) and the existing semantic segmentation model with poor segmentation effect to achieve high-precision semantic segmentation of remote sensing objects. SAM has been trained on millions of images and more than one billion masks, and can return accurate and effective segmentation masks for remote sensing images, thereby solving the problem that the semantic segmentation model has difficulty in detecting the accurate boundaries of remote sensing segmented objects, resulting in low detection accuracy and poor recognition effect in most current semantic segmentation tasks.

[0004] In response to the above solution, the inventors of the present application have found that the above technology has at least the following technical problems: 1. In the process of generating traditional coral image semantic samples, manual labeling is still the mainstream method. Researchers need to manually identify and mark each category of objects in each coral image, which not only consumes a lot of manpower but also has extremely low efficiency. Faced with the ever-increasing coral image data, manual labeling is difficult to achieve timely processing, which seriously hinders research progress. Even with the assistance of semi-automatic labeling tools, its processing speed is still subject to the degree of manual intervention, and full process automation cannot be achieved. In addition, traditional computing architectures lack distributed parallel processing capabilities. When a single device processes high-resolution underwater images, computing resource bottlenecks are prone to occur, making it difficult to meet application scenarios with high real-time requirements, such as rapid monitoring and response to coral bleaching events.

[0005] 2. Existing general image segmentation models have significant limitations when processing underwater coral images. The underwater environment is complex, and factors such as light refraction, water turbidity, and occlusion by symbiotic organisms lead to degraded image quality and blurred target features. Traditional models lack the integration of prior knowledge of coral morphology, making it difficult to accurately distinguish between hard corals, soft corals, dead corals, and other categories. Especially in scenarios with complex coral structures and diverse symbiotic relationships, the segmentation boundary accuracy is insufficient, making misjudgments and missed detections prone to occur. For example, in areas where corals and algae coexist, general models cannot effectively identify the boundaries between the two, resulting in annotation results that cannot truly reflect the ecological status of the corals. In addition, traditional methods have difficulty achieving multi-granularity classification and cannot meet the needs of coral species research for fine annotation of genus-level or even species-level classification.

[0006] 3. Traditional coral image annotation systems are mostly single-function modules that lack modular and integrated design, making it difficult to achieve functional expansion and technological upgrades. When research needs change, such as adding new coral category annotations or changing data formats, the system is difficult to adapt quickly and often requires redevelopment or large-scale modifications. At the same time, existing technologies lack an effective mechanism for continuous model optimization and are unable to automatically update the model based on newly collected data. With the dynamic changes in the coral ecological environment, new coral morphologies and symbiotic relationships continue to emerge. If the model cannot learn new knowledge in a timely manner, the annotation accuracy will gradually decline, making it difficult to provide reliable data support for coral research in the long term. In addition, traditional systems lack user feedback and community collaboration mechanisms, making it difficult to integrate experience and data from multiple parties, which limits the collaborative innovation and sustainable development of technology. Summary of the Invention

[0007] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a system and method for intelligent and automatic generation of coral species semantic samples.

[0008] To solve the above technical problems, the present invention adopts the following technical solutions: In a first aspect, the present invention provides an intelligent automatic generation system for coral species semantic samples, comprising:

[0009] S1, coral image preprocessing module: used to acquire and preprocess underwater coral images, the preprocessing including image cleaning, standardization, size adjustment and image enhancement operations;

[0010] S2, image segmentation mode module: used to select the image segmentation mode according to task requirements, including manual assisted prompt word segmentation, global segmentation or fully automatic semantic segmentation;

[0011] S3, Image Large Model Execution Module: This module is used to call the image large model to perform segmentation operations. The general image segmentation models SAM / SAM2 are used to provide zero-shot object detection capabilities. The coral-specific model CoralScop supports multi-granular label assignment at the coral / non-coral, growth form, and genus levels.

[0012] S4, high-precision semantic annotation sample module: used to fuse the semantic labels and geometric masks output by the two models through a region-level statistical matching strategy, giving priority to retaining high-frequency semantic categories and integrating SAM high-precision boundaries to form high-precision semantic annotation samples;

[0013] S5, patch editing function providing module: used to support single or batch generation of grayscale mask images and pixel-level semantic label images, and provide patch editing function, allowing users to delete or reassign incorrect patches;

[0014] S6. Integrated platform component module: used to build an integrated platform system to implement the deployment and application of the above method;

[0015] S7, Optimize model parameters module: used to dynamically update the training set and iteratively optimize model parameters by collecting new coral image data to achieve continuous evolution of the system.

[0016] In a second aspect, the present invention provides a method for intelligently and automatically generating semantic samples of coral species, comprising:

[0017] Step 1: Pre-processing of coral images: Acquire and pre-process underwater coral images, including image cleaning, standardization, size adjustment, and image enhancement operations;

[0018] Step 2: Image segmentation: Select the image segmentation mode based on the task requirements, including manual prompt word segmentation, global segmentation, or fully automatic semantic segmentation;

[0019] Step 3: Execute the large image model: Call the large image model to perform segmentation operations. The general image segmentation models SAM / SAM2 are used to provide zero-shot object detection capabilities; the coral domain-specific model CoralScop supports multi-granular label assignment at the coral / non-coral, growth form, and genus levels.

[0020] Step 4: High-precision semantically labeled samples: The semantic labels and geometric masks output by the two models are fused through a region-level statistical matching strategy, prioritizing the retention of high-frequency semantic categories and integrating the SAM high-precision boundaries to form high-precision semantically labeled samples;

[0021] Step 5: Provide image patch editing function: Support single or batch generation of grayscale mask images and pixel-level semantic label images, and provide image patch editing function, allowing users to delete or reassign incorrect image patches;

[0022] Step 6: Build an integrated platform: Build an integrated platform system to implement the deployment and application of the above methods;

[0023] Step 7: Optimize model parameters: By collecting new coral image data, dynamically updating the training set and iteratively optimizing model parameters, the system can be continuously evolved.

[0024] The beneficial effects of the present invention are as follows: 1. The embodiments of the present invention, by innovatively introducing a large image segmentation model, have completely changed the dilemma of low efficiency and high cost in traditional manual annotation of coral images. The system adopts a modular architecture, organically combining the advantages of general models and domain-specific models. At the same time, the model optimized specifically for the coral field deeply integrates prior knowledge of coral morphology, and performs special optimization for skeletal structural characteristics, symbiotic occlusion relationships, and underwater optical degradation laws, significantly improving the accuracy of segmentation boundaries, especially in the fine distinction between hard corals, soft corals, and dead corals, solving the problem of complex scene annotation that is difficult to handle with traditional methods.

[0025] 2. At the semantic recognition level, this embodiment of the present invention integrates a rich and advanced semantic segmentation model. By deeply integrating convolutional neural networks with attention mechanisms, the system not only achieves precise object segmentation but also recognizes over 50 coral species. Extensive experimental data validates the system's superior performance. Some models achieved an accuracy rate exceeding 81% on the test set and a mean Intersection Over Union (MIOU) index exceeding 55%, demonstrating robust semantic understanding and recognition capabilities, providing high-precision data support for coral species research.

[0026] 3. The embodiment of the present invention has built a complete end-to-end closed-loop process from image input to semantic annotation output, supporting flexible single or batch processing modes. Through automated process design, the dependence on expert experience is greatly reduced, the subjective errors in manual annotation are effectively avoided, and the consistency of sample generation is significantly improved. At the same time, the system has good scalability and can be continuously optimized as the amount of data increases and the model is updated to adapt to diverse research needs. In addition, combined with the continuous evolution optimization mechanism, through incremental training set construction, multi-dimensional model evaluation, intelligent iterative optimization and user feedback-driven collaborative innovation, it ensures that the system performance continues to improve, providing a long-term, stable and efficient semantic sample generation solution for the coral research field. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 It is the overall flow chart of the present invention.

[0029] Figure 2 This is a schematic diagram of the system module connection of the present invention.

[0030] Figure 3 The present invention is a flowchart of the steps for implementing the method. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0032] The present invention is implemented as follows Figure 1 As shown in the figure, the intelligent automatic generation system of coral species semantic samples includes:

[0033] S1. Coral image preprocessing module: used to acquire and preprocess underwater coral images. The preprocessing includes image cleaning, standardization, size adjustment, and image enhancement operations.

[0034] In a specific embodiment, the S1 is implemented according to the following scheme:

[0035] A1. Collect and prepare underwater coral image data for training and inference.

[0036] A2. performing a pre-processing operation on the underwater coral image, including:

[0037] Data cleaning: Remove damaged, blurry or incomplete images to ensure input data quality;

[0038] Data normalization: Perform mean normalization and standard deviation normalization on image pixel values ​​to make the image data have zero mean and uniform variance distribution;

[0039] Image resizing: uniformly scaling images to the fixed resolution required by deep learning models to meet input dimension consistency requirements;

[0040] Data augmentation: Various image transformations, including rotation, horizontal flipping, random cropping, scaling, and brightness / contrast perturbations, were used to improve model generalization and enhance coral texture information.

[0041] S2, image segmentation mode module: used to select the image segmentation mode according to task requirements, including manual assisted prompt word segmentation, global segmentation or fully automatic semantic segmentation.

[0042] In a specific embodiment, the image macro model called in step S2 includes SAM, SAM2 and CoralScop, wherein:

[0043] The large image model is implemented based on the VisionTransformer architecture and includes models of different scales: ViT-S, ViT-B, ViT-H, and ViT-L, which are used to extract multi-scale visual features respectively. The model analyzes the global contextual relationship of the image through a self-attention mechanism and generates high-quality object segmentation masks. Among them, CoralScop, a domain-specific model designed specifically for coral images, integrates prior knowledge of coral morphology and optimizes skeletal structure, symbiotic occlusion, and underwater optical degradation characteristics, significantly improving the recognition accuracy and boundary segmentation ability of coral categories.

[0044] S3, large image model execution module: used to call the large image model to perform segmentation operations, including: the general image segmentation model SAM / SAM2 is used to provide zero-sample object detection capabilities; the coral field-specific model CoralScop supports multi-granular label assignment at the coral / non-coral, growth form and genus levels.

[0045] In a specific embodiment, in step S3, different image segmentation modes are selected according to task requirements, including:

[0046] B1. Prompt word segmentation: Users provide local guidance information by marking foreground points, background points, or drawing rectangular boxes on the interactive interface to guide the model to complete accurate segmentation;

[0047] B2. Global segmentation: Detect and extract the entire object in the entire image. This is suitable for tasks without clear target area cues.

[0048] B3. Semantic Segmentation: Use a trained semantic segmentation model, which is trained based on an automatically labeled dataset and uses an efficient convolutional neural network or attention mechanism to achieve multi-category pixel-level classification and fine segmentation.

[0049] S4, high-precision semantic annotation sample module: used to fuse the semantic labels and geometric masks output by the two types of models through a region-level statistical matching strategy, giving priority to retaining high-frequency semantic categories and fusing SAM high-precision boundaries to form high-precision semantic annotation samples.

[0050] In a specific embodiment, step S4 includes the following specific implementation methods:

[0051] The preprocessed underwater coral images are segmented at the object level using a large image segmentation model to generate a grayscale mask that can clearly identify the boundary contour between the target area and the background. Simultaneously, pixel-level semantic prediction is performed on the image based on the trained semantic segmentation model, and the semantic category label corresponding to each pixel is output. The semantic labels output by the two models are fused with the geometric masks through a region-level statistical matching strategy, prioritizing the retention of high-frequency semantic categories and integrating the SAM high-precision boundaries to form high-precision semantically labeled samples.

[0052] S5. Module for providing the patch editing function: used to support single or batch generation of grayscale mask images and pixel-level semantic label images, and provide patch editing functions, allowing users to delete or reassign erroneous patches.

[0053] In a specific embodiment, S5 is implemented according to the following scheme:

[0054] C1. Automatically assign categories to all patches in the grayscale mask generated by prompt word segmentation or global segmentation, and allow users to manually delete falsely detected or invalid patches;

[0055] C1. Post-process the semantic segmentation results to identify misclassified or unclear coral areas, and support manual reassignment and boundary correction operations to further improve the accuracy and usability of the segmentation results.

[0056] S6. Integrated platform component module: used to build an integrated platform system to implement the deployment and application of the above methods.

[0057] In a specific embodiment, S6 is implemented according to the following scheme:

[0058] Build an integrated semantic sample generation platform system that integrates image preprocessing modules, model inference engines, human-computer interaction interfaces, result visualization components, and data export interfaces; the platform system supports local deployment or cloud service deployment.

[0059] S7, Optimize model parameters module: used to dynamically update the training set and iteratively optimize model parameters by collecting new coral image data to achieve continuous evolution of the system.

[0060] In a specific embodiment, the S7 is implemented according to the following scheme:

[0061] Establish a continuous model optimization mechanism, build incremental training sets by collecting and annotating new coral image data; regularly evaluate model performance indicators, and iteratively update existing models using online learning or fine-tuning strategies.

[0062] The present invention is implemented as follows Figure 2 As shown in FIG, the method for intelligently and automatically generating semantic samples of coral species includes:

[0063] Step 1: Pre-processing of coral images: Acquire and pre-process underwater coral images, including image cleaning, standardization, size adjustment, and image enhancement operations;

[0064] Step 2: Image segmentation: Select the image segmentation mode based on the task requirements, including manual prompt word segmentation, global segmentation, or fully automatic semantic segmentation;

[0065] Step 3: Execute the large image model: Call the large image model to perform segmentation operations. The general image segmentation models SAM / SAM2 are used to provide zero-shot object detection capabilities; the coral domain-specific model CoralScop supports multi-granular label assignment at the coral / non-coral, growth form, and genus levels.

[0066] Step 4: High-precision semantically labeled samples: The semantic labels and geometric masks output by the two models are fused through a region-level statistical matching strategy, prioritizing the retention of high-frequency semantic categories and integrating the SAM high-precision boundaries to form high-precision semantically labeled samples;

[0067] Step 5: Provide image patch editing function: Support single or batch generation of grayscale mask images and pixel-level semantic label images, and provide image patch editing function, allowing users to delete or reassign incorrect image patches;

[0068] Step 6: Build an integrated platform: Build an integrated platform system to implement the deployment and application of the above methods;

[0069] Step 7: Optimize model parameters: By collecting new coral image data, dynamically updating the training set and iteratively optimizing model parameters, the system can be continuously evolved.

[0070] The above content is merely an example and explanation of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined in this specification, they should all fall within the scope of protection of the present invention.

Claims

1. An intelligent and automated system for generating semantic samples of coral species, characterized by: include: S1, coral image preprocessing module: used to acquire and preprocess underwater coral images, the preprocessing including image cleaning, standardization, size adjustment and image enhancement operations; S2, image segmentation mode module: used to select the image segmentation mode according to task requirements, including manual assisted prompt word segmentation, global segmentation or fully automatic semantic segmentation; S3, Image Large Model Execution Module: This module is used to call the image large model to perform segmentation operations. The general image segmentation models SAM / SAM2 are used to provide zero-shot object detection capabilities. The coral-specific model CoralScop supports multi-granular label assignment at the coral / non-coral, growth form, and genus levels. S4, high-precision semantic annotation sample module: used to fuse the semantic labels and geometric masks output by the two models through a region-level statistical matching strategy, giving priority to retaining high-frequency semantic categories and integrating SAM high-precision boundaries to form high-precision semantic annotation samples; S5, patch editing function providing module: used to support single or batch generation of grayscale mask images and pixel-level semantic label images, and provide patch editing function, allowing users to delete or reassign incorrect patches; S6. Integrated platform component module: used to build an integrated platform system to implement the deployment and application of the above method; S7, Optimize model parameters module: used to dynamically update the training set and iteratively optimize model parameters by collecting new coral image data to achieve continuous evolution of the system.

2. The intelligent automatic generation system for coral species semantic samples according to claim 1, characterized in that: The S1 is specifically implemented according to the following scheme: A1. Collect and prepare underwater coral image data for training and inference. A2. performing a pre-processing operation on the underwater coral image, including: Data cleaning: Remove damaged, blurry or incomplete images to ensure input data quality; Data normalization: Perform mean normalization and standard deviation normalization on image pixel values ​​to make the image data have zero mean and uniform variance distribution; Image resizing: uniformly scaling images to the fixed resolution required by deep learning models to meet input dimension consistency requirements; Data augmentation: Various image transformations, including rotation, horizontal flipping, random cropping, scaling, and brightness / contrast perturbations, are used to improve model generalization and enhance coral texture information.

3. The intelligent automatic generation system for coral species semantic samples according to claim 2, characterized in that: The image macro models called in step S2 include SAM, SAM2 and CoralScop, where: The large image model is implemented based on the VisionTransformer architecture and includes models of different scales: ViT-S, ViT-B, ViT-H, and ViT-L, which are used to extract multi-scale visual features respectively. The model analyzes the global contextual relationship of the image through a self-attention mechanism and generates high-quality object segmentation masks. Among them, CoralScop, a domain-specific model designed specifically for coral images, integrates prior knowledge of coral morphology and optimizes the skeletal structure, symbiotic occlusion, and underwater optical degradation characteristics, significantly improving the recognition accuracy and boundary segmentation ability of coral categories.

4. The intelligent automatic generation system for coral species semantic samples according to claim 3, characterized in that: In step S3, different image segmentation modes are selected according to task requirements, including: B1. Prompt word segmentation: Users provide local guidance information by marking foreground points, background points, or drawing rectangular boxes on the interactive interface to guide the model to complete accurate segmentation; B2. Global segmentation: Detect and extract the entire object in the entire image. This is suitable for tasks without clear target area cues. B3. Semantic Segmentation: Use a trained semantic segmentation model, which is trained based on an automatically labeled dataset and uses an efficient convolutional neural network or attention mechanism to achieve multi-category pixel-level classification and fine segmentation.

5. The intelligent automatic generation system for coral species semantic samples according to claim 4, characterized in that: The step S4 includes the following specific implementations: The preprocessed underwater coral images are segmented at the object level using a large image segmentation model to generate a grayscale mask that can clearly identify the boundary contour between the target area and the background. Simultaneously, pixel-level semantic prediction is performed on the image based on the trained semantic segmentation model, and the semantic category label corresponding to each pixel is output. The semantic labels output by the two models are fused with the geometric masks through a region-level statistical matching strategy, prioritizing the retention of high-frequency semantic categories and integrating the SAM high-precision boundaries to form high-precision semantically labeled samples.

6. The intelligent automatic generation system of coral species semantic samples according to claim 5, characterized in that: The S5 is specifically implemented according to the following scheme: C1. Automatically assign categories to all patches in the grayscale mask generated by prompt word segmentation or global segmentation, and allow users to manually delete falsely detected or invalid patches; C1. Post-process the semantic segmentation results to identify misclassified or unclear coral areas, and support manual reassignment and boundary correction operations to further improve the accuracy and usability of the segmentation results.

7. The intelligent automatic generation system for coral species semantic samples according to claim 6, characterized in that: The S6 is specifically implemented according to the following scheme: Build an integrated semantic sample generation platform system that integrates image preprocessing modules, model inference engines, human-computer interaction interfaces, result visualization components, and data export interfaces; the platform system supports local deployment or cloud service deployment.

8. The intelligent automatic generation system for coral species semantic samples according to claim 7, characterized in that: The S7 is specifically implemented according to the following scheme: Establish a continuous model optimization mechanism, build incremental training sets by collecting and annotating new coral image data; regularly evaluate model performance indicators, and iteratively update existing models using online learning or fine-tuning strategies.

9. A method for intelligently and automatically generating coral species semantic samples using the system for intelligently and automatically generating coral species semantic samples according to any one of claims 1 to 8, characterized in that: include: Step 1: Pre-processing of coral images: Acquire and pre-process underwater coral images, including image cleaning, standardization, size adjustment, and image enhancement operations; Step 2: Image segmentation: Select the image segmentation mode based on the task requirements, including manual prompt word segmentation, global segmentation, or fully automatic semantic segmentation; Step 3: Execute the large image model: Call the large image model to perform segmentation operations. The general image segmentation models SAM / SAM2 are used to provide zero-shot object detection capabilities; the coral domain-specific model CoralScop supports multi-granular label assignment at the coral / non-coral, growth form, and genus levels. Step 4: High-precision semantically labeled samples: The semantic labels and geometric masks output by the two models are fused through a region-level statistical matching strategy, prioritizing the retention of high-frequency semantic categories and integrating the SAM high-precision boundaries to form high-precision semantically labeled samples; Step 5: Provide image patch editing function: Support single or batch generation of grayscale mask images and pixel-level semantic label images, and provide image patch editing function, allowing users to delete or reassign incorrect image patches; Step 6: Build an integrated platform: Build an integrated platform system to implement the deployment and application of the above methods; Step 7: Optimize model parameters: By collecting new coral image data, dynamically updating the training set and iteratively optimizing model parameters, the system can be continuously evolved.

Images