A method and system for generating and repairing multi-modal features of a coral reef remote sensing image
The multimodal feature generation and restoration system solves the problem of coral reef remote sensing image restoration in complex marine environments, achieving high-precision, stable and adaptable coral reef remote sensing image restoration, and constructing an intelligent and automated restoration system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA AERO GEOPHYSICAL SURVEY & REMOTE SENSING CENT FOR LAND & RESOURCES
- Filing Date
- 2025-08-29
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies struggle to accurately restore remote sensing images of coral reefs in complex marine environments. Traditional methods cannot recover complex texture structures and spectral features, while deep learning methods lack the ability to integrate multimodal data. Furthermore, GAN training is difficult, the generated images are unstable, and there is a lack of environmental adaptability.
A multimodal feature generation and repair system is constructed. Data is acquired through multiple sources of sensors, features are extracted using convolutional neural networks, pixel-level reconstruction is performed using generative adversarial networks, and a data feedback mechanism is introduced to optimize the model. Combined with a lightweight semantic analysis module and self-supervised training, dynamic feature weighting and adaptive adjustment are achieved.
It significantly improves the adaptability and accuracy of coral reef remote sensing image restoration, accurately restores the morphology and spectral characteristics of coral reefs in complex marine environments, improves the stability and consistency of generated images, and enhances the robustness and generalization ability of the model.
Smart Images
Figure CN121120447B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing image restoration technology, and in particular to a method and system for generating and restoring multimodal features of coral reef remote sensing images. Background Technology
[0002] In complex marine environments, remote sensing images of coral reefs often suffer from texture loss due to factors such as cloud cover and water reflection, necessitating image restoration.
[0003] For image restoration, existing methods include traditional image restoration methods, image restoration methods based on convolutional neural networks (CNN), image restoration methods using generative adversarial networks (GAN) and restoration methods using multi-source remote sensing data fusion. Among them, (1) traditional image restoration methods mainly include mathematical interpolation algorithms (such as bilinear interpolation and cubic spline interpolation) and image filtering methods (such as Gaussian filtering and median filtering), which are mostly used in early remote sensing image restoration, image noise processing, and image edge smoothing. (2) Image restoration methods based on convolutional neural networks (CNN) use CNN models to learn image structure information from large-scale remote sensing images and fill in missing areas. Common architectures include variants such as U-Net and ResNet, which are used in remote sensing scenarios such as water body detection, vegetation classification, and urban built-up area identification. (3) The method of applying generative adversarial networks (GAN) to image restoration uses the image generation capabilities of GAN to try to restore missing areas in remote sensing images and achieve a more natural restoration effect. It has achieved preliminary results in urban remote sensing images and surface classification scenarios. (4) Some studies on multi-source remote sensing data fusion and restoration methods have begun to attempt to integrate optical images, radar, lidar and other data to enhance remote sensing images, which are mostly applied to three-dimensional reconstruction of the earth's surface, disaster assessment and dynamic changes of water bodies.
[0004] Due to the complex ecological structure of coral reefs, traditional image interpolation or single-modal deep learning methods are difficult to meet the requirements. That is, existing methods have some defects in dealing with remote sensing data in marine environments. (1) Traditional interpolation and filtering methods in traditional image restoration methods have limitations. They are based only on local pixel relationships and cannot restore complex texture structures and real spectral features. Their performance is extremely poor in complex marine environments such as turbid seawater and light changes, and they cannot accurately restore the ecological information of coral reefs. (2) Image restoration methods based on convolutional neural networks (CNN) have limited expressive power. Although they can extract local features, they have weak integration capabilities for different modal data and easily ignore the interaction of time, space and multi-source information. When facing areas with severe missing information or complex textures (such as coral reefs), the restoration effect is unstable. (3) Generative adversarial networks (GANs) applied to image restoration have training bottlenecks. They have high requirements for labeled data. It is difficult to obtain remote sensing data of coral reefs and the labeling cost is high, which restricts the training quality of the model. GAN models are prone to mode collapse during training, and the generated images lack detail and authenticity. Especially in the restoration of ecologically sensitive areas, it is difficult to meet the accuracy requirements. (4) Existing methods lack dynamic weight adjustment and feedback mechanisms. Existing methods rarely have the ability to dynamically weight features and generate self-feedback training. They lack the mechanism to improve environmental adaptability, resulting in poor model generalization ability and difficulty in adapting to coral reef remote sensing images collected in different sea areas, seasons and equipment.
[0005] To address the challenge of restoring remote sensing images of coral reefs, there is an urgent need to develop a new method for restoring remote sensing images, improve the quality of remote sensing image restoration in complex marine environments, and achieve intelligent, automated, and high-precision remote sensing image restoration of coral reefs. Summary of the Invention
[0006] To address the aforementioned issues, this invention provides a method and system for generating and restoring multimodal features of coral reef remote sensing images. It constructs a complete closed-loop system that encompasses multi-source information scheduling, bidirectional structural semantic expression, pixel-level guided reconstruction, and semantic feedback verification. This system not only supplements image information but also rationally reconstructs ecological information, providing a high-quality data foundation for subsequent classification, monitoring, and protection efforts.
[0007] To achieve the above objectives, the present invention provides a method for generating and restoring multimodal features of coral reef remote sensing images, comprising:
[0008] Coral reef image data acquired from multiple source sensors is preprocessed.
[0009] For the preprocessed coral reef image data, convolutional neural networks are used to extract image morphological features and spectral features, and a feature database is established.
[0010] Based on the feature combination network, multi-source features are weighted and fused to generate a comprehensive feature representation and store it in the feature database.
[0011] The images of the coral reef to be repaired are matched with the feature database to generate guidance parameters;
[0012] The guidance parameters are input into a generative adversarial network to perform pixel-level reconstruction of the missing areas in the image of the coral reef to be repaired.
[0013] As a further improvement of the present invention, the repaired image is fed back to the training system for optimized training of the convolutional neural network.
[0014] As a further improvement of the present invention, the repair area obtained by reconstructing the missing area is further verified by a lightweight semantic analysis module to determine the category, boundary, and spatial location of the repair area. If it is found that the ecological category of the repair area is inconsistent with that of the coral reef image to be repaired, the guidance parameters are regenerated and the missing area is reconstructed.
[0015] As a further improvement of the present invention, the coral reef image data includes current optical images, SAR radar images, LiDAR point cloud data, and historical images of the same period and the same area.
[0016] After denoising, correction, radiometric consistency verification, and geometric registration, the coral reef image data is divided into ecological semantic slices.
[0017] As a further improvement of the present invention, image morphological features and spectral features are extracted using a convolutional neural network, including:
[0018] By extracting edges, texture direction, and contour morphology through convolutional neural networks, the morphological features of coral growth structures are obtained. Regional ecological labels and depth location information are integrated, and a lightweight transformer is used to capture ecological coherence and category boundaries in the context. The results form a structural feature map, a semantic embedding map, and a modal response matrix.
[0019] As a further improvement of the present invention, the image of the coral reef to be repaired is matched with the feature database to generate guiding parameters; including:
[0020] Extract image fragments from the feature database that are similar to the missing areas in the image of the coral reef to be repaired;
[0021] Calculate the geometric structure differences, spectral response differences, and ecological label differences between the image of the coral reef to be restored and the image fragment, construct a three-dimensional difference tensor, and generate a weight field with a controllable spatial distribution based on the three-dimensional difference tensor;
[0022] The weights of the weight field are dynamically adjusted guiding fields in the image domain, i.e., guiding parameters.
[0023] As a further improvement of the present invention, the guiding parameters are input into a generative adversarial network to perform pixel-level reconstruction of the missing regions in the coral reef image to be repaired; including:
[0024] The image of the coral reef to be repaired, the guiding parameters, and the semantic embedding labels are input into a generative adversarial network. In the generative adversarial network, the generator uses a local-global attention fusion mechanism to coordinate the transition between the missing texture and the surrounding image structure. The discriminator judges whether there are abrupt structural changes and spectral variations between the generated region and the context, thus obtaining the repaired coral reef image.
[0025] As a further improvement of the present invention, a multimodal feature generation and repair method is trained by a self-supervised training set, wherein high-quality restoration samples are added to the self-supervised training set.
[0026] As a further improvement of the present invention, the domain adaptive module in the convolutional neural network integrates environmental variables, including sea area, season and sensors, to construct a domain conditional embedding vector, enabling the parameters of the convolutional neural network to self-adjust.
[0027] The present invention also provides a multimodal feature generation and restoration system for coral reef remote sensing images, including: a multi-source remote sensing data preprocessing module, an image feature classification and database construction module, a feature combination and dynamic weighting module, a feature matching and guidance parameter generation module, a restoration module based on a generative neural network, and a data backflow training enhancement module.
[0028] The multi-source remote sensing data preprocessing module is used for:
[0029] Coral reef image data acquired from multiple source sensors is preprocessed.
[0030] The image feature classification and database construction module is used for:
[0031] For the preprocessed coral reef image data, convolutional neural networks are used to extract image morphological features and spectral features, and a feature database is established.
[0032] The feature combination and dynamic weighting module is used for:
[0033] Based on the feature combination network, multi-source features are weighted and fused to generate a comprehensive feature representation and store it in the feature database.
[0034] The feature matching and guidance parameter generation module is used for:
[0035] The images of the coral reef to be repaired are matched with the feature database to generate guidance parameters;
[0036] The repair module based on the generative neural network is used for:
[0037] The guidance parameters are input into a generative adversarial network to perform pixel-level reconstruction of the missing areas in the image of the coral reef to be repaired.
[0038] The data backflow training enhancement module is used for:
[0039] The repaired image is fed back to the training system for optimized training of the convolutional neural network.
[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0041] Compared to traditional interpolation, filtering methods, and conventional CNN or GAN image restoration methods, this invention significantly improves adaptability and restoration quality in complex marine environments. By fusing multimodal remote sensing data and dynamically weighting and combining various features, the system can more accurately restore the morphology and spectral characteristics of coral reefs, avoiding the blurring, misjudgment, and edge fragmentation problems caused by limited information in traditional methods. Furthermore, the introduced prior guidance parameter mechanism significantly enhances the controllability and stability of GAN-generated images.
[0042] This invention establishes a data feedback mechanism, which enhances the robustness and self-learning ability of the model under different environmental and equipment conditions, and is suitable for automated processing and ecological assessment of large-scale remote sensing images.
[0043] This invention addresses the problem of insufficient modal information in single-source remote sensing data by employing a feature extraction and dynamic weighted fusion mechanism based on multi-source remote sensing data. It guides a generative neural network (GAN) for controlled restoration by generating guidance parameters through feature matching, significantly improving the realism and consistency of the restored images. A data feedback enhancement mechanism is designed to allow the restoration results to feed back into the model training process, continuously improving the network model's generalization ability and robustness in varying environments. Based on these advancements, this invention constructs an intelligent, automated, and high-precision coral reef remote sensing image restoration system, demonstrating significant technological advancement and application value.
[0044] This invention constructs a complete closed-loop system, encompassing multi-source information scheduling, bidirectional structural semantic expression, pixel-level guided reconstruction, and semantic feedback verification. It can not only supplement image information but also reasonably reconstruct ecological information, providing a high-quality data foundation for subsequent classification, monitoring, and protection work. Attached Figure Description
[0045] Figure 1 This is a flowchart of a method for generating and restoring multimodal features of coral reef remote sensing images, as disclosed in one embodiment of the present invention.
[0046] Figure 2This is a schematic diagram of a coral reef remote sensing image multimodal feature generation and restoration system disclosed in one embodiment of the present invention. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0048] The present invention will now be described in further detail with reference to the accompanying drawings:
[0049] like Figure 1 As shown, the present invention provides a method for generating and restoring multimodal features of coral reef remote sensing images, comprising:
[0050] S1. Acquire coral reef image data based on multi-source sensors and perform preprocessing;
[0051] in,
[0052] Coral reef image data are acquired from multiple sources of sensors, including optics, radar, and sonar.
[0053] Based on factors such as ocean optical transparency, water depth distribution, and historical occlusion probability, multimodal data is intelligently utilized. The multimodal coral reef image data includes: current optical images, SAR radar images, LiDAR point cloud data, and historical images of the same period and the same area.
[0054] After denoising, correction, radiometric consistency verification, and geometric registration of the multimodal coral reef image data, it is divided into several ecological semantic slices, which serve as the smallest analysis unit for subsequent modules.
[0055] S2. For the preprocessed coral reef image data, use a convolutional neural network (CNN) to extract image morphological features and spectral features, and establish a feature database.
[0056] in,
[0057] In convolutional neural networks, the domain adaptation module integrates environmental variables, including sea area, season, and sensor data, to construct a domain-conditional embedding vector, enabling the convolutional neural network parameters to self-adjust.
[0058] Furthermore,
[0059] By extracting edges, texture direction, and contour morphology through convolutional neural networks, the morphological features of coral growth structures are obtained. Regional ecological labels and depth location information are integrated, and a lightweight transformer is used to capture ecological coherence and category boundaries in the context. The results form a structural feature map, a semantic embedding map, and a modal response matrix.
[0060] Specifically,
[0061] A dual-channel structure-semantic feature extractor is designed to simultaneously capture spatial structural information and ecological semantic features. Specifically, the spatial structural channel utilizes dilated convolution to extract edges, texture orientation, and contour morphology, thereby reflecting the spatial morphological characteristics of coral growth structures; the semantic channel integrates regional ecological labels and depth location information, using a lightweight transformer to capture ecological coherence and category boundaries within the context. The results form a structural feature map S, a semantic embedding map M, and a modal response matrix T.
[0062] S3. Based on feature combination networks (such as MLP or Transformer), multi-source features are weighted and fused to generate a comprehensive feature representation and store it in the feature database;
[0063] S4. Match the images of the coral reef to be repaired with the feature database to generate guidance parameters;
[0064] in,
[0065] Extract image fragments from the feature database that are similar to the missing areas in the image of the coral reef to be repaired;
[0066] The geometric structure difference, spectral response difference, and ecological label difference between the image of the coral reef to be restored and the image fragment are calculated. A three-dimensional difference tensor is constructed, and a weight field with a controllable spatial distribution is generated based on the three-dimensional difference tensor.
[0067] The weights of the weight field are dynamically adjusted guiding fields in the image domain, i.e., guiding parameters.
[0068] Specifically,
[0069] The Structural Difference Guided Weight Field Construction Module (SDWF-G) extracts image fragments similar to the missing region from the feature database. A three-dimensional difference tensor is constructed by calculating geometric structural differences (ΔG), spectral response differences (ΔS), and ecolabel differences (ΔE). This generates a set of weight fields W(x,y) with a controllable spatial distribution. These weights are not static parameters but dynamically adjusted guided fields in the image domain, allowing for customized control over the generation during the inpainting phase.
[0070] S5. Input the guidance parameters into the Generative Adversarial Network (GAN) to perform pixel-level reconstruction of the missing areas in the image of the coral reef to be repaired.
[0071] in,
[0072] The image of the coral reef to be repaired, the guiding parameters, and the semantic embedding labels are input into the generative adversarial network. In the generative adversarial network, the generator uses a local-global attention fusion mechanism to coordinate the transition between the missing texture and the surrounding image structure. The discriminator judges whether there are abrupt structural changes and spectral changes between the generated region and the context, and obtains the repaired coral reef image.
[0073] Furthermore,
[0074] For the restored image, the restored area obtained by reconstructing the missing area is further verified by a lightweight semantic analysis module to determine the category, boundary, and spatial location of the restored area. If the ecological category of the restored area is found to be inconsistent with that of the coral reef image to be restored, the guidance parameters are regenerated and the missing area is reconstructed.
[0075] Specifically,
[0076] A Conditional Generation and Contextual Structure Constraint Network (CSG-Net) is constructed. The CSG-Net model is fed with the image region to be repaired, a differential guided vector (RE-GUIDE), and semantic embedding labels. The generator employs a local-global attention fusion strategy to achieve a harmonious transition between the missing texture and the surrounding image structure. The discriminator goes beyond true / false classification, judging whether there are abrupt structural and spectral changes between the generated region and its context, thereby improving the naturalness of the repair.
[0077] S6. The repaired image is fed back to the training system for optimized training of the convolutional neural network, continuously improving the network performance and generalization ability.
[0078] in,
[0079] A semantic feedback mechanism for repair results was set up. By designing a lightweight semantic analysis module, this mechanism established a micro-loop of "supervision-feedback-update" to improve the adaptability of the model.
[0080] In this invention, a multimodal feature generation and restoration method is trained using a self-supervised training set. High-quality restored samples are added to the self-supervised training set to enhance feature extraction and generation models. Furthermore, a domain adaptation module integrates environmental variables such as sea area, season, and sensor data to construct a domain-conditional embedding vector, enabling parameter self-adjustment and improving the model's regional generalization ability and long-term stability.
[0081] like Figure 2As shown, the present invention also provides a multimodal feature generation and restoration system for coral reef remote sensing images, including: a multi-source remote sensing data preprocessing module, an image feature classification and database construction module, a feature combination and dynamic weighting module, a feature matching and guidance parameter generation module, a restoration module based on a generative neural network, and a data backflow training enhancement module.
[0082] The multi-source remote sensing data preprocessing module is used for:
[0083] Coral reef image data acquired from multiple source sensors is preprocessed.
[0084] The image feature classification and database construction module is used for:
[0085] For the preprocessed coral reef image data, convolutional neural networks are used to extract image morphological features and spectral features, and a feature database is established.
[0086] The feature combination and dynamic weighting module is used for:
[0087] Based on the feature combination network, multi-source features are weighted and fused to generate a comprehensive feature representation and store it in the feature database;
[0088] The feature matching and guidance parameter generation module is used for:
[0089] The images of the coral reefs to be restored are matched with a feature database to generate guiding parameters;
[0090] The repair module based on generative neural networks is used for:
[0091] The guiding parameters are input into the generative adversarial network to perform pixel-level reconstruction of the missing areas in the image of the coral reef to be repaired.
[0092] The data backflow training enhancement module is used for:
[0093] The repaired image is fed back to the training system for optimized training of the convolutional neural network.
[0094] Example:
[0095] The process of pixel-level restoration of coral reef remote sensing images based on this invention includes:
[0096] Step 1: Load and slice data from multiple sources
[0097] First, an image occlusion detection algorithm (a hybrid vegetation index and cloud detection model) is used to generate occlusion masks to determine the boundaries of missing areas. Then, the system loads current SAR data, historical optical images, and terrain information (laser point clouds or DEMs) from a remote sensing data warehouse to unify the projected coordinate system and resolution. All images are divided into 256x256 pixel spatial tiles and assigned ecolabels.
[0098] Step 2: Extract structural and semantic features
[0099] For each slice, DSR-Net simultaneously extracts structural features (edge tensor, curvature map, and texture orientation distribution) and semantic embeddings (ecological category and semantic boundary based on a benthic classification model). The model generates a joint feature map and its modal response matrix for each slice, representing the distribution of feature weights across different modalities.
[0100] Step 3: Construct the differential guided weight field
[0101] The SDWF-G module uses corresponding regions of historical images as reference samples to calculate geometric, spectral, and semantic differential quantization tensors, forming a structural difference field W(x,y). This field is used to guide texture construction and region morphology restoration during the generation stage.
[0102] Step 4: Condition Generation and Structural Constraints
[0103] The missing region image W(x,y) and the ecological embedding vector are input into the CSG-Net model. The generator constructs the missing texture using local convolutional layers and a cross-scale attention fusion module. The discriminator evaluates the spatial structure and semantic consistency scores of the generated regions in real time. Poor-quality reconstructed images are discarded and included in the reconstruction process.
[0104] Step 5: Semantic Consistency Verification and Reverse Annotation
[0105] The system triggers the LSA-Net module to perform semantic inference on the reconstructed image and compares it with the predefined ecolabels of the original image. If a semantic boundary type mismatch or distortion is detected, the system immediately reverse-labels the region, generates a new recovery vector, and regenerates the image to ensure a high degree of consistency between the recovered region and the overall semantic field.
[0106] Step 6: Feedback Learning and Generalization Improvement
[0107] Finally, high-confidence samples obtained from the inpainting techniques are registered in a self-supervised sample set. The system uses this information to improve DSR-Net and CSG-Net, thereby training a model version adapted to the current domain. The system then automatically updates this strategy using domain embedding vectors to improve the speed and accuracy of future image inpainting.
[0108] Advantages of this invention:
[0109] Compared to traditional interpolation, filtering methods, and conventional CNN or GAN image restoration methods, this invention significantly improves adaptability and restoration quality in complex marine environments. By fusing multimodal remote sensing data and dynamically weighting and combining various features, the system can more accurately restore the morphology and spectral characteristics of coral reefs, avoiding the blurring, misjudgment, and edge fragmentation problems caused by limited information in traditional methods. Furthermore, the introduced prior guidance parameter mechanism significantly enhances the controllability and stability of GAN-generated images.
[0110] This invention establishes a data feedback mechanism, which enhances the robustness and self-learning ability of the model under different environmental and equipment conditions, and is suitable for automated processing and ecological assessment of large-scale remote sensing images.
[0111] This invention addresses the problem of insufficient modal information in single-source remote sensing data by employing a feature extraction and dynamic weighted fusion mechanism based on multi-source remote sensing data. It guides a generative neural network (GAN) for controlled restoration by generating guidance parameters through feature matching, significantly improving the realism and consistency of the restored images. A data feedback enhancement mechanism is designed to allow the restoration results to feed back into the model training process, continuously improving the network model's generalization ability and robustness in varying environments. Based on these advancements, this invention constructs an intelligent, automated, and high-precision coral reef remote sensing image restoration system, demonstrating significant technological advancement and application value.
[0112] This invention constructs a complete closed-loop system, encompassing multi-source information scheduling, bidirectional structural semantic expression, pixel-level guided reconstruction, and semantic feedback verification. It can not only supplement image information but also reasonably reconstruct ecological information, providing a high-quality data foundation for subsequent classification, monitoring, and protection work.
[0113] This invention introduces a dual-channel network for asynchronous extraction, clearly distinguishing spatial structure from ecological semantics. Based on a region guidance mechanism using a three-dimensional structural difference field, it achieves pixel-level on-demand repair control. The combination of a condition generator and a context consistency discriminator significantly improves the texture continuity and ecological credibility of the image. A semantic feedback mechanism and a reverse injection strategy are introduced to establish an iterative quality assurance chain. The system has a completely closed-loop structure, supporting model self-evolution and cross-domain transfer.
[0114] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for generating and restoring multimodal features of coral reef remote sensing images, characterized in that, include: Coral reef image data acquired from multiple source sensors is preprocessed. For the preprocessed coral reef image data, a convolutional neural network was used to extract the morphological features and ecological semantic features of coral growth structure through a dual-channel structure-semantic feature extractor, and a feature database was established. Based on the feature combination network, multi-source features are weighted and fused to generate a comprehensive feature representation and store it in the feature database. The images of the coral reefs to be restored are matched with the feature database, and the differences in geometric structure, spectral response and ecological labels are calculated to construct a three-dimensional difference tensor. A weight field is generated based on the three-dimensional difference tensor as a guiding parameter. The image of the coral reef to be repaired, the guiding parameters, and the semantic embedding labels are input into a generative adversarial network to reconstruct the missing regions of the image of the coral reef to be repaired at the pixel level, thereby obtaining the repaired coral reef image. For the reconstructed restoration area, a lightweight semantic analysis module is used to perform secondary verification of the category, boundary, and spatial location of the restoration area. If the ecological category of the restoration area is found to be inconsistent with that of the coral reef image to be restored, guidance parameters are regenerated and the missing area is reconstructed.
2. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that: The repaired image is fed back to the training system for optimized training of the convolutional neural network.
3. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that: The coral reef image data includes current optical images, SAR radar images, LiDAR point cloud data, and historical images of the same period and the same area. After denoising, correction, radiometric consistency verification, and geometric registration, the coral reef image data is divided into ecological semantic slices.
4. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that: A convolutional neural network was used to extract morphological and ecological semantic features of coral growth structures through a dual-channel structure-semantic feature extractor, including: By extracting edges, texture direction, and contour morphology through convolutional neural networks, the morphological features of coral growth structures are obtained. Regional ecological labels and depth location information are integrated, and a lightweight transformer is used to capture ecological coherence and category boundaries in the context. The results form a structural feature map, a semantic embedding map, and a modal response matrix.
5. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that: The images of the coral reef to be restored are matched with the feature database to generate guiding parameters, including: Extract image fragments from the feature database that are similar to the missing areas in the image of the coral reef to be repaired; The weights of the weight field are dynamically adjusted guiding fields in the image domain, i.e., guiding parameters.
6. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that, The image of the coral reef to be repaired, guiding parameters, and semantic embedding labels are input into a generative adversarial network (GAN) to perform pixel-level reconstruction of the missing regions in the image of the coral reef to be repaired, resulting in a repaired coral reef image; including: In the generative adversarial network, the generator uses a local-global attention fusion mechanism to coordinate the transition between missing textures and surrounding image structures, while the discriminator determines whether there are abrupt structural changes and spectral variations between the generated region and the context, thus obtaining the repaired coral reef image.
7. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that: A multimodal feature generation and repair method is trained using a self-supervised training set, in which high-quality restored samples are added.
8. The method for generating and restoring multimodal features of coral reef remote sensing images according to claim 1, characterized in that: The domain adaptation module in the convolutional neural network integrates environmental variables, including sea area, season, and sensors, to construct a domain conditional embedding vector, enabling the convolutional neural network parameters to self-adjust.
9. A system for generating and restoring multimodal features of coral reef remote sensing images, used to implement the method for generating and restoring multimodal features of coral reef remote sensing images as described in any one of claims 1 to 8, characterized in that: include: The system includes a multi-source remote sensing data preprocessing module, an image feature classification and database construction module, a feature combination and dynamic weighting module, a feature matching and guidance parameter generation module, a generative neural network-based repair module, and a data backflow training enhancement module. The multi-source remote sensing data preprocessing module is used for: Coral reef image data acquired from multiple source sensors is preprocessed. The image feature classification and database construction module is used for: For the preprocessed coral reef image data, a convolutional neural network was used to extract the morphological features and ecological semantic features of coral growth structure through a dual-channel structure-semantic feature extractor, and a feature database was established. The feature combination and dynamic weighting module is used for: Based on the feature combination network, multi-source features are weighted and fused to generate a comprehensive feature representation and store it in the feature database. The feature matching and guidance parameter generation module is used for: The images of the coral reefs to be restored are matched with the feature database, and the differences in geometric structure, spectral response and ecological labels are calculated to construct a three-dimensional difference tensor. A weight field is generated based on the three-dimensional difference tensor as a guiding parameter. The repair module based on the generative neural network is used for: The image of the coral reef to be repaired, the guiding parameters, and the semantic embedding labels are input into a generative adversarial network to reconstruct the missing regions of the image of the coral reef to be repaired at the pixel level, thereby obtaining the repaired coral reef image. For the reconstructed restoration area, a lightweight semantic analysis module is used to perform secondary verification of the category, boundary, and spatial location of the restoration area. If it is found that the ecological category of the restoration area is inconsistent with that of the coral reef image to be restored, the guidance parameters are regenerated and the missing area is reconstructed. The data backflow training enhancement module is used for: The repaired image is fed back to the training system for optimized training of the convolutional neural network.