A semi-supervised enteromorpha detection method and system based on mutual learning
By combining a semi-supervised framework of mutual learning and a high-confidence pseudo-label selection strategy with data augmentation, the shortcomings of thresholding and supervised classification in remote sensing image Ulva detection are addressed, achieving efficient and automated Ulva detection, improving detection accuracy and reducing labor costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NATIONAL MARINE ENVIRONMENTAL MONITORING CENTRE
- Filing Date
- 2023-05-26
- Publication Date
- 2026-04-28
AI Technical Summary
Among existing remote sensing image detection methods for seaweed, the fixed threshold method is difficult to determine the threshold, the adaptive threshold method has low accuracy when the gray-level differences are not obvious or overlap, and supervised classification requires a lot of prior information, resulting in false detection, false negative detection and high labor costs.
A semi-supervised framework based on mutual learning is adopted to design a high-confidence pseudo-label selection strategy and data augmentation. Pseudo-labels are generated by two UNet network models and weighted and fused. Combined with unlabeled data for iterative training, high-quality detection of Ulva prolifera is achieved.
It improves the accuracy and efficiency of *Ulva prolifera* detection, reduces labor costs, and automates model training and testing.
Smart Images
Figure CN116523897B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visible light remote sensing image scene recognition technology, and in particular to a semi-supervised method, system and storage medium for detecting seaweed based on cross-learning. Background Technology
[0002] Currently, the main methods for detecting Ulva prolifera in remote sensing images include thresholding, classification, and radiative transfer inversion. Thresholding separates Ulva prolifera from the background by setting thresholds for each band or vegetation index, including fixed thresholding and adaptive thresholding, and is the most widely used method.
[0003] Due to spectral differences between remote sensing images, it is difficult to determine a fixed threshold. Although the adaptive thresholding method can adjust the threshold, it is difficult to obtain accurate results when there are no obvious gray-level differences in the images or when the gray-level value ranges overlap significantly. In addition, the thresholding method only considers gray-level information and does not consider the spatial information of the image, which can lead to false positives and false negatives in the detection of Ulva prolifera in remote sensing images.
[0004] Neural network-based classification has become a major research method in recent years. Its powerful feature learning capabilities ensure a basic level of accuracy in *Ulva prolifera* detection, with supervised classification being the primary approach. However, supervised classification requires a large amount of accurate prior information about *Ulva prolifera*, placing high demands on the interpreters. Summary of the Invention
[0005] To address the problem of insufficient labeled samples in remote sensing images, this invention provides a semi-supervised method and system for detecting Ulva prolifera based on cross-learning. Based on the semi-supervised framework of cross-learning, a high-confidence pseudo-label selection strategy is designed in conjunction with data augmentation to fully leverage the value of unlabeled data and achieve high-quality detection of Ulva prolifera in remote sensing images.
[0006] Therefore, the present invention provides the following technical solution:
[0007] This invention provides a semi-supervised method for detecting *Ulva prolifera* based on cross-learning, comprising:
[0008] Preprocessing and labeling of remote sensing images;
[0009] Construct a seaweed detection model based on the convolutional neural network model UNet;
[0010] The mutual learning model for detecting *Ulva prolifera* includes: pre-training two UNet network models with different initialization parameters using labeled data; using the two UNet network models to predict and generate corresponding pseudo-label sets for the unlabeled dataset, and then weighting and fusing the corresponding pseudo-labels to generate a corresponding pseudo-label set; then, according to the pseudo-label selection strategy, selecting high-confidence pseudo-label data from the fused pseudo-label set, and retraining the network model by mixing it with labeled data and data augmentation operations, while updating the labeled and unlabeled datasets during this process, iterating until the unlabeled dataset is empty;
[0011] Two trained Ulva prolifera detection models were used to test Ulva prolifera data, and the prediction results were weighted and fused.
[0012] Further data preprocessing includes: radiometric calibration of satellite imagery, atmospheric correction, geometric correction, NDVI extraction, and segmentation of the distribution area of Ulva prolifera on the imagery.
[0013] Furthermore, the annotations include:
[0014] Separate land and sea areas based on latitude and longitude;
[0015] Manual labeling combined with thresholding methods was used to assist in the labeling of *Ulva prolifera*.
[0016] Furthermore, pseudo-label selection strategies include:
[0017] For the pseudo-labels generated by the two UNet network models, cross-entropy loss is used as a quantitative metric to determine the similarity between the two pseudo-labels.
[0018] Select the top-ranked pseudo-labels based on similarity as high-confidence pseudo-labels.
[0019] Furthermore, it also includes combining the selected high-confidence pseudo-labels with the corresponding input images to expand the training set.
[0020] Furthermore, the data augmentation operation includes performing corresponding geometric transformations on both the unlabeled image and its high-confidence pseudo-labeled image to augment the data.
[0021] Furthermore, geometric transformations include random horizontal, vertical, rotational, and cropping operations with a certain probability.
[0022] This invention also provides a semi-supervised Ulva prolifera detection system based on cross-learning, comprising:
[0023] The preprocessing module is used to preprocess and label remote sensing image data;
[0024] The model building module is used to build a seaweed detection model based on the UNet convolutional neural network model;
[0025] The training module is used to learn the seaweed detection model built by the mutual learning model building module. It includes: pre-training two UNet network models with different initialization parameters using labeled data; using the two UNet network models to predict the unlabeled dataset to generate corresponding pseudo-label sets, and then weighting and fusing the corresponding pseudo-labels to generate corresponding pseudo-label sets; then, according to the pseudo-label selection strategy, selecting high-confidence pseudo-label data from the fused pseudo-label set, and retraining the network model by mixing it with labeled data and data augmentation operations, while updating the labeled and unlabeled datasets during this process, iterating until the unlabeled dataset is empty.
[0026] The detection module is used to test the Ulva prolifera detection models trained by the training module on Ulva prolifera data and to perform weighted fusion of the prediction results.
[0027] Advantages and positive effects of this invention: The semi-supervised method and system for detecting *Ulva prolifera* based on mutual learning provided by this invention efficiently utilizes unlabeled data by designing a high-confidence pseudo-label strategy combined with data augmentation during the mutual learning network modeling process, thereby achieving *Ulva prolifera* detection in remote sensing images. Simultaneously, the model training and data testing processes are automated, reducing labor costs and improving efficiency. Attached Figure Description
[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1 This is a flowchart of a semi-supervised method for detecting Ulva prolifera based on cross-guided learning, as described in an embodiment of the present invention.
[0030] Figure 2 This is a schematic diagram of the images and labels in an embodiment of the present invention;
[0031] Figure 3 This is a schematic diagram of the UNet network model in an embodiment of the present invention;
[0032] Figure 4 This is a flowchart of the semi-supervised learning process based on UNet cross-learning in an embodiment of the present invention;
[0033] Figure 5 This is a schematic diagram of the data expansion process in an embodiment of the present invention;
[0034] Figure 6This is a comparison chart of detection results in an embodiment of the present invention. Detailed Implementation
[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0036] This invention was implemented on Windows 10 and using the PyTorch deep learning framework, with experiments conducted using an NVIDIA 3080 GPU. First, MODIS remote sensing data was downloaded, and image preprocessing and label creation were completed. Then, within the PyTorch framework, a network model was built, trained using both labeled and unlabeled data for cross-learning, to obtain optimal parameters. Finally, the two trained cross-learning network models were used for prediction, and the test results were weighted and fused.
[0037] like Figure 1 As shown, this embodiment of the invention provides a semi-supervised method for detecting *Ulva prolifera* based on cross-learning. The method specifically includes:
[0038] S1: Data preprocessing and labeling.
[0039] MODIS remote sensing image data preprocessing includes radiometric calibration, atmospheric correction, geometric correction, NDVI extraction, and segmentation of Ulva prolifera distribution areas. For label creation, Ulva prolifera detection can be considered a binary classification problem. First, the Python package `global-land-mask` is used to separate land and sea areas based on latitude and longitude. Then, manual labeling combined with a thresholding method is used to assist in Ulva prolifera labeling. The image interpretation results are finally converted to grayscale images, with the background color channel set to 0 and the Ulva prolifera area set to 1. After data processing, training sets, test sets, and an unlabeled dataset are obtained. Examples of images and labels are shown below. Figure 2 As shown.
[0040] S2: Construct a convolutional neural network model.
[0041] The method for detecting *Ulva prolifera* in this embodiment of the invention is based on the UNet convolutional neural network model. The network structure of the UNet convolutional neural network model is as follows: Figure 3As shown, an encoder-decoder structure is used. The left side downsamples for feature extraction, while the right side upsamples to restore resolution. Each blue box corresponds to a multi-channel feature map, with the number of channels located at the top of the box: 64, 128, 256, 512, and 1024. The width-height dimensions are located at the bottom left corner of the box. Gray boxes represent copied and connected feature maps. The arrows in the bottom right corner, from top to bottom, represent 3×3 convolution, activation function, copy connection, 2×2 max pooling, 2×2 upsampling, and 1×1 convolution operations, respectively. This network structure consists of a contraction path and an expansion path. The contraction path is used to obtain contextual information, while the expansion path is used for precise localization, and the two paths are symmetrical.
[0042] All hyperparameters in the network model are set to default values, with a learning rate of 0.01 and a batch size of 2. Adam is used as the optimizer for adaptive learning rate updates of each parameter, and cross-entropy is used as the loss function. To prevent overfitting due to too many training epochs, the initial training epochs for each model are set to 100 epochs. Furthermore, network model training is stopped prematurely if the loss function does not decrease within 10 iterations.
[0043] S3: Mutual learning network model.
[0044] This invention, within a semi-supervised learning framework, utilizes UNet networks with identical structures but different initializations for mutual learning and iterative training. The semi-supervised process based on UNet mutual learning is as follows: Figure 4 As shown. The specific process is as follows:
[0045] S31, using labeled data Two network models, UNet1 and UNet2, with the same structure but different initializations are pre-trained.
[0046] S32. Use the pre-trained UNet1 and UNet2 network models to process the unlabeled dataset. Perform predictions and generate two pseudo-label sets. and D u The two pseudo-labels corresponding to each unlabeled data point are weighted and fused to obtain the result. As a set of high-confidence pseudo-labels to be selected.
[0047] S33. Based on the pseudo-label selection strategy, select a set of pseudo-label data with high confidence from the pseudo-label set. At the same time, combine data augmentation operations with labeled data to retrain the network model. In this process, update the labeled dataset and the unlabeled dataset.
[0048] Repeat S31 to S33 until the unlabeled data is empty.
[0049] Two key steps in the mutual learning network model are: first, a high-confidence pseudo-label selection strategy, and second, data augmentation.
[0050] High-Confidence Pseudo-Label Selection Strategy: During the network model iteration process, when generating pseudo-labels for unlabeled data, the quality of the generated pseudo-labels varies. This invention designs a high-confidence pseudo-label selection strategy, ranking the generated pseudo-labels according to their quality. Regarding the selection of "high-quality" pseudo-labels, this invention proposes selecting reliable pseudo-labels based on the overall similarity of the pseudo-labels, using the cross-entropy loss in formula (2). i Quantitative measurement. Two sufficiently accurate network models producing predictions (pseudo-labels) for the same remote sensing image input will have very similar results, indicating high-quality pseudo-labels. Conversely, if the predictions differ significantly, at least one of the pseudo-labels is substandard. In other words, for pseudo-labels generated by two different network models, the more similar they are, the closer the pseudo-label is to the true prediction, and thus the higher its "quality." This invention uses a weighted fusion of pseudo-labels generated by different network models for unlabeled data to create a set of pseudo-labels to be selected.
[0051]
[0052] In formula (1), the cross-entropy L CC (p,q) can be used as a loss function in deep learning, where p represents the distribution of the true labels and q represents the distribution of the predicted labels of the trained model. The cross-entropy loss function can be used to measure the similarity between p and q. In formula (2) These are pseudo-labels generated by two different network models for the i-th unlabeled data, and the Loss is... i P represents the cross-entropy loss between the two. In formula (3), P i It is a pseudo-label pair and The weighted average. According to the cross-entropy loss formula: and The more similar, the lower the loss. i The smaller P is i The higher the confidence level, the better.
[0053] Based on this, the quantification metric Loss can be adjusted during the network model iteration process. i Sort the pseudo-labels from smallest to largest and select the highest confidence pseudo-labels P. i Then, the selected pseudo-labels are combined with the corresponding input images to expand the training set.
[0054] Data Augmentation: In this invention, high-confidence pseudo-labels are first selected from the weighted fused high-quality *Ulva prolifera* image pseudo-labels using a high-confidence pseudo-label selection strategy within a semi-supervised mutual learning framework. Then, the unlabeled images and their high-confidence pseudo-labeled images are simultaneously subjected to corresponding geometric transformations for data augmentation, gradually expanding the training set. Geometric transformations involve random horizontal, vertical, rotational, and cropping operations with a certain probability p. The overall process for expanding the dataset is as follows: Figure 5 Enriching limited images of seaweed through data augmentation allows the network to learn additional features and mitigates oversimulation of noise.
[0055] S4: Network model testing.
[0056] Once the UNet1 and UNet2 network models have been iteratively trained on limited labeled and unlabeled data, they are then used to test the seaweed data and the prediction results are weighted and fused.
[0057] In the above embodiments, during the mutual learning and iterative training of the network model, a high-confidence pseudo-label strategy combined with data augmentation is designed to efficiently utilize unlabeled data to achieve the detection of *Ulva prolifera* in remote sensing images. Meanwhile, the model training and data testing processes are automated, reducing labor costs and improving efficiency.
[0058] To verify the effectiveness of the *Ulva prolifera* detection method in the above embodiments, experimental verification was conducted in this invention. Comparative experiments were performed using thresholding, DeepLabv3+, SegNet, UNet, and UNet++ network models. The experimental method UNet(1 / k+DA, k=2, 4, 8) represents selecting high-confidence pseudo-labels and performing DA operations on a 1 / k proportion of unlabeled samples in the UNet network model based on cross-referencing learning. All hyperparameters in the network model were set to default values, with a learning rate of 0.01 and a batch size of 2. Adam was used as the optimizer for adaptive learning rate updates of each parameter, and cross-entropy was used as the loss function. To prevent overfitting due to too many training epochs, the initial training epochs for each model were set to 100 epochs. Furthermore, network model training was stopped early when the loss function did not decrease within 10 iterations. All deep learning network models were implemented on the PyTorch platform, and experiments were conducted using an NVIDIA 3080 GPU. Thirty medium-resolution MODIS remote sensing satellite images from 2018 to 2020, representing the Ulva prolifera outbreak in June and July, were used. After preprocessing and annotation of this dataset, 380 labeled MODIS image pairs (512×512 pixels each) and 700 unlabeled images were obtained. A pre-trained network model was trained using 200 labeled images; experimental validation was performed using 180 MODIS Ulva prolifera images. The detection results are as follows: Figure 6As shown in the figure, recall, precision, and F1 score were used as evaluation metrics to quantitatively evaluate the algorithm performance. Compared with the thresholding method and fully supervised network models, the UNet semi-supervised network model with mutual learning exhibits higher detection accuracy. Among them, the network model learning features under the condition of selecting high-confidence pseudo-labels at a ratio of 1 / 8 and combining data augmentation performed best, with a recall rate of 72.19%, precision of 94.79%, and F1 score of 81.96%. Experiments demonstrate that the method of this invention can effectively improve the detection accuracy of Ulva prolifera areas in remote sensing images, providing technical support for disaster monitoring during large-scale Ulva prolifera outbreaks.
[0059] In another embodiment, a semi-supervised Ulva prolifera detection system based on cross-learning is provided, comprising:
[0060] The preprocessing module is used to preprocess and label remote sensing image data;
[0061] The model building module is used to build a seaweed detection model based on the UNet convolutional neural network model;
[0062] The training module, used for mutual learning of the *Ulva prolifera* detection model constructed by the model building module, includes: pre-training two UNet network models with different initialization parameters using labeled data; using the two UNet network models to predict the unlabeled dataset to generate corresponding pseudo-label sets, and then weighting and fusing the corresponding pseudo-labels to generate corresponding pseudo-label sets; then, according to the pseudo-label selection strategy, selecting high-confidence pseudo-label data from the fused pseudo-label set, and simultaneously combining data augmentation operations with labeled data to retrain the network model, updating the labeled and unlabeled datasets during this process, iterating until the unlabeled dataset is empty;
[0063] The detection module is used to test the Ulva prolifera data using the two Ulva prolifera detection models trained by the training module, and to perform weighted fusion of the prediction results.
[0064] For the semi-supervised seaweed detection system based on mutual learning in this embodiment of the invention, since it corresponds to the semi-supervised seaweed detection method based on mutual learning in the above embodiment, the description is relatively simple. For related similarities, please refer to the description of the semi-supervised seaweed detection method based on mutual learning in the above embodiment, which will not be described in detail here.
[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A semi-supervised method for detecting *Ulva prolifera* based on mutual learning, characterized in that, include: Preprocessing and labeling of remote sensing images; Construct a seaweed detection model based on the convolutional neural network model UNet; The mutual learning of the *Ulva prolifera* detection model includes: pre-training two UNet network models with different initialization parameters using labeled data; using the two UNet network models to predict and generate corresponding pseudo-label sets for the unlabeled dataset, and then weighting and fusing the corresponding pseudo-labels to generate corresponding pseudo-label sets; then, according to the pseudo-label selection strategy, selecting high-confidence pseudo-label data from the fused pseudo-label set, and simultaneously retraining the network model by combining data augmentation operations with labeled data, while updating the labeled and unlabeled datasets during this process, iterating until the unlabeled dataset is empty; wherein, the pseudo-label selection strategy includes: using cross-entropy loss as a quantitative indicator to determine the similarity between the two pseudo-labels generated by the two UNet network models; selecting a preset number of pseudo-labels with the highest similarity ranking as high-confidence pseudo-labels; Two trained Ulva prolifera detection models were used to test Ulva prolifera data, and the prediction results were weighted and fused.
2. The semi-supervised method for detecting *Ulva prolifera* based on cross-learning as described in claim 1, characterized in that, The preprocessing includes: radiometric calibration of satellite imagery, atmospheric correction, geometric correction, NDVI extraction, and segmentation of the distribution area of Ulva prolifera on the imagery.
3. The semi-supervised method for detecting *Ulva prolifera* based on cross-learning as described in claim 1, characterized in that... The annotations include: Separate land and sea areas based on latitude and longitude; Manual labeling combined with thresholding methods was used to assist in the labeling of *Ulva prolifera*.
4. The semi-supervised method for detecting *Ulva prolifera* based on cross-learning as described in claim 1, characterized in that, Also includes: The selected high-confidence pseudo-labels are combined with the corresponding input images to expand the training set.
5. The semi-supervised method for detecting *Ulva prolifera* based on cross-learning according to claim 1, characterized in that, The data augmentation operation includes performing corresponding geometric transformations on both the unlabeled image and its high-confidence pseudo-labeled image to augment the data.
6. The semi-supervised method for detecting *Ulva prolifera* based on cross-learning according to claim 5, characterized in that, The geometric transformations include random horizontal, vertical, rotational, and cropping operations with a certain probability.
7. A semi-supervised system for detecting *Ulva prolifera* based on mutual learning, characterized in that, include: The preprocessing module is used to preprocess and label remote sensing image data; The model building module is used to build a seaweed detection model based on the UNet convolutional neural network model; The training module, used for mutual learning of the *Ulva prolifera* detection model constructed by the model building module, includes: pre-training two UNet network models with different initialization parameters using labeled data; using the two UNet network models to predict and generate corresponding pseudo-label sets for the unlabeled dataset, and then weighting and fusing the corresponding pseudo-labels to generate corresponding pseudo-label sets; then, according to the pseudo-label selection strategy, selecting high-confidence pseudo-label data from the fused pseudo-label set, and simultaneously retraining the network model by combining data augmentation operations with labeled data, while updating the labeled and unlabeled datasets during this process, iterating until the unlabeled dataset is empty; wherein, the pseudo-label selection strategy includes: using cross-entropy loss as a quantitative indicator to determine the similarity between the two pseudo-labels generated by the two UNet network models; selecting a preset number of pseudo-labels with the highest similarity ranking as high-confidence pseudo-labels; The detection module is used to test the Ulva prolifera data using the two Ulva prolifera detection models trained by the training module, and to perform weighted fusion of the prediction results.