A Deep Learning-Based Multispectral Reflectance Method for Marine Mucus Detection

Abstract

This application discloses a deep learning-based multispectral reflectance method for detecting marine slime, belonging to the field of marine observation. It addresses the problems of insufficient spectral data and class imbalance in existing technologies. The technical solution of this application uses multi-band spectral reflectance data as model input, combining near-infrared and short-wave infrared bands to accurately distinguish marine slime from other floating objects; it utilizes the quality control band of MODIS data for cloud masking to ensure that the model input data is not affected by cloud cover; and it introduces a novel loss function combining BCE and F1 scores to make the model pay more attention to sub-class samples during training. This application's solution effectively solves the class imbalance problem in existing technologies and improves the model's detection accuracy and sensitivity to sub-class samples.

Description

Technical Field

[0001] This invention relates to a deep learning-based multispectral reflectance method for detecting marine mucus, belonging to the field of marine observation. Background Technology

[0002] Current technologies for detecting marine seasnot primarily rely on remote sensing data and machine learning algorithms to analyze the spectral characteristics of floating debris on the ocean surface. In recent years, technologies based on multispectral and hyperspectral satellite imagery have been widely applied. By using satellite imagery (such as MODIS, Sentinel-2, and AVHRR), researchers can monitor the spread and drift of marine seasnot. These images provide important monitoring tools for large-scale, long-term seasnot events. For example, using multispectral data, researchers utilize RGB image input for detection and classification. These images typically use data from the red, green, and blue bands to build machine learning models for identifying marine seasnot. Using these bands, seasnot can be distinguished to some extent from other floating debris such as algae and plastic waste.

[0003] Despite the progress made in the detection of marine slime, existing technologies still have many limitations.

[0004] Limitations of spectral data: Existing technologies are mostly based on RGB images, but RGB data has a limited spectral coverage, making it difficult to capture the detailed characteristics of marine mucus in different wavelengths. For example, in the near-infrared region, the reflectivity of marine mucus differs significantly from that of seawater or other floating matter (such as plankton and algae), but RGB data cannot effectively capture these differences. This makes existing detection models perform poorly in distinguishing mucus from other floating matter, resulting in inaccurate detection results.

[0005] Class imbalance is a significant issue: marine slime covers a relatively small area compared to seawater, resulting in a low percentage of slime samples in datasets. Most existing machine learning models are trained using the binary cross-entropy (BCE) loss function; however, this function is less sensitive to small-class samples, causing the model to tend to learn seawater features and ignore small-class slime detection. Class imbalance severely impacts the detection performance of current techniques, especially in detecting small areas of slime against a large seawater background.

[0006] Atmospheric conditions pose challenges to existing detection technologies under cloud cover or complex atmospheric conditions. Existing remote sensing technologies suffer from reflectivity overlap when distinguishing between clouds and slime. Slime and clouds exhibit very similar reflectance characteristics in certain spectral bands, making effective differentiation difficult and affecting the accuracy of detection results.

[0007] Limitations of real-time and spatiotemporal dynamic monitoring: Existing technologies mainly rely on satellite data such as Sentinel, which are acquired at a low frequency, making it difficult to achieve real-time dynamic monitoring of marine mucus. Furthermore, existing technologies lack monitoring methods based on time-series data, failing to effectively capture the diffusion, drift, and spatiotemporal changes of marine mucus. This results in delayed monitoring results and an inability to respond promptly to the development of mucus events. Summary of the Invention

[0008] To address the problems existing in the prior art, this application proposes a deep learning-based multispectral reflectance method for detecting marine slime. By using multi-band spectral reflectance data as the input to the model and combining it with a novel loss function, the method can effectively solve the class imbalance problem in the prior art and improve the detection accuracy and sensitivity of the model to samples with smaller class characteristics.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is a multispectral reflectance marine mucus detection method based on deep learning, comprising the following steps:

[0010] Step 1) Acquire medium-resolution multi-band spectral remote sensing data;

[0011] Use the resampling processing snapshot view provided by WorldView as a reference to generate labels;

[0012] Step 2) Perform cloud masking using the quality control bands in MODIS satellite data;

[0013] Step 3) Standardize the data for all bands;

[0014] Step 4) Manually annotate the marine slime areas in the WorldView quick view image using the annotation tool;

[0015] The labeled information is paired with the corresponding MODIS multispectral data to extract the spectral reflectance data of the mucus region in the MODIS data.

[0016] Step 5) Construct a deep neural network model based on spectral reflectance data.

[0017] The optimized deep learning-based multispectral reflectance marine mucus detection method described above, in step 5), includes the following steps for constructing a deep neural network model:

[0018] 501) Construction of Model Input Parameters

[0019] Using the spectral reflectance data of the mucus region in step 4) as the input layer parameters of the model, a preliminary CNN neural network model is constructed.

[0020] 502) Division of training and test sets

[0021] The paired data from step 4) are randomly assigned to a certain percentage of the training set and the test set;

[0022] 503) Separate label

[0023] After the segmentation is completed, the water depth column is extracted to become the label; the label serves as the target during model training and the output when the model is applied after training.

[0024] 504) Define the model structure

[0025] The model structure is built on the U-Net architecture.

[0026] In the optimized deep learning-based multispectral reflectance marine slime detection method described above, when acquiring medium-resolution multiband spectral remote sensing data in step 1), MOD09GA / MOYD09GA data is acquired using the MODIS sensor carried on the Terra and Aqua satellites; MOD09GA / MOYD09GA data contains surface reflectance information at a resolution of 500m.

[0027] Using the resampling quick view provided by WorldView as a reference, and after finely annotating the marine mucus areas, labels are generated.

[0028] The optimized deep learning-based multispectral reflectance marine mucus detection method described above includes the following specific steps in step 2):

[0029] The quality control band is stored in binary data form, where the 10th bit being 1 indicates that the pixel is covered by clouds. By reading the quality control band data, pixels affected by clouds can be filtered out.

[0030] In step 2), the WorldView data is downsampled to the same 500-meter resolution as the MODIS data using bilinear interpolation or nearest neighbor interpolation.

[0031] In the optimized deep learning-based multispectral reflectance marine mucus detection method described above, in step 3),

[0032] The raw input data is standardized; the raw input data includes MOD09GA / MOYD09GA data.

[0033] During standardization, the z-score standardization method is used. The formula for the z-score standardization method is: Where x is the surface reflectance value in the MOD09GA / MOYD09GA data before conversion, x* is the surface reflectance value in the MOD09GA / MOYD09GA data after conversion, and μ is the average value of the original data; The standard deviation of the original data;

[0034] The processed data conforms to a standard normal distribution, i.e., the mean is 0 and the standard deviation is 1.

[0035] In the optimized deep learning-based multispectral reflectance marine mucus detection method described above, in step 4),

[0036] The marine slime regions in the WorldView quick view image were manually annotated using the annotation tool; based on the manual annotation results, the corresponding label data was generated.

[0037] During manual labeling, the labeled information is paired with the corresponding MODIS multispectral data, and seven sets of spectral reflectance values ​​of the mucus region in the MODIS data are extracted.

[0038] The seven groups of spectral reflectance values ​​are: red light band with wavelengths of 620-670 nm, near-infrared band with wavelengths of 841-876 nm, blue light band with wavelengths of 459-479 nm, green light band with wavelengths of 545-565 nm, short-wave infrared band 1 with wavelengths of 1230-1250 nm, short-wave infrared band 2 with wavelengths of 1628-1652 nm, and short-wave infrared band 3 with wavelengths of 2105-2155 nm.

[0039] The extracted data is converted into NumPy array format;

[0040] LabelMe is one of the labeling tools.

[0041] In the optimized deep learning-based multispectral reflectance marine mucus detection method described above, in step 501), different parameter combinations are selected as input parameters for the DNN neural network model.

[0042] In step 502), the spectral reflectance data of all matched mucus regions in step 4) are randomly allocated as training and test sets according to a percentage. The training set is used to train the CNN model, and during the training of the CNN model, 20% of the training set is further divided as a validation set.

[0043] The test set is used as a separate part to test the recognition performance of the model after training;

[0044] The optimized deep learning-based multispectral reflectance marine slime detection method described above employs a novel loss function combining binary cross-entropy and F1 score in step 504.

[0045] The novel loss function formula combining binary cross-entropy (BCE) and F1 score is as follows: ;

[0046] Here, BCE is the binary cross-entropy that calculates the difference between predicted and true values; F1 score is the weight adjustment parameter, used by the CNN model to adjust the focus on less detected slime regions. Since slime detection is a binary classification problem, binary cross-entropy is typically used as the loss function. However, compared to ocean backgrounds, slime detection is more of an imbalanced classification problem because slime patches may only occupy a small portion of the entire image. Therefore, using only the "accuracy" metric cannot comprehensively evaluate the model. α is the balancing factor between F1 and BCE. The optimal model is found by changing α.

[0047] The beneficial effects of this application are as follows:

[0048] Existing technologies typically use RGB images for training and detection, but this approach only utilizes information from three spectral bands, failing to fully capture the spectral characteristics of marine mucus. The proposed solution, however, employs multi-band spectral reflectance data as model input, particularly combining near-infrared and short-wave infrared bands, enabling more precise differentiation between marine mucus and other floating debris (such as algae and plastic waste). This approach allows the model to acquire richer feature information about marine mucus across different spectral bands, resulting in more accurate detection results.

[0049] The technical solution of this application utilizes the quality control band of MODIS data for cloud masking to ensure that the model input data is not affected by clouds. Compared with existing technologies that rely on manual annotation or simple threshold screening methods, automated cloud masking can more accurately remove pixels interfered with by clouds, thereby improving the accuracy and reliability of detection results.

[0050] The technical solution of this application introduces a novel loss function that combines BCE and F1 scores, which makes the model pay more attention to small class samples during training, improves the detection sensitivity of marine mucus, and significantly improves the detection effect, especially when dealing with small marine mucus areas. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the steps of the technical solution in this application. Detailed Implementation

[0052] The technical features of the present invention will be further illustrated below with reference to specific embodiments.

[0053] This application provides a deep learning-based multispectral reflectance method for detecting marine slime. Addressing the challenge of optical detection of marine slime due to the increasing frequency of marine slime events worldwide caused by eutrophication and global climate change, this method combines multi-band spectral reflectance data with remote sensing data and a CNN model to improve the accuracy of quantitative slime information extraction and achieve intelligent remote sensing monitoring of marine slime.

[0054] The deep learning-based multispectral reflectance marine mucus detection method of this application includes the following steps:

[0055] Step 1)

[0056] Acquire medium-resolution multi-band spectral remote sensing data. The primary method used is the MODIS sensor aboard the Terra and Aqua satellites, which provides rich multispectral reflectance data covering multiple bands from visible to near-infrared. The MOD09GA / MOYD09GA data contains surface reflectance information at a resolution of 500m, suitable for large-scale ocean monitoring.

[0057] Using the resampling quick view provided by WorldView as a reference, it can be used to finely annotate marine slime regions and generate labels in combination with MODIS data.

[0058] Step 2)

[0059] Cloud masking is performed using the quality control bands in MODIS satellite data. The quality control bands are stored in binary data format, where a 1 in the 10th bit indicates that the pixel is covered by a cloud. By reading the quality control band data, pixels affected by clouds are filtered out, ensuring the quality of the input data.

[0060] Because the MOD09GA / MOYD09GA data and the WorldView data have different resolutions, they need to be resampled to ensure data alignment consistency. Using bilinear interpolation or nearest-neighbor interpolation methods, the high-resolution WorldView data is downsampled to the same 500-meter resolution as the MODIS data. This ensures computational efficiency and consistency in subsequent processing.

[0061] Step 3)

[0062] To avoid the impact of scale differences between different bands on model training performance, data from all bands needs to be standardized. When the input parameters differ significantly in value, directly using the raw values ​​for analysis will emphasize the role of parameters with higher values ​​in the model, while relatively weakening the role of parameters with lower values.

[0063] Therefore, to ensure the reliability of the results, the original input data needs to be standardized. Here, z-score standardization (zero-mean normalization) is used. Where x and x* are the values ​​before and after the transformation, respectively, and μ is... The mean and standard deviation of the original data are not specified. The processed data conforms to a standard normal distribution, i.e., the mean is 0 and the standard deviation is 1, thereby improving the convergence speed and stability of the model.

[0064] Step 4)

[0065] Marine slime regions in WorldView quick-view imagery were manually labeled using annotation tools such as LabelMe. Based on these manually labeled high-resolution imagery results, corresponding label data was generated. Marine slime distributions often exhibit complex morphologies; therefore, high-precision manual annotation ensures the quality of the training data.

[0066] The labeled information was paired with the corresponding MODIS multispectral data to extract seven sets of spectral reflectance values ​​for the slime region from the MODIS data. The seven sets of spectral reflectance values ​​are distributed across the following wavelengths: red (620-670 nm), near-infrared (841-876 nm), blue (459-479 nm), green (545-565 nm), short-wave infrared (1230-1250 nm), short-wave infrared (1628-1652 nm), and short-wave infrared (2105-2155 nm). These extracted data will be converted into NumPy array format for later model input.

[0067] Step 5)

[0068] A deep neural network (DNN) model is constructed based on spectral reflectance data.

[0069] The model's input layer consists of multi-band spectral reflectance values, including reflectance information from the visible and near-infrared bands. By combining data from these different bands, the model can capture the complex spectral characteristics of mucus, improving detection accuracy.

[0070] Among them, the marine mucus monitoring DNN model based on a novel loss function of binary cross-entropy (BCE) and F1 score was loaded into Python software, as follows: 501) Construction of model input parameters

[0071] MOD / MYD 09GA data involves many spectral bands during imaging, and monitoring marine mucus is relatively difficult.

[0072] In this application, the spectral reflectance data of the mucus region in step 4) is used as the input layer parameters of the model to initially construct a DNN neural network model. Different parameter combinations are selected as the input parameters of the DNN neural network model.

[0073] 502) The training and test sets are divided by randomly allocating the spectral reflectance data of all matched mucus regions in step 4) into training and test sets according to a percentage. The training set is used to train the CNN model, and during the CNN model training process, 20% of the training set is further divided into a validation set. The test set is used as an independent part to test the recognition performance of the model after training.

[0074] 503) After the label separation is completed, the water depth column is extracted as the label (also known as the ground truth), which is the target during model training and the output when the model is applied after training (i.e. the identified slime pixels).

[0075] 504) Define the model structure

[0076] This invention is based on the classic U-Net architecture, with customized improvements. U-Net performs excellently in image segmentation, effectively fusing features from different levels. For the binary classification problem of marine slime detection, the model input is multi-band spectral reflectance, and the output is the segmentation result of the marine slime. To address the class imbalance problem in marine slime detection, a novel loss function combining binary cross-entropy (BCE) and F1 score is adopted. The formula for this loss function is as follows: Among them, BCE is used to calculate the difference between the predicted value and the true value, while the F1 score is adjusted by weights to make the model pay more attention to the less detected mucus regions, thereby improving the model's detection sensitivity for small class samples.

[0077] After defining the model structure, the model's accuracy is evaluated to assess its performance and display the model's recognition results during training.

[0078] In this application, the following metrics are used to evaluate the accuracy of the model: (Accuracy): The proportion of correctly predicted ocean slime pixels out of the total pixels. (Accuracy): The proportion of pixels that the model correctly predicts as algae that are actually algae. A high accuracy value means that the model produces fewer false positives. (Recall): The proportion of actual ocean slime pixels that the model correctly predicts as ocean slime.

[0079] A high recall rate means that the model misses fewer marine slime pixels. (F1 score): The harmonic mean of precision and recall, used to achieve a balance between precision and recall. ( Intersection over Union (IoU): The ratio of the intersection to the union of the predicted ocean slime pixels and the actual ocean slime pixels. The higher the value, the more accurate the model is in predicting marine slime regions.

[0080] (3)

[0081] (4)

[0082] (5)

[0083] (6)

[0084] (7)

[0085] in, For a real example, For a true negative example, As a false positive example, This is a false negative.

[0086] Through multiple experiments, the model performance under different band combinations was compared to further verify the effectiveness of the spectral reflectance data.

[0087] This invention effectively improves the detection accuracy of marine mucus through this method, solves the problems of insufficient spectral data and class imbalance in the prior art, and achieves efficient monitoring of marine mucus.

[0088] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A deep learning-based multispectral reflectance method for detecting marine mucus, characterized in that: Includes the following steps: Step 1) Acquire medium-resolution multi-band spectral remote sensing data; Use the resampling processing snapshot view provided by WorldView as a reference to generate labels; Step 2) Perform cloud masking using the quality control bands in MODIS satellite data; Step 3) Standardize the data for all bands; Step 4) Manually annotate the marine slime areas in the WorldView quick view image using the annotation tool; The labeled information is paired with the corresponding MODIS multispectral data to extract the spectral reflectance data of the mucus region in the MODIS data. Step 5) Construct a deep neural network model based on spectral reflectance data.

2. The deep learning-based multispectral reflectance marine mucus detection method according to claim 1, characterized in that: In step 5), the steps for constructing the deep neural network model include: 501) Construction of Model Input Parameters Using the spectral reflectance data of the mucus region in step 4) as the input layer parameters of the model, a preliminary DNN neural network model is constructed. 502) Division of training and test sets The paired data from step 4) are randomly assigned to a certain percentage of the training set and the test set; 503) Separate label After the segmentation is completed, the water depth column is extracted to become the label; the label serves as the target during model training and the output when the model is applied after training. 504) Define the model structure The model structure is built on the U-Net architecture.

3. The deep learning-based multispectral reflectance marine mucus detection method according to claim 2, characterized in that: In step 1), when acquiring medium-resolution multi-band spectral remote sensing data, the MODIS sensor onboard the Terra and Aqua satellites is used to acquire MOD09GA / MOYD09GA data; the MOD09GA / MOYD09GA data contains surface reflectance information at a resolution of 500m. Using the resampling quick view provided by WorldView as a reference, and after finely annotating the marine mucus areas, labels are generated.

4. The deep learning-based multispectral reflectance marine mucus detection method according to claim 3, characterized in that: The specific process in step 2) includes: The quality control bands are stored in binary data. If the 10th bit of the binary data of one pixel is 1, it means that the pixel is covered by clouds. By reading the quality control band data, pixels affected by clouds can be filtered out. In step 2), the WorldView data is downsampled to the same 500-meter resolution as the MODIS data using bilinear interpolation or nearest neighbor interpolation.

5. The deep learning-based multispectral reflectance marine mucus detection method according to claim 4, characterized in that: In step 3), The raw input data is standardized; the raw input data includes MOD09GA / MOYD09GA data. During standardization, the z-score standardization method is used. The formula for the z-score standardization method is: Where x is the surface reflectance value in the MOD09GA / MOYD09GA data before conversion, x* is the surface reflectance value in the MOD09GA / MOYD09GA data after conversion, and μ is the average value of the original data; The standard deviation of the original data; The processed data conforms to a standard normal distribution, i.e., the mean is 0 and the standard deviation is 1.

6. The deep learning-based multispectral reflectance marine mucus detection method according to claim 5, characterized in that: In step 4), Manually annotate the marine slime areas in the WorldView quick view image using the annotation tool; Based on the results of manual annotation, corresponding label data is generated; During manual labeling, the labeled information is paired with the corresponding MODIS multispectral data, and seven sets of spectral reflectance values ​​of the mucus region in the MODIS data are extracted. The seven groups of spectral reflectance values ​​are: red light band with wavelengths of 620-670 nm, near-infrared band with wavelengths of 841-876 nm, blue light band with wavelengths of 459-479 nm, green light band with wavelengths of 545-565 nm, short-wave infrared band 1 with wavelengths of 1230-1250 nm, short-wave infrared band 2 with wavelengths of 1628-1652 nm, and short-wave infrared band 3 with wavelengths of 2105-2155 nm. The extracted data is converted into NumPy array format; LabelMe is one of the labeling tools.

7. The deep learning-based multispectral reflectance marine mucus detection method according to claim 6, characterized in that: In step 501), different combinations of parameters are selected as input parameters for the DNN neural network model; In step 502), the spectral reflectance data of all matched mucus regions in step 4) are randomly allocated as training and test sets according to a percentage. The training set is used to train the CNN model, and during the training of the CNN model, 20% of the training set is further divided as a validation set. The test set is used as a separate part to test the recognition performance of the model after training.

8. The deep learning-based multispectral reflectance marine mucus detection method according to claim 7, characterized in that: In step 504), a novel loss function combining binary cross-entropy (BCE) and F1 score is adopted; the formula for the novel loss function combining binary cross-entropy (BCE) and F1 score is as follows: ;in, BCE is the binary cross-entropy that calculates the difference between the predicted and the true values; F1 score is the weight adjustment parameter, and the CNN model uses F1 to adjust the mucus regions that are less frequently detected.

Images