Deep learning-based multispectral reflectivity ocean mucus detection method

Abstract

The invention discloses a multispectral reflectivity ocean mucus detection method based on deep learning, and belongs to the field of ocean observation. The problems of insufficient spectral data, unbalanced categories and the like in the prior art are solved. According to the technical scheme, the multiband spectral reflectivity data is adopted as the input of the model, and the ocean mucus and other floaters are accurately distinguished in combination with near-infrared and short-wave infrared bands; cloud mask operation is carried out by using a quality control wave band of MODIS data, and it is ensured that data input by the model is not affected by a cloud layer; a novel loss function combining BCE and F1 scores is introduced, so that the model pays more attention to small-class samples in the training process. According to the scheme, the problem of class imbalance in the prior art is effectively solved, and the detection precision of the model and the sensitivity to small-class samples are improved.

Description

Technical Field

[0001] The invention relates to a multi-spectral reflectivity marine mucus detection method based on deep learning, and belongs to the field of marine observation. Background Art

[0002] Existing seasnot detection technologies mainly rely on remote sensing data and machine learning algorithms to analyze the spectral characteristics of floating objects on the ocean surface. In recent years, technologies based on multispectral and hyperspectral satellite images have been widely used. By using satellite images (such as MODIS, Sentinel-2, AVHRR, etc.), researchers can monitor the spread and drift of marine mucus. These images provide an important monitoring tool for large-scale and long-term mucus events. For example, with multispectral data, researchers use RGB image input for detection and classification. These images usually use data from three bands of red, green, and blue to build machine learning models to identify marine mucus. Using these bands, mucus can be distinguished from other floating objects such as algae and plastic waste to a certain extent.

[0003] Although existing technologies have made progress in marine mucus detection, they still have many limitations.

[0004] Limitations of spectral data: Existing technologies are mostly based on RGB images, but the spectral coverage of RGB data is limited, making it difficult to capture the detailed characteristics of marine mucus in different bands. For example, in the near-infrared region, the reflective properties of marine mucus are significantly different from those of seawater or other floating objects (such as plankton and algae), but RGB data cannot effectively capture these differences. This makes the existing detection models perform poorly when distinguishing mucus from other floating objects, resulting in inaccurate detection results.

[0005] Class imbalance problem: The area of ​​marine mucus is relatively small compared to seawater, and the proportion of mucus samples in the dataset is very low. Most existing machine learning models use the binary cross entropy (BCE) loss function for training. However, this loss function is less sensitive to small category samples, causing the model to be more inclined to learn seawater characteristics and ignore small category mucus detection. The class imbalance problem seriously affects the detection effect of existing technologies, especially in the detection of small areas of mucus against a large area of ​​seawater.

[0006] Influence of atmospheric conditions: Existing detection technologies face challenges when there is cloud cover or complex atmospheric conditions. Existing remote sensing technologies have a reflectivity overlap problem when distinguishing between clouds and mucus. The reflectivity characteristics of mucus and clouds in certain spectral bands are very similar, making it difficult to effectively distinguish them, affecting the accuracy of the detection results.

[0007] Insufficient real-time monitoring and spatiotemporal dynamic monitoring: Existing technologies mainly rely on satellite data such as Sentinel, which are acquired at a low frequency and are difficult to achieve real-time dynamic monitoring of marine mucus. In addition, existing technologies lack monitoring methods based on time series data and cannot effectively capture the diffusion, drift and spatiotemporal changes of marine mucus, which makes the monitoring results lag behind and unable to respond to the development of mucus events in a timely manner. Summary of the invention

[0008] In response to the problems existing in the prior art, this application proposes a multispectral reflectance marine mucus detection method based on deep learning. Using multi-band spectral reflectance data as the input of the model and combining it with a new loss function can effectively solve the category imbalance problem in the prior art and improve the model's detection accuracy and sensitivity to small category samples.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is a multispectral reflectance marine mucus detection method based on deep learning, comprising the following steps: Step 1) Acquire medium-resolution multi-band spectral remote sensing data; Get the resampling process quick view provided by WorldView as a reference for label generation; Step 2) Use the quality control bands in MODIS satellite data to perform cloud mask operations; Step 3) Standardize the data of all bands; Step 4) Use the annotation tool to manually annotate the marine mucus area in the WorldView quick view image; Pair the annotated label information with the corresponding MODIS multispectral data to extract the spectral reflectance data of the mucus area in the MODIS data; Step 5) Build a deep neural network model based on the spectral reflectance data.

[0010] Optimized, in the above multispectral reflectance marine mucus detection method based on deep learning, in step 5), the step of constructing a deep neural network model includes: 501) Construction of model input parameters The spectral reflectance data of the mucus area in step 4) is used as the input layer parameter of the model to preliminarily construct a CNN neural network model; 502) Division of training set and test set Randomly allocate the paired data in step 4) into a certain percentage of training sets and test sets; 503) Detach Tag After the division is completed, the water depth column is extracted to become the label. The label is used as the target of model training and the output of the model application after training. 504) Define the model structure The model structure is built based on the U-Net architecture.

[0011] Optimized, the above-mentioned deep learning-based multispectral reflectance marine mucus detection method, in step 1), when obtaining medium-resolution multi-band spectral remote sensing data, the MODIS sensor carried by Terra and Aqua satellites is used to obtain MOD09GA / MOYD09GA data; MOD09GA / MOYD09GA data contains surface reflectance information with a resolution of 500m; Get the resampling process quick view provided by WorldView as a reference, finely mark the marine mucus area, and then generate labels.

[0012] The optimized multispectral reflectance marine mucus detection method based on deep learning includes the following specific processes in step 2): The quality control band is saved in binary data form, where the 10th bit is 1, indicating that the pixel is covered by clouds. By reading the quality control band data, the pixels affected by clouds are filtered out; In step 2), the WorldView data is downsampled to the same 500-meter resolution as the MODIS data by bilinear interpolation or nearest neighbor interpolation.

[0013] The optimized, deep learning-based multispectral reflectance marine mucus detection method, in step 3), Standardize the original input data; the original input data includes MOD09GA / MOYD09GA data; In the standardization process, the z-score standardization method is used. The formula of the z-score standardization method is: Among them, x is the value of the surface reflectance value in MOD09GA / MOYD09GA data before conversion, x* is the value of the surface reflectance value in MOD09GA / MOYD09GA data after conversion; μ is the average value of the original data; is the standard deviation of the original data; The processed data conforms to the standard normal distribution, with a mean of 0 and a standard deviation of 1.

[0014] The optimized, deep learning-based multispectral reflectance marine mucus detection method, in step 4), Use annotation tools to manually annotate the marine mucus areas in the WorldView quick view image; based on the manual annotation results, generate corresponding label data; During manual annotation, the annotated label information was paired with the corresponding MODIS multispectral data, and seven groups of spectral reflectance values ​​of the mucus area in the MODIS data were extracted; The seven groups of spectral reflectance value bands are red light band with a wavelength of 620-670nm, near infrared band with a wavelength of 841-876nm, blue light band with a wavelength of 459-479nm, green light band with a wavelength of 545-565nm, short-wave infrared band 1 with a wavelength of 1230-1250nm, short-wave infrared band 2 with a wavelength of 1628-1652nm, and short-wave infrared band 3 with a wavelength of 2105-2155nm. The extracted data is converted into numpy array format; Labeling tools include LabelMe.

[0015] Optimized, the above-mentioned multispectral reflectance marine mucus detection method based on deep learning, in step 501), selects different parameter combinations as input parameters of the DNN neural network model; In step 502), the spectral reflectance data of all matched mucus areas in step 4) are randomly allocated into a training set and a test set according to percentage, and the training set is used to train the CNN model, and during the CNN model training process, 20% of the training set is further divided as a validation set; The test set is used as an independent part to test the recognition effect of the model after training; Optimized, the above-mentioned deep learning-based multispectral reflectance marine mucus detection method adopts a new loss function combining binary cross entropy and F1 score in step 504); The new loss function formula combining binary cross entropy (BCE) and F1 score is ; Among them, BCE is the binary cross entropy that calculates the difference between the predicted value and the true value; F1 score is a weight adjustment parameter, and the CNN model uses F1 adjustment to focus on the mucus area with less detection. Since mucus detection is a binary classification problem, binary cross entropy is usually used as the loss function. However, compared with the ocean background, mucus detection is more of an unbalanced classification problem because mucus patches may only occupy a small part of the entire image. Therefore, using only the "accuracy" indicator cannot fully evaluate the model. α is the balance factor between F1 and BCE. Find the optimal model by changing α.

[0016] The beneficial effects of this application are: The existing technology usually uses RGB images for training and detection. This method only uses information from three bands and cannot fully capture the spectral characteristics of marine mucus. The technical solution of this application uses multi-band spectral reflectance data as the input of the model, especially combining near-infrared and short-wave infrared bands, which can more accurately distinguish marine mucus from other floating objects (such as algae and plastic waste). In this way, the model obtains richer marine mucus feature information in different bands, and the detection results are more accurate.

[0017] The technical solution of this application uses the quality control band of MODIS data to perform cloud mask operations to ensure that the data input to the model is not affected by clouds. Compared with the existing technology that relies on manual annotation or simple threshold screening methods, automated cloud mask processing can more accurately remove pixels affected by clouds, thereby improving the accuracy and reliability of the detection results.

[0018] The technical solution of the present application, by introducing a new loss function that combines BCE and F1 scores, enables the model to pay more attention to small category samples during the training process, improves the detection sensitivity of marine mucus, and significantly improves the detection effect, especially when dealing with a smaller area of ​​marine mucus. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of the steps of the technical solution of this application. DETAILED DESCRIPTION

[0020] The technical features of the present invention are further described below in conjunction with specific embodiments.

[0021] The present application provides a multi-spectral reflectance marine mucus detection method based on deep learning. With the intensification of water eutrophication, global climate change and other phenomena, the frequency of marine mucus events in global waters has increased year by year, and the optical detection of marine mucus is relatively difficult. A multi-band spectral reflectance data remote sensing data combined with a CNN model is proposed to improve the accuracy of quantitative extraction of mucus information and realize remote sensing intelligent monitoring of marine mucus.

[0022] The multispectral reflectance marine mucus detection method based on deep learning of the present application comprises the following steps: Step 1) Obtain medium-resolution multi-band spectral remote sensing data. Mainly using the MODIS sensors on Terra and Aqua satellites, these sensors provide rich multi-spectral reflectance data, covering multiple bands from visible light to near infrared. MOD09GA / MOYD09GA data contains surface reflectance information with a resolution of 500m, which is suitable for large-scale ocean monitoring.

[0023] Get the resampling process quick view provided by WorldView as a reference, which can be used to fine-tune the marine mucus area and generate labels in combination with MODIS data.

[0024] Step 2) The cloud mask operation is performed using the quality control band in the MODIS satellite data. The quality control band is saved in the form of binary data, where the 10th bit is 1, indicating that the pixel is covered by clouds. By reading the quality control band data, pixels affected by clouds are filtered out to ensure the quality of the input data.

[0025] Since the resolutions of MOD09GA / MOYD09GA data and WorldView data are different, in order to ensure the alignment consistency of the data, both need to be resampled. Through bilinear interpolation or neighbor interpolation methods, the high-resolution WorldView data is downsampled to the same 500-meter resolution as the MODIS data, which can ensure the computational efficiency and consistency of subsequent processing.

[0026] Step 3) In order to prevent the scale differences of data in different bands from affecting the training effect of the model, the data of all bands need to be standardized. When the values ​​of the input parameters differ greatly, if the original values ​​are directly used for analysis, the role of the parameters with higher values ​​in the model will be highlighted, and the role of the parameters with lower values ​​will be relatively weakened.

[0027] Therefore, in order to ensure the reliability of the results, the original input data needs to be standardized. The z-score normalization method is used here, that is, Where x and x* are the values ​​before and after the conversion, respectively, and μ and The processed data conforms to the standard normal distribution, that is, the mean is 0 and the standard deviation is 1, which improves the convergence speed and stability of the model.

[0028] Step 4) Labeling tools such as LabelMe are used to manually annotate the marine mucus areas in the WorldView quick view images. Based on the manual annotation results of these high-resolution images, the corresponding label data is generated. The distribution of marine mucus usually has complex morphology, so high-precision manual annotation can ensure the quality of training data.

[0029] Pair the annotated label information with the corresponding MODIS multispectral data to extract seven groups of spectral reflectance values ​​of the mucus area in the MODIS data. The seven groups of spectral reflectance values ​​are red light band with a wavelength of 620-670nm, near infrared band with a wavelength of 841-876nm, blue light band with a wavelength of 459-479nm, green light band with a wavelength of 545-565nm, short-wave infrared band 1 with a wavelength of 1230-1250nm, short-wave infrared band 2 with a wavelength of 1628-1652nm, and short-wave infrared band 3 with a wavelength of 2105-2155nm. These extracted data will be converted into numpy array format to facilitate the input of subsequent models.

[0030] Step 5) Based on the spectral reflectance data, a deep neural network (DNN) model is constructed.

[0031] The input layer of the model is multi-band spectral reflectance values, including reflectance information of visible light and near-infrared bands. By combining data from these different bands, the model can capture the complex spectral characteristics of mucus and improve the accuracy of detection.

[0032] Among them, the marine mucus monitoring DNN model based on the novel loss function of binary cross entropy (BCE) and F1 score was loaded in python (a computer programming language) software as follows: 501) Construction of model input parameters The MOD / MYD 09GA data involve many bands in imaging, and it is difficult to monitor marine mucus.

[0033] In the present application, the spectral reflectance data of the mucus area in step 4) is used as the input layer parameter of the model to preliminarily construct a DNN neural network model. Different parameter combinations are selected as the input parameters of the DNN neural network model.

[0034] 502) Division of training set and test set The spectral reflectance data of all matched mucus areas in step 4) are randomly divided into training set and test set according to percentage. The training set is used to train the CNN model, and 20% of the training set is divided as the validation set during the CNN model training process. The test set is used as an independent part to test the recognition effect of the model after training.

[0035] 503) After the separation label division is completed, the water depth column is extracted as the label (Label, also known as the true value), which is the target during model training and the output when the model is applied after training is completed (i.e., the identified mucus pixels).

[0036] 504) Define the model structure This paper is based on the classic U-Net architecture and makes customized improvements. U-Net performs well in the field of image segmentation and can effectively integrate features at different levels. For the binary classification problem of marine mucus detection, the model input is multi-band spectral reflectance and the output is the segmentation result of marine mucus. In order to deal with the problem of class imbalance in marine mucus detection, a new loss function combining binary cross entropy (BCE) and F1 score is adopted. The loss function formula is as follows: Among them, BCE is used to calculate the difference between the predicted value and the true value, and the F1 score is adjusted through weights, so that the model pays more attention to the mucus areas with less detection, thereby improving the detection sensitivity of the model for small category samples.

[0037] After defining the model structure, the accuracy of the model is evaluated to evaluate the performance of the model and display the recognition results of the model during the training process.

[0038] In this application, the following indicators are used to evaluate the accuracy of the model: (Accuracy): The proportion of correctly predicted ocean mucus pixels to the total pixels. (Precision): The proportion of pixels that the model correctly predicted to be algae that are actually algae. A high precision value means that the model produces fewer false positives. (Recall): The proportion of actual marine mucus pixels that the model correctly predicts to be marine mucus.

[0039] A high recall means that the model misses fewer marine mucus pixels. (F1 score): The harmonic mean of precision and recall, used to strike a balance between precision and recall. ( , intersection / union ratio): the ratio of the intersection and union of the predicted ocean mucus pixels and the actual ocean mucus pixels. The higher it is, the more accurate the model is in predicting the areas of marine mucus.

[0040] (3) (4) (5)

[0041] (6)

[0042] (7) in, For a real example, is a true negative example, For a false positive example, is a false negative example.

[0043] Through multiple experiments, the model performance under different band combinations was compared to further verify the validity of the spectral reflectance data.

[0044] Through this method, the present invention can effectively improve the detection accuracy of marine mucus, solve the problems of insufficient spectral data and category imbalance in the prior art, and realize efficient monitoring of marine mucus.

[0045] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A multispectral reflectance marine mucus detection method based on deep learning, characterized in that: The following steps are involved: Step 1) Acquire medium-resolution multi-band spectral remote sensing data; Get the resampling process quick view provided by WorldView as a reference for label generation; Step 2) Use the quality control bands in MODIS satellite data to perform cloud mask operations; Step 3) Standardize the data of all bands; Step 4) Use the annotation tool to manually annotate the marine mucus area in the WorldView quick view image; Pair the annotated label information with the corresponding MODIS multispectral data to extract the spectral reflectance data of the mucus area in the MODIS data; Step 5) Build a deep neural network model based on the spectral reflectance data.

2. The method for detecting marine mucus based on multispectral reflectance based on deep learning according to claim 1, characterized in that: In step 5), the steps of building a deep neural network model include: 501) Construction of model input parameters The spectral reflectance data of the mucus area in step 4) is used as the input layer parameter of the model to preliminarily construct a DNN neural network model; 502) Division of training set and test set Randomly allocate the paired data in step 4) into a certain percentage of training sets and test sets; 503) Detach Tag After the division is completed, the water depth column is extracted to become the label. The label is used as the target of model training and the output of the model application after training. 504) Define the model structure The model structure is built based on the U-Net architecture.

3. The multispectral reflectance marine mucus detection method based on deep learning according to claim 2, characterized in that: In step 1), when obtaining medium-resolution multi-band spectral remote sensing data, the MODIS sensors carried by Terra and Aqua satellites are used to obtain MOD09GA / MOYD09GA data; MOD09GA / MOYD09GA data contain surface reflectance information with a resolution of 500m; Get the resampling process quick view provided by WorldView as a reference, finely mark the marine mucus area, and then generate labels.

4. The method for detecting marine mucus based on multispectral reflectance based on deep learning according to claim 3, characterized in that: The specific process in step 2) includes: The quality control band is saved in the form of binary data. If the 10th bit of the binary data of one pixel is 1, it means that the pixel is covered by clouds. By reading the quality control band data, the pixels affected by clouds are filtered out; In step 2), the WorldView data is downsampled to the same 500-meter resolution as the MODIS data by bilinear interpolation or nearest neighbor interpolation.

5. The method for detecting marine mucus based on multispectral reflectance based on deep learning according to claim 4, characterized in that: In step 3), Standardize the original input data; the original input data includes MOD09GA / MOYD09GA data; In the standardization process, the z-score standardization method is used. The formula of the z-score standardization method is: Where x is the value of the surface reflectance value in MOD09GA / MOYD09GA data before conversion, x* is the value of the surface reflectance value in MOD09GA / MOYD09GA data after conversion; μ is the average value of the original data; is the standard deviation of the original data; The processed data conforms to the standard normal distribution, with a mean of 0 and a standard deviation of 1.

6. The method for detecting marine mucus based on multispectral reflectance based on deep learning according to claim 5, characterized in that: In step 4), Use the annotation tool to manually annotate the marine mucus areas in the WorldView quick view image; Generate corresponding label data based on manual annotation results; During manual annotation, the annotated label information was paired with the corresponding MODIS multispectral data, and seven groups of spectral reflectance values ​​of the mucus area in the MODIS data were extracted; The seven groups of spectral reflectance value bands are red light band with a wavelength of 620-670nm, near infrared band with a wavelength of 841-876nm, blue light band with a wavelength of 459-479nm, green light band with a wavelength of 545-565nm, short-wave infrared band 1 with a wavelength of 1230-1250nm, short-wave infrared band 2 with a wavelength of 1628-1652nm, and short-wave infrared band 3 with a wavelength of 2105-2155nm. The extracted data is converted into numpy array format; Labeling tools include LabelMe.

7. The method for detecting marine mucus based on multispectral reflectance based on deep learning according to claim 6, characterized in that: In step 501), different parameter combinations are selected as input parameters of the DNN neural network model; In step 502), the spectral reflectance data of all matched mucus areas in step 4) are randomly allocated into a training set and a test set according to percentage, and the training set is used to train the CNN model, and during the CNN model training process, 20% of the training set is further divided as a validation set; The test set is used as an independent part to test the recognition effect of the model after training.

8. The method for detecting marine mucus based on multispectral reflectance based on deep learning according to claim 7, characterized in that: In step 504), a new loss function combining binary cross entropy and F1 score is used; the formula of the new loss function combining binary cross entropy (BCE) and F1 score is: ;in, BCE is the binary cross entropy that calculates the difference between the predicted value and the true value; F1 score is the weight adjustment parameter, and the CNN model focuses on the mucus area with less detection through F1 adjustment.

Images