Soil micro-plastic detection method based on self-supervised hyperspectral image classification

Abstract

The invention relates to a soil micro-plastic detection method based on self-supervised contrast learning hyperspectral image classification, and solves the problems that the traditional soil micro-plastic detection efficiency is low and time and labor are wasted when a soil micro-plastic sample is labeled. According to the method, a self-supervised contrast learning technology and a hyperspectral image are combined and introduced into soil micro-plastic detection, so that the detection speed of the soil micro-plastic can be increased under the condition that an original sample is not damaged; meanwhile, a plurality of feature extraction modules are combined to design an HDTNet network model to serve as a feature extractor of self-supervised contrast learning, so that feature representation of soil micro-plastics is better extracted; and finally, enhancement of spectral information is introduced into a data enhancement part of self-supervised contrast learning, so that the spectral information of the hyperspectral image can be better utilized. Compared with several existing soil micro-plastic detection technologies, the method has better performance, and realizes accurate identification of micro-plastic particles in the soil environment.

Description

Technical Field

[0001] The present application relates to the field of hyperspectral image classification technology and soil detection, and more specifically, to a soil microplastic detection method based on self-supervised comparative learning hyperspectral image classification. Background Art

[0002] In recent years, there have been more and more studies and reports on microplastic pollution in soil. Although microplastic particles can be accurately detected by Raman spectroscopy or Fourier transform infrared spectroscopy, the inspection process requires each particle to be analyzed independently and the suspected microplastic particles to be visually identified first. In order to obtain reliable research on the presence of microplastic contaminants in the environment, a large number of particles must be analyzed, which will consume a lot of time using current spectral detection technology. Therefore, a new detection technology must be developed to quickly and accurately determine and map the distribution of microplastics in soil.

[0003] Hyperspectral imagery (HSI) technology has attracted much attention due to its rich spectral and spatial information. HSI helps to identify the characteristics of different substances through high-resolution spectral information, which makes it a powerful tool for detecting and identifying soil microplastics. Hyperspectral image classification research currently focuses on improving supervised hyperspectral image classification algorithms to obtain more accurate ground object information. These supervised learning-based methods can improve classification performance when there are sufficient training samples. However, in the soil microplastic scenario, it is difficult to label a large number of training samples. Therefore, this application proposes to use self-supervised contrastive learning to classify hyperspectral soil microplastic images. This method can improve the accuracy and efficiency of soil microplastic detection, and help assess the pollution level of the soil environment and formulate appropriate environmental protection policies. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of low efficiency of traditional soil microplastic detection methods and insufficient samples of soil microplastic data sets, and to provide a soil microplastic detection method based on self-supervised contrast learning hyperspectral image classification to accurately and quickly identify microplastics in soil.

[0005] This application provides a soil microplastic detection method for self-supervised contrast learning hyperspectral image classification, including:

[0006] Collect hyperspectral images of soil microplastics;

[0007] Preprocessing the collected soil microplastic hyperspectral images;

[0008] Label each pixel of the preprocessed hyperspectral image as a dataset;

[0009] Divide the dataset into training and testing sets;

[0010] Build the self-supervised contrastive learning BYOL model and the improved feature extractor part, namely the HDTNet network model;

[0011] Train the model using unlabeled data and save the feature extractor model parameters;

[0012] Train the feature extractor model using a small amount of labeled data and fine-tune the model parameters;

[0013] Apply the test set to the trained model to check the classification accuracy;

[0014] The entire soil microplastic hyperspectral image is input into the above model to obtain the detection results of soil microplastics.

[0015] The ratio of the training set to the test set is 0.1:0.9.

[0016] The self-supervised contrastive learning model iterates for 100 epochs, and the feature extractor model iterates for 200 epochs.

[0017] It can be seen from the above technical scheme that the present application provides a soil microplastic detection method based on self-supervised contrastive learning hyperspectral image classification, including collecting soil microplastic hyperspectral images; preprocessing the collected hyperspectral image data of four common soil microplastics; labeling each pixel of the preprocessed hyperspectral image as a data set; dividing the data set into a training set and a test set; building a self-supervised contrastive learning BYOL model and an improved feature extractor model; using unlabeled data to train the model and save the feature extractor model parameters; using a small amount of labeled data to train the feature extractor model and fine-tune the model parameters; applying the test set to the trained model to detect the classification accuracy; inputting the entire soil microplastic hyperspectral image into the above model to obtain the detection results of soil microplastics.

[0018] This application uses a self-supervised contrastive learning framework to classify hyperspectral images, uses a small amount of labeled data to train the model, and after the training is completed, the collected soil microplastic hyperspectral images are input into the above-mentioned combined model to obtain soil microplastic detection results, thereby achieving the purpose of accurate and rapid detection of soil microplastics. This application can not only improve the rapidity of detecting soil microplastics, but also achieve a higher accuracy of soil microplastic hyperspectral image classification using only a small amount of labeled samples, which is helpful for assessing the pollution level of the soil environment and formulating corresponding environmental protection policies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 Flowchart of the proposed method for detecting soil microplastics

[0021] Figure 2 The overall structure diagram of the improved feature extraction model (HDTNet)

[0022] Figure 3 (a) is the expansion attention module; (b) is the poolformer module; (c) is the MLP structure diagram

[0023] Figure 4 For the fused convolution module

[0024] Figure 5 Schematic diagram of the long-term expanded attention (MHDA)

[0025] Figure 6 Visualization of false color composite images, reference images, and classified images of four types of soil microplastics. (a) PP. (b) PVC. (c) PE. (d) PET. DETAILED DESCRIPTION

[0026] The present application will be described in detail below with reference to the accompanying drawings and in combination with implementation modes.

[0027] In order to overcome the problems of time-consuming, labor-intensive and inefficient existing soil microplastic detection methods, the present invention provides a soil microplastic detection method based on self-supervised hyperspectral image classification. This method has good detection effect and efficiency for soil microplastics, and can be used to quickly and accurately determine and map the distribution of microplastics in the soil.

[0028] First, the soil samples were air-dried, then ground and sieved through a 2 mm metal sieve. Four types of microplastic particles (PP, PVC, PE, PET) were artificially added to the soil. These plastic particles were manually cut to a size of less than five millimeters and added to the soil samples to simulate the distribution of microplastics in real soil. The soil samples were scanned using a SPECIMFX10e hyperspectral camera. A patch with a spatial size of 7×7 was cropped out with each pixel as the center. The label of this patch is the label of its central pixel, and these patches are used as a dataset.

[0029] Furthermore, the self-supervised model is BYOL in the contrastive learning model, which consists of an online network and a target network. The online network consists of three stages: encoder, projector, and predictor. The target network has the same architecture as the online network, but uses a different set of weights. The target network provides a regression target to train the online network, whose parameters are the exponential moving average of the online parameters. Given the target decay rate τ∈[0,1], after each training step, the following update is performed: ξ←τξ+(1-τ)θ.

[0030] Furthermore, the pretext task is to use the unlabeled original image and its enhanced image to learn features. 1 and x 2 The deep features f 1 and f 2 It can be obtained through a layered expansion transformer network. In order to learn deep features from unlabeled samples, the training strategy is to make the distance between similar features smaller and smaller, and the distance between different features larger and larger, that is, to minimize the similarity loss between the output of the prediction in the online network and the output of the projection in the target network. The formula for calculating the contrast loss is as follows: Where Y is the output of the online network prediction, and Y′ is the output of the target network projection. At the end of training, only the encoder part is retained, and the rest is discarded.

[0031] Finally, the parameters of the encoder part learned in the pretext task are passed to the downstream classification network, and a small amount of labeled data is input into the hierarchical expansion transformer model for training, that is, the model parameters are fine-tuned to solve the classification problem by minimizing the classification objective function.

[0032] Where C is the total number of classes, y i is the basic truth value, is the predicted label.

[0033] Furthermore, the image enhancement part mainly includes spatial horizontal mirroring, image random noise, spatial horizontal rotation and spectral mirroring. Let I be the input HSI data, firstly through spatial horizontal mirroring, the data is obtained And add random Gaussian noise to the data In the above example, we get noisy data. In the data Perform spatial horizontal rotation operation to obtain the rotated data in Finally, in the spectral domain, the spectral mirror operation is used to obtain richer spectral features, thereby obtaining the data The spectral mirroring technique is used to increase the amount of training data, so that the feature extraction model can better learn the commonalities of the same type. During the training phase, the augmented data is randomly selected. As input to the pretext task.

[0034] Furthermore, the model adopts a four-stage structure to generate feature maps of different scales. The first two stages use the LMFFM module, and the last two stages use the GPLM module. The first two layers are mainly used to extract local feature information of the image, and the last two layers are mainly used to extract global feature information.

[0035] Furthermore, the entire patch is provided to the four-stage feature extraction framework, which can fully extract the global and local spatial spectral features of the image. The four stages will generate corresponding feature maps with sizes of P×P×C 1 ,P×P×C 2 ,P×P×C 3 and P×P×C 4 In our model, there are four stages {C 1 ,C 2 ,C 3 ,C 4} are set to {256, 128, 128, 64} respectively. Using dimensionality reduction to form a hierarchical style of the model can reduce resource consumption and mine useful features.

[0036] Furthermore, the local multi-scale feature fusion module (LMFFM) consists of grouped convolution, fused convolution module and multi-head dilated attention (MHDA). The first two layers use the LMFFM module mainly to extract the local feature information of soil microplastic hyperspectral images.

[0037] Furthermore, the MHSA mechanism is a key part of the transformer model, through which the intrinsic connection of the global spatial spectral features of the entire hyperspectral image can be calculated. Grouped convolution is added before the MHSA mechanism to divide the channels of the feature map into non-overlapping sub-channels to extract spatial spectral features from the sub-channels. Ordinary convolution is replaced by grouped convolution to better extract the feature representation of soil microplastics. Compared with ordinary convolution, grouped convolution has fewer parameters and is not easy to overfit. At the same time, grouped convolution can extract the discriminative information shared by continuous sub-channels of the feature map. Therefore, grouped convolution can better capture the local information of the feature. In this model, the number of input channels is divided into 16 groups.

[0038] Furthermore, the fused convolution module consists of a 3×3 expanded convolution, a SE module, and a 1×1 projection layer. The 3×3 expanded convolution is used to double the number of channels. The SE module includes a global pooling operation, a fully connected layer, a Relu activation function layer, a fully connected layer, and a sigmoid activation function layer. Finally, a 1×1 projection layer is used to convert the number of channels to the original number of channels, which is convenient for adding the input and output later.

[0039] Furthermore, the dilated attention module consists of alternating layers of multi-head dilated attention (MHDA) and multi-layer perceptron (MLP). MHDA is an improvement on the multi-head self-attention mechanism (MHSA). MHSA is a mechanism designed to build global long-range dependencies and extract important parts of features. MHSA enriches the diversity of feature subspaces by parallel computing through linear projection of inputs into multiple feature subspaces without requiring additional computing time, thus improving computational efficiency.

[0040] Furthermore, the features are mapped into four different heads, and dilated convolution with different dilation rates is used in each head feature, and the dilation rate is set to {r|r∈[1,2,3,4]}, so as to extract features of different scales in different heads. Specifically, first input X∈R C×H×W Through linear transformation, it is mapped to three different matrices, namely, query Q∈R C×H×W , key K∈R C×H×W , value V∈R C×H×W , as shown below: Q = XW Q ,K=XW K ,V=XW V

[0041] Where W Q ,W K and W V is a trainable parameter matrix. Then Q, K, and V are divided by 4 along the dimension axis as follows:

[0042] Q={Q 1 ,Q 2 ,Q 3 ,Q 4}

[0043] K={K 1 ,K 2 ,K 3 ,K 4}

[0044] V={V 1 ,V 2 ,V 3 ,V 4}

[0045] Where Qi ,K i ,V i ∈R s×(d / 4) Then each Q i Using dilated convolution with different dilation rates, we get Q ci , as shown below: Q ci =DilatedConv(Q i )

[0046] Where DilatedConv(·) represents dilated convolution. Next, the attention score of each i is calculated by a scaled softmax function:

[0047] The final output is to convert all Z i ∈R s×(d / n) The concatenated projection can be expressed as Output = Concat(Z 1 ,Z 2 ,Z 3 ,Z 4 )W o

[0048] Where W o ∈ represents the output projection matrix.

[0049] The MLP is a two-layer linear projection with a Gaussian error linear unit (GELU) nonlinearity in the middle to extract feature representation. The MLP module can be summarized as MLP(X) = FC 2 (GELU(FC 1 (X)))

[0050] The linear projection part is implemented by the fully connected layer (FC). GELU is a linear projection function commonly used in ViT models and is defined as follows:

[0051] In the formula is the standard Gaussian cumulative distribution function, In the transformer block, a LayerNorm layer is applied before the MHSA module and the MLP layer. In addition, residual connections are introduced to alleviate the overfitting problem commonly seen in deep learning methods.

[0052] Furthermore, the global pooling lightweight module (GPLM) consists of a grouped convolution and a poolformer module. The GPLM module is used in the last two layers to extract the global feature information of soil microplastic hyperspectral images. The PoolFormer module proposes to use a simple pooling operation as a token mixer. It can not only extract global features, but also reduce the number of calculation parameters and improve calculation efficiency compared to the self-attention mechanism of the original transformer model. Compared with ordinary RGB images, hyperspectral images have rich spectral features, which leads to a large number of calculation parameters and low model training efficiency. Placing the PoolFormer module in the last two stages of this model can make the model more lightweight and help obtain soil microplastic detection results more quickly.

[0053] Furthermore, the training of the model is divided into two parts. The first part uses unlabeled soil microplastic hyperspectral images to pre-train the BYOL model. The second part uses a small number of labeled hyperspectral images to fine-tune the model parameters.

[0054] Furthermore, the soil microplastics will be verified using four common microplastics found in soil, including polypropylene (PP), polyvinyl chloride (PVC), polyethylene (PE) and polyethylene terephthalate (PET).

[0055] Experimental Section

[0056] All experiments were run in the deep learning framework of pytorch. The specific computer configuration is shown in Table 2. In this work, a stochastic gradient descent (SGD) optimizer with a minimum batch size of 128, a momentum of 0.9, and a weight decay of 0.0001 was added to update the training parameters. The total number of iterations was set to 300, of which the number of iterations in the pretext task was set to 100 and the number of iterations in the downstream classification task was set to 200. In the downstream classification task, we did not use any learning rate scheduler. For simplicity, the learning rate was set to a constant of 0.001. The experiment used self-built soil microplastic hyperspectral images as the dataset, and randomly selected 10% of the labeled samples for training, 10% for validation, and 80% for testing.

[0057] For the evaluation indicators of the model, we select the three most widely used objective evaluation indicators in the industry: Overall accuracy (OA), Average Accuracy (AA) and Kappa coefficient (K). These three evaluation indicators are calculated based on the Confusion Matrix. The following is an introduction to these evaluation indicators:

[0058] (1) Confusion Matrix: A confusion matrix consisting of n rows and n columns is an error matrix that can be expressed in a standard form for calculating accuracy, where each column represents the predicted value and each row represents the true category. The confusion matrix can clearly show whether there is confusion between multiple categories, that is, whether one category is predicted as another category.

[0059] Table 1 Composition of confusion matrix

[0060]

[0061] (2) Overall classification accuracy (OA): OA is the overall classification accuracy of the evaluation algorithm, which refers to the ratio of the number of correctly classified pixels to the total number of categories. Overall classification accuracy is a basic evaluation indicator, which indicates the proportion of correctly classified samples to the total number of samples:

[0062] (3) Average classification accuracy (AA): AA is an evaluation index of classification accuracy in the classification algorithm. First, the classification accuracy of each type of object is summed up, and then the average is calculated as the classification accuracy. The average classification accuracy is different from the overall classification accuracy. It focuses on evaluating the classification accuracy of each type of object in the algorithm:

[0063] (4) Kappa coefficient (K): The Kappa coefficient indicates the proportion of the reduction in completely random classification errors. It is also an evaluation indicator of classification accuracy. As a supplement to the error matrix, which cannot clearly show the classification accuracy, it can clearly reflect the effect of classification accuracy:

[0064]

[0065] Among them, n represents the number of rows and columns of the classification matrix, m ij Represents the value of the i-th row and j-th column in the confusion matrix, m i+ is the row sum of the confusion matrix, and m +i It is the column integration of the confusion matrix, and N represents all the elements contained in the confusion matrix.

[0066] The objective classification results of the network model used in the present invention on four data sets are shown in Table 2:

[0067] Table 2 Classification results of network models for different data

[0068]

[0069] From left to right, the first column is the data set; the second column is the OA data of the classification results; the third column is the AA data of the classification results; and the fourth column is the Kappa data of the classification results.

Claims

1. A soil microplastic detection method based on self-supervised contrastive learning hyperspectral image classification, characterized in that: The self-supervised contrastive learning technology is combined with the hyperspectral image classification method and applied to the detection of soil microplastics. The method specifically includes the following steps: Step 1: Obtaining soil microplastic hyperspectral images; Step 2: preprocess the hyperspectral image obtained in step 1, and annotate each pixel with a category label as a data set; Step 3: Build a self-supervised contrastive learning BYOL network model. Step 4: Use unlabeled data to train the self-supervised contrastive learning model, save the parameters of the feature extractor part, and use a small amount of labeled data to fine-tune the model parameters. Step 5: Input the collected soil microplastic hyperspectral image into the trained model, and output the soil microplastic detection results through the constructed network.

2. The BYOL network model according to claim 1, characterized in that: Only a small number of labeled samples are needed to achieve good classification performance, saving the cost of manual labeling; at the same time, the BYOL model can be trained without negative samples.

3. The BYOL model according to claim 1, characterized in that: Spatial horizontal mirroring, image random noise, spatial horizontal rotation, and spectral mirroring are used as data augmentation methods.

4. The BYOL model according to claim 1, characterized in that: The feature extractor part in the model was changed to a feature extractor that is more suitable for soil microplastic hyperspectral images, namely the hierarchical dilated transformer (HDTNet) network model.

5. The feature extractor model according to claim 4, characterized in that It consists of a four-layer structure. The first two layers include grouped convolution, fused convolution modules and multi-head dilated attention (MHDA) modules, and the last two layers include grouped convolution and poolformer modules.

Images