Soil plastic detection method and system based on multi-modal fusion and deep learning

Abstract

The invention relates to the field of soil substance detection, and discloses a soil plastic detection method and system based on multi-modal fusion and deep learning, and the method comprises the following steps: obtaining one-dimensional hyperspectral data of a to-be-detected soil sample; converting the one-dimensional hyperspectral data based on an image conversion algorithm to obtain two-dimensional image representation; performing feature extraction on the one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features; performing feature extraction on the two-dimensional image representation based on a second preset algorithm to obtain image features; performing multi-view probability fusion on the spectral features and the image features to obtain fusion features; and inputting the fusion features into a pre-trained double-path attention residual convolutional network model for classification to obtain a detection result of the micro-plastics in the to-be-detected soil sample. According to the method, the detection accuracy is improved, the model robustness is enhanced, and low-concentration detection is realized.

Description

Technical Field

[0001] The present invention relates to the field of soil material detection, and in particular to a soil plastic detection method and system based on multimodal fusion and deep learning. Background Art

[0002] Microplastic pollution, a new environmental issue, is widespread in various environmental media, including soil, water, and the atmosphere, posing a potential threat to ecosystems and human health. Soil is a key sink and migration and transformation hub for microplastics. Therefore, rapid and accurate detection and assessment of microplastic contamination in soil is crucial for environmental protection and risk management.

[0003] Traditional soil microplastic detection methods, such as visual inspection and density separation, are usually time-consuming and labor-intensive, and have limited ability to identify small or transparent microplastics.

[0004] Hyperspectral imaging (HSI) technology can simultaneously obtain spatial image information and spectral information of a substance, making it possible to quickly and non-destructively identify the substance. However, in soil microplastic detection, single-modality hyperspectral analysis (for example, relying only on spectral curves or only on spatial images) faces many challenges:

[0005] 1. The soil matrix is ​​complex and easily interferes with spectral signals. Microplastics are of various types, shapes, and sizes, and their concentrations in soil are usually low, resulting in weak or easily masked spectral characteristics.

[0006] 2. Directly using the spatial information of hyperspectral images for classification has limited accuracy.

[0007] Most existing research focuses on direct analysis of one-dimensional spectral data or simple feature cascades, failing to fully explore and utilize the structured information contained in spectral data and the complementarity between spectral and image information, resulting in detection accuracy and model robustness that are difficult to meet practical application requirements in complex environments.

[0008] Therefore, how to effectively extract and fuse multimodal features in hyperspectral data to overcome the limitations of single modal information and improve the accuracy and reliability of soil microplastic detection is a technical problem that needs to be solved urgently. Summary of the Invention

[0009] The purpose of the present invention is to provide a soil plastic detection method and system based on multimodal fusion and deep learning, so as to solve the problem that most existing research focuses on the direct analysis of one-dimensional spectral data or simple feature cascade, fails to fully explore and utilize the structured information contained in spectral data and the complementarity between spectral and image information, resulting in detection accuracy and model robustness that are difficult to meet the actual application requirements in complex environments.

[0010] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0011] In a first aspect, the present invention provides a soil plastic detection method based on multimodal fusion and deep learning, the method comprising:

[0012] Obtain one-dimensional hyperspectral data of the soil sample to be tested;

[0013] Based on the image conversion algorithm, the one-dimensional hyperspectral data is converted into a two-dimensional image representation;

[0014] Performing feature extraction on the one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features;

[0015] Performing feature extraction on the two-dimensional image representation based on a second preset algorithm to obtain image features;

[0016] Perform multi-view probability fusion on spectral features and image features to obtain fusion features;

[0017] The fused features are input into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested.

[0018] Preferably, the image conversion algorithm is at least one of continuous wavelet transform, recursive graph and Gramian angular field.

[0019] Preferably, the first preset algorithm is at least one of a continuous projection algorithm, an uninformative variable elimination method, a genetic algorithm, a principal component analysis, and an isometric mapping.

[0020] Preferably, the second preset algorithm is at least one of a histogram of oriented gradients, a local binary pattern, a generalized search tree, a Gabor filter feature, and an edge gradient feature.

[0021] Preferably, the spectral features and the image features are subjected to multi-view probability fusion to obtain fused features, including:

[0022] Performing standardization processing on the spectral features and the image features to obtain processed spectral features and processed image features;

[0023] Construct two basic classifiers and train the two basic classifiers to obtain two trained basic classifiers;

[0024] The processed spectral features are input into one of the trained basic classifiers to obtain the spectral view class probability prediction vector;

[0025] The processed image features are input into another trained basic classifier to obtain the image view category probability prediction vector;

[0026] The spectral view category probability prediction vector and the image view category probability prediction vector are concatenated to obtain the fusion feature.

[0027] Preferably, the basic classifier is at least one of a support vector machine, a random forest and a multi-layer perceptron.

[0028] Preferably, the dual-path attention residual convolutional network model includes an input module, a convolutional neural network module, a multi-head attention module, a dual-branch processing module and an output module connected in sequence;

[0029] The fused features are input into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested, including:

[0030] The input module receives the fusion feature;

[0031] The convolutional neural network module performs local feature extraction on the fusion features to obtain local features;

[0032] The multi-head attention module captures the long-distance dependencies of local features to obtain attention features;

[0033] The dual-branch processing module performs multi-scale learning on the attention features to obtain main branch features and secondary branch features;

[0034] The output module performs weighted fusion on the main branch features and the secondary branch features to obtain the final features, and inputs the final features into the activation function for classification to obtain the category probability distribution, and determines the detection results based on the category probability distribution.

[0035] In a second aspect, the present invention provides a soil plastic detection system based on multimodal fusion and deep learning, which is used to implement the above-mentioned soil plastic detection method based on multimodal fusion and deep learning. The system includes:

[0036] A data acquisition module is used to obtain one-dimensional hyperspectral data of the soil sample to be tested;

[0037] A data conversion module is used to convert one-dimensional hyperspectral data into a two-dimensional image representation based on an image conversion algorithm;

[0038] A spectral feature extraction module is used to extract features from one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features;

[0039] An image feature extraction module, configured to extract features from the two-dimensional image representation based on a second preset algorithm to obtain image features;

[0040] Feature fusion module, used to perform multi-view probability fusion of spectral features and image features to obtain fused features;

[0041] The classification and detection module is used to input the fused features into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested.

[0042] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned soil plastic detection method based on multimodal fusion and deep learning is implemented.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned soil plastic detection method based on multimodal fusion and deep learning.

[0044] The beneficial effects of the present invention are concentrated in:

[0045] 1. Improved detection accuracy: This invention utilizes the complementarity of multimodal information by fusing spectral features and image features converted from the spectrum, overcoming the deficiency of insufficient single modal information and significantly improving the accuracy of microplastic identification;

[0046] 2. Enhanced model robustness: This paper combines multi-perspective probabilistic fusion of spectral features and image features with the feature learning and classification capabilities of a dual-path attention residual convolutional network model, making the model more adaptable and stable to complex soil background interference and changes in microplastic concentration;

[0047] 3. Low-concentration detection is achieved: The detection method of the present invention can maintain high detection performance even when the concentration of microplastics is low (for example, 0.5%), which is of great significance for the discovery and assessment of early pollution in actual environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0049] Figure 1 This is a flowchart of a soil plastic detection method based on multimodal fusion and deep learning provided by one embodiment of the present invention;

[0050] Figure 2 This is a diagram of the architecture of a dual-path attention residual convolutional network model provided by one embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of two stages provided by one embodiment of the present invention;

[0052] Figure 4 This is a block diagram of a soil plastic detection system based on multimodal fusion and deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0054] Example 1

[0055] Figure 1 This is a flow chart of a soil plastic detection method based on multimodal fusion and deep learning provided by one embodiment of the present invention. Figure 1 As shown, this embodiment provides a soil plastic detection method based on multimodal fusion and deep learning, the method comprising:

[0056] Step S10: Acquire one-dimensional hyperspectral data of the soil sample to be tested.

[0057] In this embodiment, a soil microplastic sample is first prepared. For example, surface soil from a specific area is collected, air-dried, decontaminated, ground, and sieved. Common microplastic types are selected, such as polyethylene (PE), polypropylene (PP), and polyvinyl chloride (PVC), and their particle size range can be set to 100 to 500 μm. To simulate different pollution levels, multiple concentration gradients (such as 0.5% to 10%) can be set, and a pure soil control group can be included. Then, a hyperspectral imaging system (such as a near-infrared hyperspectral sorter) is used to collect hyperspectral image data of the sample. The system typically includes a light source, a spectrometer (such as a CCD camera with a grating), an electrically controlled mobile platform, and data acquisition software. The wavelength range, for example, covers 387-1034 nm. The collected data is one-dimensional hyperspectral curve data for each pixel. The collected raw one-dimensional hyperspectral data can be preprocessed to reduce noise, baseline drift, and scattering effects to obtain the one-dimensional hyperspectral data of the soil sample to be tested required in this embodiment.

[0058] The preprocessing of the original one-dimensional hyperspectral data includes the following steps:

[0059] 1. Avitzky-Golay (SG) smoothing: for example, window size 11, polynomial order 2, used to remove noise;

[0060] 2. Standard Normal Variable Transformation (SNV): used to correct spectral variations caused by particle size and surface scattering;

[0061] 3. First-order derivative calculation: used to eliminate baseline drift and enhance absorption characteristics.

[0062] Step S20: converting the one-dimensional hyperspectral data based on an image conversion algorithm to obtain a two-dimensional image representation.

[0063] In this embodiment, the image conversion algorithm is at least one of continuous wavelet transform, recursive graph and Gramian angle field.

[0064] Among them, continuous wavelet transform (CWT) convolves the spectral signal with wavelet mother functions of different scales and displacements to generate a time-frequency diagram that shows the intensity of the spectrum at different wavelength positions and the fluctuation characteristics at different scales.

[0065] Among them, the recurrence plot (RP) calculates the distance between pairs of data points in the spectral sequence and generates a binary image based on a threshold. Its structural pattern can reveal the periodicity, stability and mutation of the sequence.

[0066] Among them, Gramian Angular Field (GAF): encodes the normalized spectrum into polar coordinates and constructs an image through trigonometric functions of paired angles, preserving time dependence and numerical correlation.

[0067] This embodiment uses an image conversion algorithm to map one-dimensional spectral information to a two-dimensional image space from different angles (such as time-frequency characteristics, dynamic behavior, and time correlation), which can reveal deep features that are difficult to detect in the original spectrum.

[0068] Step S30: extracting features from the one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features.

[0069] In this embodiment, the first preset algorithm is at least one of a continuous projection algorithm, an uninformative variable elimination method, a genetic algorithm, a principal component analysis, and an isometric mapping.

[0070] Among them, the Successive Projection Algorithm (SPA) aims to select the subset of bands with the least collinearity and effectively extract key information;

[0071] Among them, the uninformative variable elimination method (UVE) eliminates uninformative variables based on the stability of the partial least squares regression coefficients;

[0072] Among them, genetic algorithm (GA): a heuristic search algorithm used to optimize feature subsets;

[0073] Among them, principal component analysis (PCA): a classic linear dimensionality reduction method that maximizes the projected variance;

[0074] Among them, Isomap: a nonlinear dimensionality reduction method that maintains the geodesic distance between data points.

[0075] Through experimental evaluation in this embodiment, SPA can generally achieve better performance. The first preset algorithm of this embodiment preferably adopts the successive projection algorithm (SPA); this embodiment uses the first preset algorithm to extract features and reduce the dimension of one-dimensional spectral data to obtain spectral features.

[0076] Step S40: extracting features from the two-dimensional image representation based on a second preset algorithm to obtain image features.

[0077] In this embodiment, the second preset algorithm is at least one of a histogram of oriented gradients, a local binary pattern, a generalized search tree, a Gabor filter feature, and an edge gradient feature.

[0078] Among them, the Histogram of Directed Gradients (HOG) constructs features by calculating and counting the histogram of gradient directions in local areas of the image, which can well describe the edges and shapes of the image;

[0079] Among them, Local Binary Pattern (LBP): an operator that describes the local texture features of an image;

[0080] Among them, Generalized Search Tree (GIST): used to capture the global scene structure of the image;

[0081] Among them, Gabor filter: a linear filter used for edge detection and texture analysis, which can extract multi-scale and multi-directional texture features;

[0082] Among them, edge gradient feature (EGF) extracts structural features based on edge detection and gradient information.

[0083] Through experimental evaluation in this embodiment, HOG can usually achieve better performance. The second preset algorithm of this embodiment preferably uses the histogram of oriented gradients (HOG) to capture the texture, structure and shape information represented by the two-dimensional image to obtain image features.

[0084] Step S50: performing multi-view probability fusion on the spectral features and the image features to obtain fused features.

[0085] In this embodiment, the core idea of ​​multi-view probabilistic fusion is to regard the features from the two modalities of spectrum and image as different "views" and to achieve adaptive weighted fusion by learning the contribution probability of the features at each view to the classification task.

[0086] Therefore, the spectral features and image features are fused with multi-view probability to obtain fused features, including:

[0087] Step S501: performing standardization processing on the spectral features and the image features to obtain processed spectral features and processed image features, so that the processed spectral features and the processed image features have similar scales and distributions.

[0088] Step S502: construct two basic classifiers, and train the two basic classifiers to obtain two trained basic classifiers.

[0089] In this embodiment, a set of basic classifiers is trained independently for each modality (spectral view and image view). For example, a support vector machine (SVM), random forest (RF), multi-layer perceptron (MLP), etc. can be selected as the basic classifier, and the sample data is used as the training set to train the basic classifier.

[0090] For each sample in the training set, its spectral features are input into the basic classifier of the spectral perspective to obtain the probability prediction vector P of the sample belonging to each predefined category. spec ;

[0091] Similarly, its image features are input into the basic classifier of the image perspective to obtain the corresponding category probability prediction vector P img ;

[0092] After training is completed, save all trained basic classifiers.

[0093] Step S503: input the processed spectral features into one of the trained basic classifiers to obtain a spectral view category probability prediction vector.

[0094] Step S504: input the processed image features into another trained basic classifier to obtain an image view category probability prediction vector.

[0095] In this embodiment, the new sample to be tested (processed spectral features and processed image features) is input into the trained basic classifier to obtain the spectral view category probability prediction vector and image view category probability prediction vector

[0096] Step S505: concatenate the spectral view category probability prediction vector and the image view category probability prediction vector to obtain a fusion feature.

[0097] Concatenate these two groups (or multiple groups, if multiple base classifiers are used for each modality) of probability prediction vectors to form an extended fusion probability feature vector F fusion :

[0098]

[0099] This F fusion This is the fusion feature of the dual-path attention residual convolutional network model that is finally input to the downstream.

[0100] Compared with directly concatenating the original high-dimensional features, the fused features of this embodiment are more resistant to the influence of feature scale and distribution differences, and utilize the discriminant information of each modality classifier.

[0101] Step S60: Input the fused features into a pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil sample to be tested.

[0102] The dual-path attention residual convolutional network (DAR-CNN) model of this embodiment is used to perform the fusion feature F obtained in step S50. fusion For the final classification, the structure of the dual-path attention residual convolutional network (DAR-CNN) model is as follows Figure 2 As shown in Figure 3, the dual-path attention residual convolutional network (DAR-CNN) model includes an input module, a convolutional neural network module, a multi-head attention module, a dual-branch processing module, and an output module connected in sequence.

[0103] The input module is used to receive the fusion feature F obtained in step S50. fusion , a normalization process may be performed first.

[0104] Among them, the convolutional neural network module is composed of several one-dimensional convolutional layers and one-dimensional maximum pooling layers stacked together, which is used to extract local features of the fused features to obtain local features.

[0105] Among them, the multi-head attention module is used to capture the long-distance dependencies between local features based on the attention mechanism and output attention features.

[0106] The formula for attention calculation is:

[0107]

[0108] Where Q is the query vector, K is the key vector, V is the value matrix, T is the transposed symbol, and d k is the dimension of the key vector.

[0109] The output of MHSA (Multi-Head Attention Module) is usually connected to the input of MHSA or Conv through a residual connection. att The outputs of the convolutional neural network module are added and then layer normalized to obtain the global context feature H att (Attention characteristics).

[0110] Global context feature H att The calculation expression is:

[0111] H att =LN(Conv att (X in )+Multi Head(Conv att (X in )));

[0112] Where, X in is the input of the module, i.e., local features, MultiHead is the multi-head attention function, and LN is the normalization function.

[0113] The dual-branch processing module includes: a main branch and a sub-branch. The main branch performs further nonlinear transformation and feature learning through one or more fully connected layers and Dropout layers, and outputs a set of Z main (i.e., main branch features), whose dimension is equal to the number of categories; the secondary branch also outputs another set of Z through another set of similar structures (fully connected layers and Dropout layers) aux (i.e., secondary branch characteristics).

[0114] Among them, the output module is used for:

[0115] First, Z of the main branch main and Z of the auxiliary branch aux Perform weighted fusion to obtain the final feature: Z final =w main Z main +w aux Z aux , where weight w main and w aux Can be preset or learnable.

[0116] Then, the final feature Z final Input into the activation function (preferably using the Softmax function) to calculate the probability distribution P of each category: P(y=i|Z final ), thereby completing the classification and discrimination; the category with the highest probability is the predicted microplastic type (or soil) of the sample.

[0117] Finally, based on the output probability of the DAR-CNN model, it is determined whether microplastics exist in the sample, as well as the type of microplastics (such as PE, PP, PVC or their mixture).

[0118] Therefore, in step S60, the fused features are input into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil sample to be tested, including:

[0119] Step S601: the input module receives fusion features;

[0120] Step S602: The convolutional neural network module extracts local features from the fused features to obtain local features;

[0121] Step S603: The multi-head attention module captures the long-range dependency of local features to obtain attention features;

[0122] Step S604: the dual-branch processing module performs multi-scale learning on the attention features to obtain main branch features and secondary branch features;

[0123] Step S605: The output module performs weighted fusion on the main branch features and the secondary branch features to obtain the final features, and inputs the final features into the activation function for classification to obtain the category probability distribution, and determines the detection result based on the category probability distribution; therefore, the detection result is whether microplastics exist in the sample and the type of microplastics.

[0124] As a further optimization of this embodiment, before executing steps S10 to S60, the basic classifier and the DAR-CNN model need to be trained, such as Figure 3 As shown, it includes two stages, a training stage and a detection stage. The detection stage includes the contents of steps S10 to S60. The steps of the training stage are as follows:

[0125] 1. Dataset preparation: Collect a large amount of hyperspectral data of soil samples with accurate labels (microplastic type and concentration); divide the dataset into training set, validation set and test set.

[0126] 2. Basic classifier training: Use the training set to train each basic classifier (SVM, RF, MLP, etc.) on the data after spectral feature extraction and image feature extraction.

[0127] 3. DAR-CNN model training:

[0128] ① Using the training set, first obtain the fusion features through the multi-view probability fusion (MPFF) process;

[0129] ② These fused features are used as the input of the DAR-CNN model, and the true labels of the samples are used as supervision signals.

[0130] Use appropriate loss functions (such as cross entropy loss) and optimizers (such as Adam) for backpropagation and parameter updates until the model achieves satisfactory performance on the validation set.

[0131] Therefore, the present invention utilizes the complementarity of multimodal information by fusing spectral features and image features converted from the spectrum, overcomes the defect of insufficient single modal information, and significantly improves the recognition accuracy of microplastics; by performing multi-perspective probabilistic fusion of spectral features and image features, combined with the feature learning and classification capabilities of the dual-path attention residual convolutional network model, the model has better adaptability and stability to complex soil background interference and changes in microplastic concentration; the present invention can still maintain a high detection performance even when the microplastic concentration is low (for example, 0.5%), which is of great significance for the discovery and assessment of early pollution in actual environments; and the multi-perspective probabilistic fusion MPFF and dual-path attention residual convolutional network DAR-CNN model of the present invention provide new ideas and technical support for the fields of hyperspectral data analysis and multimodal information fusion, and are particularly suitable for the accurate identification of target substances in complex systems.

[0132] Example 2

[0133] Figure 4 This is a soil plastic detection system based on multimodal fusion and deep learning provided by an embodiment of the present invention. Figure 4 As shown, this embodiment provides a soil plastic detection system based on multimodal fusion and deep learning. The system is used to implement the soil plastic detection method based on multimodal fusion and deep learning in Example 1. The system includes:

[0134] A data acquisition module is used to obtain one-dimensional hyperspectral data of the soil sample to be tested;

[0135] A data conversion module is used to convert one-dimensional hyperspectral data into a two-dimensional image representation based on an image conversion algorithm;

[0136] A spectral feature extraction module is used to extract features from one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features;

[0137] An image feature extraction module, configured to extract features from the two-dimensional image representation based on a second preset algorithm to obtain image features;

[0138] Feature fusion module, used to perform multi-view probability fusion of spectral features and image features to obtain fused features;

[0139] The classification and detection module is used to input the fused features into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested.

[0140] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned soil plastic detection method based on multimodal fusion and deep learning is implemented.

[0141] This embodiment also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the above-mentioned soil plastic detection method based on multimodal fusion and deep learning is implemented.

[0142] The present invention utilizes the complementarity of multimodal information by fusing spectral features and image features converted from the spectra, overcomes the defect of insufficient single modal information, and significantly improves the recognition accuracy of microplastics; by multi-perspective probabilistic fusion of spectral features and image features, combined with the feature learning and classification capabilities of the dual-path attention residual convolutional network model, the model has better adaptability and stability to complex soil background interference and changes in microplastic concentration; the present invention can still maintain a high detection performance even when the microplastic concentration is low (for example, 0.5%), which is of great significance for the discovery and assessment of early pollution in actual environments; and the multi-perspective probabilistic fusion MPFF and dual-path attention residual convolutional network DAR-CNN model of the present invention provide new ideas and technical support for the fields of hyperspectral data analysis and multimodal information fusion, and are particularly suitable for the accurate identification of target substances in complex systems.

[0143] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0145] Example 3

[0146] In order to objectively measure the performance of the soil plastic detection method based on multimodal fusion and deep learning proposed in this embodiment, standard evaluation indicators such as accuracy (Acc), precision (P), recall (R), F1 score (F1) and relative generalization gap (RGG) are introduced for evaluation.

[0147] To establish baseline performance, the core DAR-CNN model of the present invention was compared with traditional support vector machines (SVMs), back-propagation neural networks (BPNNs), and one-dimensional convolutional neural networks (1D-CNNs) using only preprocessed raw spectral data. Table 1 shows a comprehensive performance evaluation of the four basic models. The results show that the DAR-CNN model used in the present invention achieved an accuracy (Acc) of 79.32%, significantly higher than the SVM, BPNN, and 1D-CNN models. Its precision (P), recall (R), and F1 score were also significantly better than those of the other benchmark models. This demonstrates that the DAR-CNN model has demonstrated significant performance superiority in processing basic spectral information.

[0148] Table 1

[0149]

[0150] In the performance evaluation of multimodal feature extraction methods, we first evaluated the classification performance of five commonly used methods (SPA, UVE, GA, PCA, and Isomap) combined with the DAR-CNN model for spectral feature extraction. Table 2 shows the performance evaluation of the spectral feature extraction methods. The SPA method performed best, achieving an accuracy of 93.27%, an F1 score of 93.98%, and a relative generalization gap (RGG) of only 3.90%, demonstrating the effectiveness of the SPA method for spectral feature extraction.

[0151] Table 2

[0152]

[0153] Subsequently, after converting the one-dimensional spectral data into a two-dimensional image (e.g., CWT), the classification performance of five image feature extraction methods (HOG, LBP, GIST, Gabor, and EGF) combined with the DAR-CNN model was evaluated. Table 3 shows the evaluation of the image feature extraction methods. HOG features performed best, achieving an accuracy of 84.97% and an F1 score of 84.29%, demonstrating that HOG features can effectively extract discriminative information from spectrally converted images.

[0154] Table 3

[0155]

[0156]

[0157] In the MPFF feature fusion performance evaluation stage, the spectral features (such as SPA, UVE) and image features (such as HOG, Gabor) that performed best in the above evaluation were selected for different combinations, and the performance of five fusion methods (Concat, PCA-F, WF, MKL, MPFF) was compared. Table 4 shows the performance comparison of different feature combinations under different fusion methods. The results clearly show that for all tested feature combinations, the MPFF method proposed in the present invention generally exhibits the highest classification accuracy, precision, recall rate and F1 score, and has the lowest RGG. For example, when fusing SPA and HOG features, the accuracy achieved by the MPFF method is 96.75%, which is significantly better than other compared fusion methods. The MPFF configuration that integrates all four preferred features (SPA, UVE, HOG, Gabor) is further analyzed. Table 5 is a feature contribution analysis table. The results show that spectral features (especially SPA) have the highest contribution weight to the model. Using this optimally configured MPFF+DAR-CNN model, the final classification accuracy achieved reached 96.75%, fully demonstrating the superiority of the multimodal feature fusion strategy proposed in this invention.

[0158] Table 4

[0159]

[0160]

[0161] Table 5

[0162]

[0163] To evaluate the practicality of the proposed method, its detection performance under varying microplastic concentrations was specifically examined. The classification performance of the proposed method (MPFF+DAR-CNN, using the optimal configuration of all four preferred feature fusions) was tested under seven pre-determined microplastic concentration gradients (ranging from 0.5% to 10%). Table 6 compares the detection performance at different concentrations. The experimental results show that the model's performance metrics (accuracy, precision, recall, and F1 score) steadily improve with increasing microplastic concentration. Notably, even at the lowest concentration of 0.5%, the model maintains high accuracy and F1 scores for detecting different microplastic types. For example, the average accuracy for a single microplastic type is approximately 86%. When the concentration increases to 3%, the average accuracy for a single type exceeds 95%. At the highest concentration of 10%, the model's detection accuracy for all types exceeds 98%. These data strongly demonstrate that the proposed method exhibits excellent detection performance across various soil microplastic concentrations and possesses excellent low-concentration detection capabilities, demonstrating its strong potential for practical application.

[0164] Table 6

[0165]

[0166] In summary, through a series of detailed experimental verifications, the soil plastic detection method based on multimodal fusion and deep learning proposed in this invention has shown significant advantages in detection accuracy, robustness, and the ability to detect low-concentration samples, providing a feasible and high-performance technical solution for the rapid and accurate monitoring of soil microplastic pollution.

[0167] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A soil plastic detection method based on multimodal fusion and deep learning, characterized in that: The method comprises: Obtain one-dimensional hyperspectral data of the soil sample to be tested; Based on the image conversion algorithm, the one-dimensional hyperspectral data is converted into a two-dimensional image representation; Performing feature extraction on the one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features; Performing feature extraction on the two-dimensional image representation based on a second preset algorithm to obtain image features; Perform multi-view probability fusion on spectral features and image features to obtain fusion features; The fused features are input into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested.

2. The soil plastic detection method based on multimodal fusion and deep learning according to claim 1 is characterized in that: The image conversion algorithm is at least one of continuous wavelet transform, recursive graph and Gramian angle field.

3. The soil plastic detection method based on multimodal fusion and deep learning according to claim 1 is characterized in that: The first preset algorithm is at least one of a continuous projection algorithm, an uninformative variable elimination method, a genetic algorithm, a principal component analysis, and an isometric mapping.

4. The soil plastic detection method based on multimodal fusion and deep learning according to claim 1 is characterized in that: The second preset algorithm is at least one of a histogram of oriented gradients, a local binary pattern, a generalized search tree, a Gabor filter feature, and an edge gradient feature.

5. The soil plastic detection method based on multimodal fusion and deep learning according to claim 1, characterized in that: The spectral features and image features are fused with multi-view probability to obtain fused features, including: Performing standardization processing on the spectral features and the image features to obtain processed spectral features and processed image features; Construct two basic classifiers and train the two basic classifiers to obtain two trained basic classifiers; The processed spectral features are input into one of the trained basic classifiers to obtain the spectral view class probability prediction vector; The processed image features are input into another trained basic classifier to obtain the image view category probability prediction vector; The spectral view category probability prediction vector and the image view category probability prediction vector are concatenated to obtain the fusion feature.

6. The soil plastic detection method based on multimodal fusion and deep learning according to claim 5 is characterized in that: The basic classifier is at least one of a support vector machine, a random forest and a multi-layer perceptron.

7. The soil plastic detection method based on multimodal fusion and deep learning according to claim 1, characterized in that: The dual-path attention residual convolutional network model includes an input module, a convolutional neural network module, a multi-head attention module, a dual-branch processing module and an output module connected in sequence; The fused features are input into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested, including: The input module receives the fusion feature; The convolutional neural network module performs local feature extraction on the fusion features to obtain local features; The multi-head attention module captures the long-distance dependencies of local features to obtain attention features; The dual-branch processing module performs multi-scale learning on the attention features to obtain main branch features and secondary branch features; The output module performs weighted fusion on the main branch features and the secondary branch features to obtain the final features, and inputs the final features into the activation function for classification to obtain the category probability distribution, and determines the detection results based on the category probability distribution.

8. A soil plastic detection system based on multimodal fusion and deep learning, used to implement the soil plastic detection method based on multimodal fusion and deep learning according to any one of claims 1 to 7, characterized in that: The system comprises: A data acquisition module is used to obtain one-dimensional hyperspectral data of the soil sample to be tested; A data conversion module is used to convert one-dimensional hyperspectral data into a two-dimensional image representation based on an image conversion algorithm; A spectral feature extraction module is used to extract features from one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features; An image feature extraction module, configured to extract features from the two-dimensional image representation based on a second preset algorithm to obtain image features; Feature fusion module, used to perform multi-view probability fusion of spectral features and image features to obtain fused features; The classification and detection module is used to input the fused features into the pre-trained dual-path attention residual convolutional network model for classification to obtain the detection results of microplastics in the soil samples to be tested.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the soil plastic detection method based on multimodal fusion and deep learning described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the soil plastic detection method based on multimodal fusion and deep learning described in any one of claims 1 to 7 is implemented.

Images