Micro-plastic intelligent detection method and system based on Transform-CNN (Convolutional Neural Network) hybrid model and storage medium

Abstract

The invention discloses an intelligent micro-plastic detection method and system based on a Transform-CNN hybrid model and a storage medium. The method comprises the following steps: S100, collecting micro-plastic image data of an environmental water sample; s200, performing local feature extraction on the micro-plastic image by using a CNN module; s300, the extracted local features are input into a Transform module, and global features are extracted; and S400, fusing the local features and the global features, and carrying out classification identification on the micro-plastics based on the fused features. The problems that an existing single model is insufficient in global feature extraction and insufficient in local detail feature recognition can be effectively solved, and the accuracy and precision of micro-plastic detection are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of pollutant detection technology, and in particular to a method, system and storage medium for intelligent detection of microplastics based on a Transformer-CNN hybrid model. Background Technology

[0002] As an emerging pollutant, microplastics have attracted global attention due to their environmental risks. Currently, the main technical methods for detecting microplastics can be divided into two categories: traditional physicochemical analysis methods and image recognition methods based on deep learning. The former includes Fourier transform infrared spectroscopy (FTIR), Raman spectroscopy, and thermal decomposition gas chromatography-mass spectrometry (Py-GC / MS). Although these methods are accurate and reliable in qualitative identification, they have obvious technical defects and limitations in practical application: (1) the instruments and equipment are expensive and the technical threshold is high, making them difficult to popularize in ordinary laboratories; (2) the sample pretreatment steps are complex and the detection process is time-consuming, which cannot meet the needs of rapid on-site detection and emergency monitoring; (3) some methods have limited detection sensitivity and insufficient accuracy in identifying low concentrations of microplastics in the environment. In recent years, with the development of artificial intelligence, some studies have attempted to introduce convolutional neural networks (CNN) to automatically identify microplastic images in order to improve detection efficiency.

[0003] However, the inherent technical limitations of CNN models make it difficult to meet the stringent requirements of microplastic detection in practical application scenarios. Specifically, this manifests in the following ways: (1) CNNs focus on local feature extraction, but lack the ability to recognize the overall spatial structure of images and capture long-distance dependent features, which can easily lead to missed or false detections when microplastics have diverse morphologies; (2) In complex environments, the recognition performance of CNNs drops significantly, and their generalization performance is poor, making it difficult to effectively handle real-world environmental samples. Although the Transformer model can effectively capture the global features of data through its self-attention mechanism, its sensitivity to image detail features is limited, and when applied alone to microplastic detection, it suffers from insufficient local information extraction. This makes it difficult for current single-model methods to simultaneously possess good recognition capabilities for both local and global spatial features. Summary of the Invention

[0004] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention proposes a microplastic intelligent detection method, system, and storage medium based on a Transformer-CNN hybrid model, which can effectively solve the problems of insufficient global feature extraction and inadequate local detail feature recognition in existing single models, and significantly improve the accuracy and precision of microplastic detection.

[0005] According to a first aspect of the present invention, a method for intelligent detection of microplastics based on a Transformer-CNN hybrid model includes: S100: Collect microplastic image data from environmental water samples; S200. Use the CNN module to extract local features from microplastic images; S300. Input the extracted local features into the Transformer module to extract global features; S400. The local features and the global features are fused together, and the microplastics are classified and identified based on the fused features.

[0006] According to some embodiments of the present invention, in step S200, the CNN module includes multiple convolutional layers, batch normalization layers and pooling layers, and includes the following steps; S201. Each convolutional layer scans the microplastic image through a convolution kernel to extract detailed information of the microplastic image, wherein the convolution formula is: in: : indicates the first l The position after the convolution operation is eigenvalues; : Represents the feature map output from the previous layer; : Represents the weight coefficients of the convolution kernel; : Indicates the bias term; : Represents a non-linear activation function; S202. The batch normalization layer is used to standardize the convolutional feature map. The normalization formula is: in: : Represents input features; and : These represent the mean and variance of the batch features, respectively; : Represents a small constant to prevent division by zero; S203. Perform pooling operation through the pooling layer.

[0007] According to some embodiments of the present invention, the CNN module further includes a position-aware module, which dynamically adjusts the receptive field of the convolution kernel through deformable convolution to enhance adaptability to irregular shapes of microplastics, including the following steps: The position-aware module dynamically adjusts the sampling position of the convolution kernel through deformable convolution for the input feature map. X Fixed offset of standard convolution pk,l Replaced with a learnable offset △ p k,l Its calculation form is: in; Y ( p ): Indicates the output feature map at position p Pixel value at; W ( k,l ): Indicates the position of the convolution kernel. (k,l The weight value at () X (·): Represents the pixel value function of the input feature map, indicating that a pixel at a certain position is taken from the input image; P : Represents the original image coordinates corresponding to the current output position; p k,l : indicates the (th)th term in the convolution kernel k，l The offset of ) fixed sampling positions; Δ p k,l : Indicates the offset.

[0008] According to some embodiments of the present invention, in step S300, inputting the extracted local features into the Transformer module to extract global features includes: The feature map output by the CNN module is flattened into a one-dimensional feature sequence and fed into the Transformer module. Global features are extracted through a self-attention mechanism, the output of which is calculated by the following formula: in: ; ; : Indicates the attention value used for scaling factor normalization; , , : Represents a trainable weight matrix; Given an input feature sequence, n For sequence length, d For feature dimensions.

[0009] According to some embodiments of the present invention, in step S400, fusing the local features and the global features to classify and identify the microplastic image includes: S401: The local spatial feature vector output by the CNN feature extraction module. Global spatial feature vectors extracted by the Transformer module The features are concatenated and merged to generate a unified high-dimensional feature vector. : S402: Merged features The input is fed into a fully connected layer, and after nonlinear mapping, the Softmax classification function is used to predict the microplastic category. The output probability of the Softmax classifier is calculated using the following formula: in: : Represents the output of the fully connected layer. Class score; : Indicates the total number of categories for microplastic classification; : Indicates that the input feature belongs to the th The probability of a class.

[0010] According to some embodiments of the present invention, it further includes: S500. Optimize the parameters of the hybrid CNN and Transformer model through deep learning model training, including the following steps: The cross-entropy loss function is used, and its expression is as follows: in: The actual label vector; The class probabilities predicted by the model; The total number of categories; The parameters are optimized using the stochastic gradient descent method with momentum, as shown in the following formula: Gradient update formula: Parameter update formula: in: This is the momentum term, representing the cumulative effect of historical gradients; Momentum factor; The learning rate; These are model parameters; For the loss function with respect to the parameters The gradient.

[0011] According to some embodiments of the present invention, it further includes: S600, applying the optimized model to the actual environmental image, outputting the microplastic identification result, and cross-validating it with the traditional standard detection method.

[0012] According to some embodiments of the present invention, step S100 further includes performing data augmentation on the acquired images through image preprocessing to expand the size of the training dataset.

[0013] According to some embodiments of the present invention, the data enhancement of the acquired image through image preprocessing includes cropping, rotation, flipping, brightness and contrast adjustment.

[0014] According to a second aspect of the present invention, a microplastic intelligent detection system includes a memory and a processor. The memory stores a determination program for a microplastic intelligent detection method based on a Transformer-CNN hybrid model. The processor runs the determination program for the microplastic intelligent detection method based on the Transformer-CNN hybrid model, so that the microplastic intelligent detection system performs the microplastic intelligent detection method based on the Transformer-CNN hybrid model.

[0015] According to a third aspect of the present invention, a computer-readable storage medium includes: a determination program for a microplastic intelligent detection method based on a Transformer-CNN hybrid model is stored on the computer-readable storage medium, wherein when the determination program for the microplastic intelligent detection method based on a Transformer-CNN hybrid model is executed by a processor, the microplastic intelligent detection method based on a Transformer-CNN hybrid model is implemented.

[0016] The microplastic intelligent detection method, system, and storage medium based on the Transformer-CNN hybrid model according to embodiments of the present invention have at least the following beneficial effects: The core technical idea of ​​the present invention lies in enhancing the ability to capture the global spatial features and long-distance dependencies of microplastics in images through the self-attention mechanism of the Transformer network, while utilizing the convolutional layers of the CNN network to efficiently extract and accurately identify local detail features of the image, achieving synergistic fusion of local and global features. First, the spatial detail features of the microplastic image are extracted through the CNN module, and then the extracted local features are input into the Transformer module to obtain the overall spatial structural features of microplastics in complex environmental backgrounds, ultimately completing high-precision automatic identification and detection of microplastics. This effectively solves the problems of insufficient global feature extraction and inadequate local detail feature recognition in existing single models, significantly improving the accuracy and precision of microplastic detection. It enhances the model's adaptability and generalization ability to complex environmental backgrounds, reducing false detection and false negative rates. It achieves efficient and rapid intelligent detection, which can be widely applied in the fields of environmental monitoring and pollution control, promoting the advancement of microplastic pollution control technology.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein: Figure 1 This is a flowchart of a microplastic intelligent detection method based on a Transformer-CNN hybrid model according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a microplastic intelligent detection system according to an embodiment of the present invention. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0020] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the drawings and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0021] In the description of this invention, "multiple" refers to two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features or their sequential relationship.

[0022] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0023] Terminology Explanation: (1) Microplastics: refers to plastic particles or fibers with a diameter or length of less than 5 mm. They are characterized by their small size and diverse shapes. They are widely present in the environment and are considered new pollutants.

[0024] (2) Transformer: A deep learning model based on self-attention mechanism, which can efficiently capture global features and long-distance dependencies of input data and is widely used in image recognition, natural language processing and other fields.

[0025] (3) CNN (Convolutional Neural Network): A deep learning neural network structure that effectively extracts local spatial features of data through convolution operations, and is particularly suitable for image processing tasks.

[0026] (4) Self-attention mechanism: a mechanism used in machine learning models to capture the inherent global dependencies of data by calculating the degree of correlation between different parts of the input sequence.

[0027] (5) Generalization ability: refers to the model’s ability to accurately predict unseen data or data from different scenarios, reflecting the model’s reliability and adaptability in practical applications.

[0028] Reference Figure 1 As shown, this invention discloses a microplastic intelligent detection method based on a Transformer-CNN hybrid model, comprising: S100: Collect microplastic image data from environmental water samples; S200. Use the CNN module to extract local features from microplastic images; S300. Input the extracted local features into the Transformer module to extract global features; S400: Fuse local and global features, and classify and identify microplastics based on the fused features.

[0029] In this embodiment, firstly, microplastic image data of environmental water samples is acquired. Image preprocessing methods are then used to augment the acquired images, including cropping, rotation, flipping, and brightness and contrast adjustments, thereby expanding the training dataset. Expanding the dataset through data augmentation increases image diversity and significantly improves the model's generalization ability to real-world images, effectively reducing overfitting caused by insufficient data in existing technologies.

[0030] It should be noted that wavelet thresholding denoising is added to the image preprocessing in this embodiment to effectively eliminate optical noise interference in the environmental image. The processing formula is as follows: in σ is the wavelet coefficient, T is the threshold, σ is the noise standard deviation, and N is the signal length.

[0031] The preprocessed microplastic images are then subjected to preliminary feature extraction using a convolutional neural network (CNN) module. Specifically, this module consists of multiple convolutional layers, batch normalization layers, and pooling layers, which process the image data sequentially. Each convolutional layer scans the image through a convolutional kernel (filter) to extract detailed information such as edges, textures, and colors.

[0032] CNNs effectively extract detailed features of microplastic particles or fibers in images, such as shape, texture, and edges, through local connectivity and weight sharing. Taking actual microplastic images as an example, when microplastic particles exhibit different surface morphologies or edge contours, CNN convolutional kernels accurately extract these detailed information through a sliding window approach, providing rich fine-grained features for subsequent Transformer modules.

[0033] The activation functions (such as ReLU) used in CNN modules enhance the model's feature representation ability through non-linear mapping, effectively mining the hidden features of image data and ensuring the diversity of feature extraction. For example, for microplastic samples of different types or sizes, the feature maps processed by the activation function can more clearly highlight the differences, thereby improving recognition accuracy.

[0034] The Transformer global feature extraction module of this invention aims to further extract the overall spatial information and long-distance dependency features of the output features of the CNN module, thereby enhancing the model's ability to understand the overall structure of microplastic images.

[0035] Its advantages are: 1. Traditional CNN models, due to their limited convolution operation window, tend to focus on local features while neglecting global contextual information. The Transformer module, through its self-attention mechanism, ensures that the feature representation at each location fully considers the relationships between features at all other locations. For example, in a microplastic image, the same type of particles may be distributed in different locations. The self-attention mechanism can establish dependencies between these locations, achieving globally consistent recognition of features of the same type.

[0036] 2. Through the self-attention mechanism, the Transformer model quantifies the degree of correlation between any two positions in the input sequence, thereby capturing long-distance dependencies in the image.

[0037] 3. In this invention, the Transformer employs multiple self-attention layers stacked together. Through layer-by-layer feature mapping and nonlinear transformation, it further enhances the abstraction and expressive power of features. For example, the first-layer Transformer module only recognizes basic global feature relationships, and then gradually progresses to higher-order abstract features, such as more complex information like material differences and microplastic type distinctions, thereby significantly improving the model's recognition detail and classification accuracy.

[0038] 4. Pooling operations in the later stages of a CNN module can lead to the loss of some spatial information. However, the Transformer module can effectively compensate for the information loss that may occur during feature dimensionality reduction in CNNs through global attention. For example, the features of some microplastic particles may be weakened or ignored after dimensionality reduction by CNNs. The Transformer's self-attention mechanism strengthens these easily overlooked feature points by focusing on the global information of all features in the image.

[0039] The fusion of features from CNNs and Transformers aims to compensate for the shortcomings of feature extraction from a single model, and the two have significant complementarity: (1) Advantages of local features (CNN): CNN is good at capturing local image features, such as the texture, edge and morphological features of microplastics; (2) Global features (Transformer) advantages: Transformer is good at capturing the overall structure and long-distance spatial relationships.

[0040] By merging and combining features, the final input classifier possesses the dual advantages of "local + global" features, achieving information complementarity between local and global features and minimizing the limitations of a single feature type. For example, if a certain type of microplastic has similar local features but vastly different spatial distributions (scattered locations) in an image, using a CNN alone might lead to misclassification. However, by fusing Transformer features, the model can effectively utilize global information to identify items belonging to the same category, significantly improving classification accuracy.

[0041] After feature fusion, the nonlinear mapping through a fully connected layer can further improve the discriminative power between different categories in the feature space, making it easier for the Softmax function to classify accurately.

[0042] Suppose that before the fused feature vectors are input into the fully connected layer, the feature vectors of different categories of microplastics in the feature space are close to each other, making classification difficult. After the nonlinear mapping of the fully connected layer, the feature space is remapped, and the previously difficult-to-distinguish category features become easier to distinguish, effectively improving classification accuracy.

[0043] This invention uses the Softmax classifier to achieve the final microplastic classification decision. Its probabilistic classification method can intuitively and clearly give the classification confidence of the model, improving the credibility and interpretability of the classification results.

[0044] The feature fusion module combines the local feature robustness of CNNs with the global feature generalization of Transformers, which greatly improves the generalization ability and robustness of the entire model in complex scenes and reduces the model's sensitivity to image noise and background interference.

[0045] In actual water sample image acquisition, there may be problems such as changes in lighting, water turbidity, or interference from background impurities. The fused model can combine local details and global information to perform effective classification and judgment, avoiding false detections or missed detections that may occur when using a single model.

[0046] In summary, the core technical idea of ​​this invention lies in enhancing the ability to capture the global spatial features and long-range dependencies of microplastics in images through the self-attention mechanism of the Transformer network, while simultaneously utilizing the convolutional layers of the CNN network to efficiently extract and accurately identify local detail features, achieving synergistic fusion of local and global features. This effectively solves the problems of insufficient global feature extraction and inadequate local detail feature recognition in existing single models, significantly improving the accuracy and precision of microplastic detection. It also enhances the model's adaptability and generalization ability to complex environmental backgrounds, reducing false positives and false negatives. This achieves efficient and rapid intelligent detection, which can be widely applied in environmental monitoring and pollution control, promoting the advancement of microplastic pollution control technologies.

[0047] In some embodiments of the present invention, in step S200, the CNN module includes multiple convolutional layers, batch normalization layers and pooling layers, and includes the following steps; S201. Each convolutional layer scans the microplastic image using a convolution kernel to extract detailed information about the microplastic image. The convolution formula is as follows: in: : indicates the first l The position after the convolution operation is eigenvalues; : Represents the feature map output from the previous layer; : Represents the weight coefficients of the convolution kernel; : Indicates the bias term; : Represents a non-linear activation function; S202. The convolutional feature map is standardized using a batch normalization layer. The normalization formula is: in: : Represents input features; and : These represent the mean and variance of the batch features, respectively; : Represents a small constant to prevent division by zero; S203. Perform pooling operations through the pooling layer.

[0048] In this embodiment, local information is extracted from the image through convolutional layers. This includes detailed features such as texture, edges, and color distribution. The convolutional layers of the CNN module abstract local features of the image layer by layer through multiple convolutional kernel sliding operations and activation functions, thus completing the initial information extraction. Then, batch normalization layers are used to standardize the convolutional feature maps to maintain data stability and reduce gradient vanishing. Finally, pooling operations are used to reduce the dimensionality of the feature maps, decrease the number of model parameters, suppress overfitting, and extract salient feature regions from the image. Pooling operations (such as max pooling) are mainly used to reduce the dimensionality and computational burden of the feature maps while preserving salient features. Feature compression can also be performed after extraction, and the extraction of local features mainly relies on the convolutional layers.

[0049] The purpose of pooling operations for feature compression is to reduce the dimensionality and redundancy of feature maps while retaining the main feature information, thereby providing a more compact, abstract, and computationally efficient input representation for subsequent Transformer and classification modules.

[0050] Simultaneously, it provides the Transformer with low-dimensional, high-information-density input. It also enhances the model's translation invariance and robustness. Pooling makes the model less sensitive to changes in the position of microplastics in the image; even with slight shifts in particle position, the pooled features remain largely consistent. This is crucial for real-world environmental images, as the distribution of microplastics in water samples is random.

[0051] It can also avoid feature redundancy and improve the efficiency of subsequent feature fusion. In subsequent steps, especially when fusing features with the Transformer output, if the CNN output is too large, it will result in excessively high feature dimensionality after concatenation, leading to sparse information and learning difficulties. Pooling operation compresses the feature map, making the fused representation denser and more representative, which facilitates effective discrimination by the Softmax classifier.

[0052] In some embodiments of the present invention, the CNN module further includes a position-aware module, which dynamically adjusts the receptive field of the convolution kernel through deformable convolution to enhance adaptability to irregular shapes of microplastics, including the following steps: The position-aware module dynamically adjusts the sampling position of the convolution kernel through deformable convolution for the input feature map. X Fixed offset of standard convolution p k,l Replaced with a learnable offset △ p k,l Its calculation form is: in; Y ( p ): Indicates the output feature map at position p Pixel value at; W ( k,l ): Indicates the position of the convolution kernel. (k,l The weight value at () X ( ): Represents the pixel value function of the input feature map, indicating the pixel value taken from a certain position in the input image; P : Represents the original image coordinates corresponding to the current output position; p k,l : indicates the (th)th term in the convolution kernel k，l The offset of ) fixed sampling positions; Δ p k,l : Indicates the offset.

[0053] In this embodiment, the Position-Aware Module further enhances the model's adaptability to irregular shapes of microplastics through Deformable Convolution. Deformable Convolution allows the convolutional kernel to adaptively adjust its receptive field according to the target shape, better capturing details such as edges or irregular contours. The Position-Aware Module is an optimized supplement to traditional convolution, improving deformation adaptability during the local feature recognition stage. Pooling operations (such as max pooling) are mainly used to reduce the dimensionality and computational burden of feature maps while preserving salient features. However, pooling is not used to "extract" local features, but rather to perform feature compression after extraction; the extraction of local features mainly relies on convolutional layers. Furthermore, the offset Δ p k,l Generated by a lightweight offset prediction network: This network consists of two 3×3 convolutional layers, and outputs 2K values ​​after inputting the feature map. 2 The offset field of the channel (K is the kernel size) is constrained to [-3, 3] by the Tanh function. When embedding into the CNN backbone, shallow layers (such as ResNet Stage2) deploy basic deformable convolutions to capture edge details, while deeper layers (such as Stage4) introduce modulation factors. By dynamically weighting different sampling points using the Sigmoid function, the perception of complex contours is enhanced. Here, σ is the Sigmoid function, enabling adaptive amplitude control of the offset. The CNN module employs a progressive training strategy: during the warm-up phase, the network parameters are fixed by an offset, and only the weights of the main network are optimized; during the joint training phase, the parameters are unfrozen, and the overall loss function is used for optimization. Simultaneously optimize detection accuracy and offset smoothness.

[0054] in: The loss of the main detection task, usually cross-entropy loss or IoU loss, is used to measure the accuracy of the model in identifying the location and category of microplastics. The squared L2 norm of the predicted offset values ​​in the offset network is used to constrain drastic changes in the offset and improve the smoothness and stability of sampling points in deformable convolution. λ: Weighting coefficient, used to balance the importance of detection accuracy and offset smoothing term. The larger this parameter is, the more the model emphasizes the smoothness of offset prediction; conversely, the smaller the parameter is, the more it focuses on detection accuracy.

[0055] This loss function, by jointly optimizing the detection loss and the offset smoothing term, can improve the model's detection accuracy and suppress jitter and instability in the offset prediction process, thereby improving the continuity and interpretability of the feature extraction process.

[0056] To improve the model's ability to recognize images with blurred edges or low contrast of microplastics, an edge enhancement module was introduced after the CNN convolutional layer. This module, based on the Canny edge detection algorithm and gradient magnitude fusion strategy, is used to enhance the boundary information of microplastic particles. in and Let Sobel gradients be the image's x and y directions, respectively. Then, concatenate the calculated edge map with the CNN output feature map using channel concatenation. By using the edge enhancement module, the model's recognition accuracy for microplastic images with unclear edges or those severely affected by environmental interference can be significantly improved.

[0057] In some embodiments of the present invention, step S300 involves inputting the extracted local features into the Transformer module to extract global features, including: The feature map output from the CNN module is flattened into a one-dimensional feature sequence and fed into the Transformer module. Global features are extracted through a self-attention mechanism. The output of the self-attention mechanism is calculated using the following formula: in: ; ; : Indicates the attention value used for scaling factor normalization; , , : Represents a trainable weight matrix; Given an input feature sequence, n For sequence length, d For feature dimensions.

[0058] In this embodiment, the Transformer global feature extraction module of the present invention aims to further extract the overall spatial information and long-distance dependency features of the output features of the CNN module, thereby enhancing the model's ability to understand the overall structure of microplastic images. Specifically, the feature map output by the CNN module is flattened into a one-dimensional feature sequence and fed into the Transformer module. The core mechanism of the Transformer is the self-attention mechanism, which is used to capture the global relationships between the overall features of the image, specifically defined as: Given an input feature sequence .in, For sequence length, As a feature dimension, it is transformed into a query matrix (Q), a key matrix (K), and a value matrix (V) through three different linear transformation matrices: , , is a trainable weight matrix.

[0059] The output of the self-attention mechanism is calculated by the following formula: in, The scaling factor is used to normalize the attention value, ensuring computational stability.

[0060] In practical implementation, to reduce computational cost, Window-MSA (Window-MSA) is introduced. Each feature map is divided into N windows, and self-attention is performed in each window. in This is a learnable relative position offset matrix. This structure can simplify the complex graph. Downgraded to .

[0061] Traditional CNN models, due to their limited convolution operation window, tend to focus on local features while neglecting global contextual information. The Transformer module, through its self-attention mechanism, ensures that the feature representation at each location fully considers the relationships between features at all other locations. For example, in a microplastic image, the same type of particles may be distributed in different locations. The self-attention mechanism can establish dependencies between these locations, achieving globally consistent recognition of similar features.

[0062] Through its self-attention mechanism, the Transformer model quantifies the correlation between any two positions in the input sequence, thereby capturing long-distance dependencies in the image. Specifically, the self-attention mechanism in the Transformer module not only enables the model to capture the global spatial distribution features of the image but also allows for the modeling of long-distance dependencies between distant microplastic particles, improving the model's consistency in recognizing dissimilar particles of the same type.

[0063] Among these, long-distance dependency modeling focuses more on the ability to model the associations between distant elements in the input sequence, which is a key advantage of Transformer over CNN. This is manifested as follows: after the input features are flattened into a one-dimensional sequence, each token (image patch) calculates attention weights with all other tokens in the sequence; the attention matrix is ​​a fully connected matrix—there are no "positional constraints" or "local receptive fields." For example, a microplastic particle in the upper left corner of an image can establish an explicit connection with a similar particle in the lower right corner through the attention mechanism, which is difficult for convolutional neural networks to achieve.

[0064] Furthermore, in this invention, the Transformer employs multiple self-attention layers stacked together. Through layer-by-layer feature mapping and nonlinear transformation, it further enhances the abstraction and expressive power of features. For example, the first-layer Transformer module only recognizes basic global feature relationships, and then gradually progresses to higher-order abstract features, such as more complex information like material differences and microplastic type distinctions, thereby significantly improving the model's recognition detail and classification accuracy.

[0065] Pooling operations in the later stages of a CNN module can lead to the loss of some spatial information. However, the Transformer module can effectively compensate for the information loss that may occur during feature dimensionality reduction in CNNs through global attention. For example, the features of certain microplastic particles may be weakened or ignored after dimensionality reduction in CNNs. The Transformer's self-attention mechanism strengthens these easily overlooked feature points by focusing on the global information of all features in the image.

[0066] In some embodiments of the present invention, step S400, which involves fusing local and global features to classify and identify microplastic images, includes: S401: The local spatial feature vector output by the CNN feature extraction module. Global spatial feature vectors extracted by the Transformer module The features are concatenated and merged to generate a unified high-dimensional feature vector. : S402: Merged features The input is fed into a fully connected layer, and after nonlinear mapping, the Softmax classification function is used to predict the microplastic category. The output probability of the Softmax classifier is calculated using the following formula: in: : Represents the output of the fully connected layer. Class score; : Indicates the total number of categories for microplastic classification; : Indicates that the input feature belongs to the th The probability of a class.

[0067] In practical implementation, during feature concatenation and fusion, the local detailed features of CNNs and the global features of Transformers have inconsistent dimensions (CNN features are typically H×W×C, while Transformer features are N×D). Direct concatenation may lead to feature space misalignment and information redundancy interference. Therefore, a cross-modal feature alignment module is introduced.

[0068] The specific implementation is as follows: 1. Feature dimension alignment: 2. Attention weight calculation: 3. Adaptive Feature Fusion: .

[0069] The feature fusion and classification recognition steps of this invention aim to effectively fuse the local features extracted by the CNN module with the global features extracted by the Transformer module, thereby achieving high-precision classification and recognition of microplastic images.

[0070] The specific implementation method is as follows: First, the local spatial feature vector output by the CNN feature extraction module is... Global spatial feature vectors extracted by the Transformer module The features are concatenated and merged to generate a unified high-dimensional feature vector. : The fused features will then be... The input is fed into a fully connected layer, and after nonlinear mapping, the final microplastic category prediction is achieved using the Softmax classification function. The output probability of the Softmax classifier is calculated using the following formula: in: The first output of the fully connected layer Class score; The total number of categories for classifying microplastics; For input features belonging to the first The probability of a class.

[0071] The fusion of features from CNNs and Transformers aims to compensate for the shortcomings of single-model feature extraction. The two models are significantly complementary: Local features (CNN) advantage: CNNs excel at capturing local image features, such as the texture, edges, and morphological features of microplastics. Global features (Transformer) advantage: Transformers excel at capturing overall structure and long-range spatial relationships.

[0072] By merging and combining features, the final input classifier possesses the dual advantages of "local + global" features, achieving information complementarity between local and global features and minimizing the limitations of a single feature type. For example, if a certain type of microplastic has similar local features but vastly different spatial distributions (scattered locations) in an image, using a CNN alone might lead to misclassification. However, by fusing Transformer features, the model can effectively utilize global information to identify items belonging to the same category, significantly improving classification accuracy.

[0073] After feature fusion, the nonlinear mapping through a fully connected layer can further improve the discriminative power between different categories in the feature space, making it easier for the Softmax function to classify accurately.

[0074] Suppose that before the fused feature vectors are input into the fully connected layer, the feature vectors of different categories of microplastics in the feature space are close to each other, making classification difficult. After the nonlinear mapping of the fully connected layer, the feature space is remapped, and the previously difficult-to-distinguish category features become easier to distinguish, effectively improving classification accuracy.

[0075] This invention uses the Softmax classifier to achieve the final microplastic classification decision. Its probabilistic classification method can intuitively and clearly give the classification confidence of the model, improving the credibility and interpretability of the classification results.

[0076] The feature fusion module combines the local feature robustness of CNNs with the global feature generalization of Transformers, which greatly improves the generalization ability and robustness of the entire model in complex scenes and reduces the model's sensitivity to image noise and background interference.

[0077] In actual water sample image acquisition, there may be problems such as changes in lighting, water turbidity, or interference from background impurities. The fused model can combine local details and global information to perform effective classification and judgment, avoiding false detections or missed detections that may occur when using a single model.

[0078] In some embodiments of the present invention, it further includes: S500. Optimize the parameters of the hybrid CNN and Transformer model through deep learning model training, including the following steps: The cross-entropy loss function is used, and its expression is: in: The actual label vector; The class probabilities predicted by the model; The total number of categories; The parameters are optimized using the stochastic gradient descent method with momentum, as shown in the following formula: Gradient update formula: Parameter update formula: in: This is the momentum term, representing the cumulative effect of historical gradients; Momentum factor; The learning rate; These are model parameters; For the loss function with respect to the parameters The gradient.

[0079] In this embodiment, the core objective of the model training and optimization steps is to optimize the parameters of the CNN and Transformer hybrid model through the deep learning model training process, thereby achieving accurate identification of microplastic image data.

[0080] Among them, the cross-entropy loss function can effectively quantify the difference between the true class distribution and the model-predicted class distribution, and has been widely proven to have excellent performance in classification tasks.

[0081] Stochastic gradient descent (SGD with Momentum) introduces accumulated information of historical gradients during parameter optimization, effectively solving the problems of large gradient update jitter and slow convergence in the traditional stochastic gradient descent (SGD) method.

[0082] The introduction of the momentum term means that model parameter updates depend not only on the current gradient but also on the historical gradient directions, demonstrating an "inertia" effect. When the model gets stuck in a saddle point or a local optimum, the momentum term can help the model quickly escape inefficient regions, accelerating the model's convergence speed.

[0083] If the gradient direction changes frequently during a training phase, traditional SGD can cause parameter oscillations and significant fluctuations in the training process. Introducing a momentum term makes the parameter update direction more stable, thereby improving training stability and efficiency.

[0084] The model parameter optimization process is implemented using the backpropagation algorithm, which calculates the gradient of each layer's parameters by backpropagating the loss function based on the chain rule, as shown in the following formula: Set parameters The gradient is: The gradient is calculated layer by layer from the output layer and then propagated backward, ensuring efficient parameter updates for each layer.

[0085] By carefully selecting the loss function and optimization algorithm, the training process of this invention is more likely to achieve good generalization performance, significantly reduce the risk of overfitting, and improve the model's performance in real-world image detection.

[0086] In the experiments, by appropriately setting parameters such as the learning rate and momentum factor during the training phase, the model was able to converge quickly and stabilize at its optimal performance. It demonstrated stable performance on the real test set, and its generalization performance outperformed models that did not employ a momentum term or used an unsuitable loss function.

[0087] The optimized model was applied to real-world environmental images to output microplastic identification results, including microplastic category, location, and distribution information. Actual detection results demonstrate that the method of this invention not only significantly improves the accuracy and efficiency of automatic microplastic detection but also reduces the false negative and false positive rates, verifying the technical advantages and application value of this hybrid model in practical environmental monitoring.

[0088] In some embodiments of the present invention, the method further includes: S600, applying the optimized model to an actual environmental image, outputting microplastic identification results, and cross-validating them with traditional standard detection methods.

[0089] In this embodiment, a trained and optimized hybrid model is used to automatically identify microplastic images in real environments, and cross-validation is performed with traditional standard detection methods such as Fourier transform infrared spectroscopy (FTIR) and Raman spectroscopy to ensure the accuracy and reliability of the detection.

[0090] In some embodiments of the present invention, step S100 further includes performing data augmentation on the acquired images through image preprocessing to expand the size of the training dataset.

[0091] In some embodiments of the present invention, image preprocessing is used to enhance the acquired images, including cropping, rotating, flipping, and adjusting brightness and contrast.

[0092] In this embodiment, data augmentation expands the dataset size, increases image diversity, significantly improves the model's generalization ability to real-world images, and effectively reduces overfitting caused by insufficient data in existing technologies.

[0093] In summary, this invention proposes a microplastic intelligent detection method based on a Transformer-CNN hybrid model. By innovatively integrating the advantages of both the Transformer and CNN models, it achieves efficient collaborative recognition of local fine features and global spatial information in microplastic images. Its advantages are: (1) The advantages of fusing local and global features are prominent. The innovative fusion of CNN local features and Transformer global features effectively makes up for the shortcomings of single models in capturing local details and extracting global spatial features, and significantly improves the detection accuracy and generalization performance of microplastics.

[0094] (2) Significantly enhanced generalization. The model of this invention has strong adaptability to microplastic images under different environmental conditions, reduces the impact of complex environmental interference on recognition performance, and has excellent anti-interference performance.

[0095] (3) High degree of intelligence and significantly improved detection efficiency. The method of the present invention has a high degree of automation. Compared with traditional Fourier transform infrared (FTIR) or Raman spectroscopy and other instrumental methods, it does not require complicated preprocessing and high equipment costs, and can achieve rapid on-site monitoring with a significant improvement in efficiency.

[0096] (4) The detection results are transparent and reliable, and have broad application prospects. The model output results include clear classification confidence (probability), the detection process is transparent and the results are reliable, and can be widely used in many fields such as environmental monitoring, water quality monitoring, emergency detection and pollution prevention and control.

[0097] refer to Figure 2 As shown, the present invention also discloses a microplastic intelligent detection system, including a memory and a processor. The memory stores a determination program for a microplastic intelligent detection method based on a Transformer-CNN hybrid model, and the processor runs the determination program for the microplastic intelligent detection method based on a Transformer-CNN hybrid model, so that the microplastic intelligent detection system executes the microplastic intelligent detection method based on a Transformer-CNN hybrid model.

[0098] The present invention also discloses a computer-readable storage medium, comprising: a determination program for a microplastic intelligent detection method based on a Transformer-CNN hybrid model stored on the computer-readable storage medium, wherein when the determination program for the microplastic intelligent detection method based on a Transformer-CNN hybrid model is executed by a processor, the microplastic intelligent detection method based on a Transformer-CNN hybrid model is implemented.

[0099] Since the microplastic intelligent detection system and computer-readable storage medium adopt all the technical solutions of the microplastic intelligent detection method based on the Transformer-CNN hybrid model in the above embodiments, they have at least all the beneficial effects brought about by the technical solutions in the above embodiments, and will not be repeated here.

[0100] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A microplastic intelligent detection method based on a hybrid model of a Transformer-CNN, characterized in that, The method comprises the following steps: S100, collecting microplastic image data of an environmental water sample; S200, performing local feature extraction on the microplastic image by using a CNN module; S300, inputting the extracted local features into a Transformer module to extract global features; S400, fusing the local features and the global features, and performing classification and identification of the microplastics based on the fused features.

2. The microplastic intelligent detection method based on the hybrid model of the Transformer-CNN according to claim 1, characterized in that, In step S200, the CNN module comprises a plurality of convolution layers, batch normalization layers and pooling layers, and comprises the following steps: S201, each convolution layer scans the microplastic image by using a convolution kernel to extract detailed information of the microplastic image, wherein the convolution formula is: wherein: : indicates the first l layer convolution operation position is the feature value of : represents the feature map output of the previous layer; : represents a weight coefficient of a convolution kernel; : represents a bias term; : denotes a non-linear activation function; S202, the batch normalization layer is used to perform standardization processing on the feature map after convolution, and the normalization formula is: wherein: : represents an input feature; and : mean and variance of the batch feature, respectively; : represents a small constant to prevent division by zero; S203, the pooling layer is used to perform a pooling operation.

3. The microplastic intelligent detection method based on the hybrid model of Transformer-CNN according to claim 1, characterized in that, The CNN module further comprises a position perception module, which dynamically adjusts the receptive field of the convolution kernel by using a deformable convolution to enhance the adaptability to the irregular shape of the microplastics, and comprises the following steps: The position-aware module dynamically adjusts the sampling position of the convolution kernel through deformable convolution, and the fixed offset of the standard convolution X is replaced by a learnable offset Δ p k,l The calculation form is: p k,l ​ wherein: Y ( p ) : denotes the pixel value of the output feature map at position p ; W ( k,l ) : denotes the weight value of the convolution kernel at position (k,l ) place; X (·): denotes a pixel value function of the input feature map, which denotes a pixel at a certain position taken from the input image; P : represents the original image coordinates corresponding to the current output position; p k,l : denotes the offset of the (i k，l )th fixed sampling position in the convolution kernel; Δ p k,l : indicates an offset amount.

4. The microplastic intelligent detection method based on the hybrid model of Transformer-CNN according to claim 1, characterized in that, In step S300, the inputting of the extracted local features into the Transformer module to extract global features comprises: The feature map output by the CNN module is flattened into a one-dimensional feature sequence and sent to the Transformer module, and the global features are extracted by using a self-attention mechanism, and the output of the self-attention mechanism is calculated by the following formula: wherein: ； ； : denotes a scaling factor for normalizing attention values; , , : denotes trainable weight matrix; for a given input feature sequence, n for sequence length, d for feature dimension.

5. The microplastic intelligent detection method based on the hybrid model of Transformer-CNN according to claim 1, characterized in that, In step S400, the fusing of the local features and the global features to perform classification and identification of the microplastic image comprises: S401: concatenate the local spatial feature vector output by the CNN feature extraction module with the global spatial feature vector extracted by the Transformer module to generate a unified high-dimensional feature vector : S402: Merged features The input is fed into a fully connected layer, and after nonlinear mapping, the Softmax classification function is used to predict the microplastic category. The output probability of the Softmax classifier is calculated using the following formula: wherein: : denotes the first fully connected layer output class score value; : indicates the total number of microplastic categories; : represents the probability that the input feature belongs to the class.

6. The microplastic intelligent detection method based on the hybrid model of Transformer-CNN according to claim 1, characterized in that, Further comprising: S500, optimizing the parameters of the CNN-Transformer hybrid model by deep learning model training, comprising the following steps: The cross-entropy loss function is used, and the expression of the cross-entropy loss function is: wherein: is the true label vector; is the model predicted class probability; is the total number of classification classes; The random gradient descent method with momentum is used to optimize the parameters, and the formula is as follows: Gradient update formula: Parameter update formula: wherein: is a momentum term, representing the cumulative influence of historical gradients; is a momentum factor; learning rate; model parameters; gradient of the loss function with respect to the parameters gradient of the loss function with respect to the parameters 7. The microplastic intelligent detection method based on the hybrid model of the Transformer-CNN according to claim 5, characterized in that, Further comprising: S600, applying the optimized model to an actual environmental image to output a microplastic identification result, and cross-verification with a traditional standard detection method.

8. The microplastic intelligent detection method based on the hybrid model of Transformer-CNN according to claim 1, characterized in that, In step S100, the collected image is subjected to data enhancement by image preprocessing to expand the scale of the training data set. 9.A microplastic smart detection system, characterized in that: The microplastic intelligent detection system comprises a memory and a processor, the memory stores a determination program of a microplastic intelligent detection method based on a Transformer-CNN hybrid model, and the processor runs the determination program of the microplastic intelligent detection method based on the Transformer-CNN hybrid model, so that the microplastic intelligent detection system performs the microplastic intelligent detection method based on the Transformer-CNN hybrid model in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The method comprises the following steps: The computer readable storage medium stores a determination program of the microplastic intelligent detection method based on the hybrid model of the Transformer-CNN. The computer readable storage medium stores a determination program of the microplastic intelligent detection method based on the hybrid model of the Transformer-CNN.

Images