Pollen monitoring method and device based on convolutional neural network, and convolutional neural network

Abstract

The invention provides a pollen monitoring method and device based on a convolutional neural network and the convolutional neural network, and relates to the technical field of pollen monitoring, and the specific implementation mode comprises the steps: collecting an original pollen image, and inputting the original pollen image into the convolutional neural network; and processing the original pollen image data through the convolutional neural network to obtain a feature image, and outputting a pollen type identification and classification result based on the feature image. According to the method, the low-order visual features of the pollen are extracted through the convolutional neural network, so that the network can capture key morphological features more effectively; through multistage convolution operation, deeper features can be extracted from a pollen image layer by layer, global morphological information can be gradually abstracted from local geometric features, pollen types with slight differences can be distinguished through a gradually refined feature extraction process, multiple pollen types can be accurately and efficiently identified, and the method has the advantages of high accuracy and high efficiency. The accuracy, the real-time performance and the automation degree of pollen monitoring are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pollen monitoring, and in particular to a pollen monitoring method and device based on a convolutional neural network, and a convolutional neural network. Background Art

[0002] With global warming and the acceleration of urbanization, air pollution is becoming increasingly serious, resulting in an increase in the number of pollen allergy patients year by year. In this context, pollen monitoring, as an important task in the field of public health, is of great significance for the prevention and control of pollen allergies. However, traditional pollen monitoring methods mainly rely on manual microscopy, which is labor-intensive, inefficient, and easily affected by human factors, resulting in uncertain results. Traditional pollen monitoring methods include gravity sedimentation and volumetric methods. The gravity sedimentation method uses the weight of pollen itself and air flow to sink to the ground under the action of wind, collect pollen through a slide coated with an adhesive, and then classify and count under an optical microscope. This method is economical, simple to operate, suitable for long-term investigations, and especially suitable for grassroots hospitals. The volumetric method collects pollen in the air through a specific container and then counts it. This method can provide more accurate pollen concentration data, but the operation is relatively complicated.

[0003] To overcome these challenges, machine vision and image processing technology can be used to automatically monitor and identify pollen, thereby improving monitoring efficiency and accuracy and providing more reliable data support for the public health field. The application of machine vision and image processing technology in pollen monitoring includes image acquisition, image preprocessing, feature extraction, pollen identification, and data analysis and visualization. Pollen images are collected by high-resolution cameras and microscope imaging, and then the images are preprocessed to extract characteristic information such as pollen morphology, texture, and color, and automatically identified and classified using machine learning or deep learning algorithms. The identification results are statistically analyzed to calculate the concentration, distribution, and propagation trends of different types of pollen, and are intuitively displayed through data visualization technology. This automatic pollen monitoring method based on machine vision and image processing technology is more efficient and accurate than the traditional manual microscope inspection method, and better meets the needs of pollen monitoring in the public health field.

[0004] However, due to the complex and diverse morphological characteristics of pollen and the wide variety of types, traditional image processing methods have difficulty coping with the high diversity of pollen particles in size, shape, texture and color, and cannot accurately capture the unique morphological geometric structure and surface texture of each pollen, making it difficult to distinguish between subtly different pollen types. Therefore, existing pollen monitoring technologies are unable to perform efficient and accurate automatic identification and classification of pollen types. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a pollen monitoring method, system, device and medium to perform customized optimization for pollen identification, which can accurately and efficiently identify multiple pollen types, significantly improve the accuracy, real-time and automation of pollen monitoring, thereby improving the prevention and control level of pollen allergic diseases.

[0006] The embodiment of the present application provides the following technical solution: a pollen monitoring method based on a convolutional neural network, comprising:

[0007] Collecting original pollen images, and inputting the original pollen images into a convolutional neural network;

[0008] Processing the original pollen image data by the convolutional neural network to obtain a feature image, and outputting the identification and classification results of the pollen types based on the feature image;

[0009] The convolutional neural network includes: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer; the input layer preprocesses the original pollen image to obtain a processed pollen image, the multiple convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel and a pollen surface texture extraction convolution kernel, so as to extract the local features of pollen in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract the global morphological features of pollen of different sizes in the first feature image, and feature fuse the global morphological features of pollen of different sizes to generate a second feature image; the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; the fully connected layer integrates the local features and the global morphological features of the third feature image to generate the recognition and classification results of the pollen species, and the output layer outputs the recognition and classification results of the pollen species.

[0010] According to an embodiment of the present application, the input layer preprocesses the original pollen image, including:

[0011] Acquire multiple pollen images with different focal lengths, perform multi-focus stacking on the multiple pollen images with different focal lengths, and generate a full-focus pollen image;

[0012] Performing feature fusion on the full-focus pollen images in multiple fields of view to generate a wide-field pollen image;

[0013] The wide-view pollen image is subjected to integrated denoising, contrast enhancement and normalization processing to obtain a processed pollen image.

[0014] According to an embodiment of the present application, the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image, including:

[0015] Adopting an adaptive pooling strategy to dynamically adjust the size of the pooling window according to the local feature information of the second feature image;

[0016] Using a variance pooling operation to perform local noise suppression on the second feature image;

[0017] A global average pooling operation is used to perform local illumination suppression on the second feature image.

[0018] According to an embodiment of the present application, the fully connected layer integrates the local features and the global morphological features of the third feature image, including:

[0019] The third feature image after convolution and pooling operations is transmitted to the fully connected layer through feature mapping, and the fully connected layer integrates the local features and global morphological features of the third feature image according to the set weight parameters, and outputs feature comprehensive results of different categories;

[0020] The comprehensive results of the features of different categories are converted into probability distribution through the Softmax function, and the identification and classification results of the pollen types are output according to the probability distribution.

[0021] According to an embodiment of the present application, the fully connected layer integrates the local features and the global morphological features of the third feature image according to the set weight parameters, including:

[0022] Initialize the set weight parameters, and integrate the local features and global morphological features of the third feature image using the following formula: y=W·x+b;

[0023] Wherein, W is the initialized weight parameter matrix, x is the feature vector of the third feature image input, b is the bias term used to fine-tune the output, and y is the output of the fully connected layer.

[0024] According to an embodiment of the present application, it also includes:

[0025] Reversely optimize the network model of the convolutional neural network through a cross entropy loss function and a gradient descent optimizer;

[0026] The network model of the convolutional neural network is fine-tuned through pre-training models and transfer learning.

[0027] The present application also provides a convolutional neural network, comprising:

[0028] An input layer, wherein the input layer pre-processes the original pollen image to obtain a processed pollen image;

[0029] A plurality of convolutional layers, wherein the plurality of convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, wherein the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel, and a pollen surface texture extraction convolution kernel, so as to extract pollen local features in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract pollen global morphological features of different sizes in the first feature image, and feature fuse the pollen global morphological features of different sizes to generate a second feature image;

[0030] A pooling layer, wherein the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image;

[0031] A fully connected layer, wherein the fully connected layer integrates local features and global morphological features of the third feature image to generate identification and classification results of pollen types;

[0032] An output layer outputs the identification and classification results of the pollen types.

[0033] The present application also provides a pollen monitoring device based on a convolutional neural network, comprising:

[0034] An image acquisition module, used for acquiring original pollen images and inputting the original pollen images into a convolutional neural network;

[0035] An identification module, used for processing the original pollen image data through the convolutional neural network to obtain a feature image, and outputting the identification and classification results of the pollen types based on the feature image;

[0036] The convolutional neural network includes: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer; the input layer preprocesses the original pollen image to obtain a processed pollen image, the multiple convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel and a pollen surface texture extraction convolution kernel, so as to extract the local features of pollen in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract the global morphological features of pollen of different sizes in the first feature image, and feature fuse the global morphological features of pollen of different sizes to generate a second feature image; the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; the fully connected layer integrates the local features and the global morphological features of the third feature image to generate the recognition and classification results of the pollen species, and the output layer outputs the recognition and classification results of the pollen species.

[0037] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned pollen monitoring method based on convolutional neural network when executing the computer program.

[0038] The present application also provides a computer-readable storage medium, which stores a computer program for executing the above-mentioned pollen monitoring method based on convolutional neural network.

[0039] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above technical solutions adopted in the embodiments of this specification include at least the following: The embodiments of the present invention customize and design an efficient deep learning architecture CNN machine vision module for pollen image recognition and classification, which is specifically used for high-precision automatic recognition and classification of pollen species. Since each type of pollen has its own unique geometric structure and surface texture in morphology, these features are crucial in pollen species recognition. The embodiments of the present invention extract low-order visual features of pollen, such as edges, fine lines, surface texture, etc., through the convolution layer of the CNN machine vision module. In view of the microscopic characteristics of pollen images, the design of the convolution kernel takes into account the typical shapes of pollen, such as spherical, elliptical, and the thorny and concave-convex features on the pollen surface, to ensure that the network can more effectively capture these key morphological features. In addition, through multi-level convolution operations, the network can extract deeper features from pollen images layer by layer, and gradually abstract global morphological information from local geometric features. Through this gradually refined feature extraction process, it can distinguish subtly different pollen types and accurately and efficiently identify multiple pollen types, significantly improving the accuracy, real-time and automation of pollen monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 1 is a schematic diagram of a pollen monitoring method based on a convolutional neural network according to an embodiment of the present invention;

[0042] Figure 2 is a schematic structural diagram of a pollen monitoring device based on a convolutional neural network according to an embodiment of the present invention;

[0043] Figure 3 It is a structural schematic diagram of the computer device of the present invention. DETAILED DESCRIPTION

[0044] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0045] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0046] like Figure 1 As shown, an embodiment of the present invention provides a pollen monitoring method based on a convolutional neural network, comprising:

[0047] S101. Collecting an original pollen image, and inputting the original pollen image into a convolutional neural network;

[0048] S102. Processing the original pollen image data by the convolutional neural network to obtain a feature image, and outputting the identification and classification results of the pollen types based on the feature image;

[0049] The convolutional neural network includes: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer; the input layer preprocesses the original pollen image to obtain a processed pollen image, the multiple convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel and a pollen surface texture extraction convolution kernel, so as to extract the local features of pollen in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract the global morphological features of pollen of different sizes in the first feature image, and feature fuse the global morphological features of pollen of different sizes to generate a second feature image; the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; the fully connected layer integrates the local features and the global morphological features of the third feature image to generate the recognition and classification results of the pollen species, and the output layer outputs the recognition and classification results of the pollen species.

[0050] The convolutional neural network (CNN) module in the embodiment of the present invention is specially customized and optimized for pollen image recognition and classification. Due to the complex and diverse morphological characteristics of pollen and the wide variety of types, traditional image processing methods are difficult to cope with the high diversity of pollen particles in terms of size, shape, texture and color. Therefore, in view of the uniqueness of pollen images, an efficient deep learning architecture is designed, which is specifically used for high-precision automatic recognition and classification of pollen species. The CNN module in the embodiment of the present invention first extracts low-order visual features of pollen, such as edges, fine lines, surface texture, etc., through a convolutional layer. These features are crucial in pollen species recognition because each type of pollen has its own unique geometric structure and surface texture in morphology. In view of the microscopic characteristics of pollen images, the design of the convolution kernel takes into account the typical shapes of pollen, such as spherical, elliptical, and thorny and concave-convex features on the surface of pollen, to ensure that the network can capture these key morphological features more effectively. Through multi-level convolution operations, the network can extract deeper features from the pollen image layer by layer, and gradually abstract global morphological information from local geometric features. This progressively refined feature extraction process is crucial for distinguishing subtly different pollen species.

[0051] In a specific implementation of this embodiment, the input layer preprocesses the original pollen image, including:

[0052] Acquire multiple pollen images with different focal lengths, perform multi-focus stacking on the multiple pollen images with different focal lengths, and generate a full-focus pollen image;

[0053] Performing feature fusion on the full-focus pollen images in multiple fields of view to generate a wide-field pollen image;

[0054] The wide-view pollen image is subjected to integrated denoising, contrast enhancement and normalization processing to obtain a processed pollen image.

[0055] In specific implementation, this embodiment aims at the image problem at different focal lengths during pollen imaging, and the CNN module integrates multi-focus image stacking technology. In the image pre-processing stage, this module fuses images with different focal points and passes them to CNN for deep feature extraction. This multi-focus image stacking and fusion processing method can capture the three-dimensional structural information of pollen more comprehensively, provide richer input data for CNN recognition, and improve the recognition accuracy of complex morphological pollen. The image data of each focus is fused to form a full-focus image, which comprehensively and clearly displays the detailed features of pollen, improves the overall clarity and contrast of the image, ensures that the morphological features of pollen are accurately captured, and provides high-quality visual information for subsequent image analysis; by automatically identifying the overlapping areas between images and applying image fusion technology, the field of view of the pollen monitoring system is expanded, and a wide-field image is generated to improve the monitoring efficiency and comprehensiveness, which is suitable for large-scale environmental monitoring.

[0056] 1. Multi-focus image stacking

[0057] In the pre-processing stage, multiple images from different focal lengths are fused. The fusion operation can be expressed as a weighted average:

[0058]

[0059] Among them: F i , represents the image features of the i-th image with different focal lengths; α i is the corresponding weighting coefficient, which is used to control the weight of images with different focal lengths in fusion; N is the total number of images with different focal lengths.

[0060] Through this step, the image information of multiple focal lengths is integrated into a fused feature map F fused , passed to CNN for subsequent processing.

[0061] 2. Convolutional feature extraction

[0062] The fused image is input to the convolution layer of CNN, and the features in the multi-focus image are extracted through standard convolution operation. Convolution operation formula:

[0063]

[0064] Among them: F fused(i+m,j+n,c) is the pixel value of the fused multi-focus image at position (i+m,j+n); K(m,n,c) is the weight of the convolution kernel; O(i,j) is the feature map of the convolution output.

[0065] Through this multi-focus fusion and convolutional feature extraction, CNN can more comprehensively capture the three-dimensional structural information of pollen and improve the recognition accuracy of pollen with complex morphology.

[0066] In the specific implementation, the design of multiple convolutional layers specifically includes the following contents:

[0067] (1) In order to extract the low-level visual features of pollen, it is necessary to design appropriate convolution kernels. Each feature corresponds to a different convolution kernel design:

[0068] 1. Convolution kernel design

[0069] Edge detection: Use specially designed edge detection convolution kernels, such as the Sobel kernel or the Prewitt kernel, which can detect edges in an image. For example, the formula for the Sobel kernel is as follows: -1&0&1\\-2&0&2\\-1&0&1\end{bmatrix}\] This convolution kernel can emphasize horizontal or vertical edges in an image.

[0070] Fine lines and textures: Use a specific convolution kernel to extract fine lines and textures. For example, you can use a Gabor filter: Where σ is the standard deviation of the Gaussian function, λ is the wavelength, and Φ is the phase shift.

[0071] 2. Convolution formula

[0072] Convolution formula: The design of the formula is directly related to the choice of convolution kernel. For different low-order feature extraction, the design of convolution kernel K will be different.

[0073] 3. Convolution kernel parameter setting

[0074] Convolution kernel size: Small-sized convolution kernels (such as 3×3 or 5×5) are usually used to capture detailed features such as edges and fine lines because they can process local areas in detail.

[0075] Stride: The stride determines the speed at which the convolution kernel slides across the image. A smaller stride (such as 1) can extract features more finely and retain more local information.

[0076] Padding: Padding is used to control the size of the feature map after the convolution operation. Padding can ensure that the convolution operation does not lose edge information, which is especially important for capturing details such as edges. The commonly used padding method is zero padding.

[0077] 4. Multi-layer convolution

[0078] To extract more complex features (for example, combined features), multiple layers of convolution can be used. By stacking convolutional layers, each layer extracts features at a different level: Among them, (l) represents the output of the lth layer, K (l) is the convolution kernel of the lth layer.

[0079] 5. Activation of feature maps

[0080] After the convolution operation, the activation function (such as ReLU) will further process the feature map: O′(i,j)=max(0,O(i,j)) The activation function helps introduce nonlinear transformations, enabling the model to learn more complex features.

[0081] (2) During the deep convolution process, CNN automatically adapts to the diversity of pollen through convolution kernels of different scales. Different types of pollen may have different sizes and different surface features in microscopic images, so multi-layer convolution kernels capture the subtle differences in pollen through local perception and global scanning. In order to cope with the complexity of pollen morphology, a multi-scale feature mapping layer is specially designed, which enables the network to extract global morphology and local details at different scales at the same time, ensuring that pollen of similar morphology but belonging to different types can be distinguished.

[0082] 1. Multi-scale convolution kernel

[0083] In order to extract features of different scales, you can design a variety of convolution kernels of different sizes and apply them to the input image. Assuming there are three different sizes of convolution kernels K1, K2, and K3 (such as 3×3, 5×5, 7×7), the formula for the convolution operation is:

[0084] For the convolution kernel K k (where k represents different convolution kernel sizes), output feature map O k Calculated as:

[0085] in:

[0086] k w,k and k h,k is the convolution kernel K k The width and height of the .

[0087] 2. Feature Pyramid Network (FPN)

[0088] The feature pyramid network combines feature maps of different levels to better process information of different scales. Assume that after convolution and pooling operations, feature maps of different scales F1, F2, ..., Fn , they have different resolutions. Feature fusion can be expressed as

[0089] Among them: F i are feature maps at different scales; α i is the fusion weight coefficient, which is used to weight feature maps of different scales; N is the number of feature maps of different scales.

[0090] 3. Fusion of features at different scales

[0091] When fusing features into a unified representation, feature fusion can be achieved through layer-by-layer convolution and pooling operations. The multi-scale feature maps are merged and further convolution is performed to obtain the final feature representation. For example: F final =Conv(F fused ) Where Conv represents a convolution operation, acting on the fused feature map F fused To extract the final high-level features.

[0092] According to the above formula, multi-scale convolution kernels and feature fusion technology are used to cope with the complexity of pollen morphology, and features are captured and fused from different scales to improve the accuracy of pollen classification.

[0093] In a specific implementation of this embodiment, the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image, including:

[0094] Adopting an adaptive pooling strategy to dynamically adjust the size of the pooling window according to the local feature information of the second feature image;

[0095] Using a variance pooling operation to perform local noise suppression on the second feature image;

[0096] A global average pooling operation is used to perform local illumination suppression on the second feature image.

[0097] In the specific implementation, the design of the pooling layer specifically includes the following contents:

[0098] After the convolution operation, the system further optimizes the feature extraction effect through the pooling layer (such as Max-Pooling), especially for the noise and subtle deformation that may appear in the pollen image. The pooling operation can filter out irrelevant image noise while retaining key information. Based on the needs of pollen monitoring, the design of the pooling layer pays special attention to improving the ability to retain spatial details and ensure sensitivity to different pollen morphologies.

[0099] At the same time, considering that pollen images under a microscope are easily affected by uneven lighting or focal deviation, a specific pooling algorithm suppresses noise in the input image, making the network more robust to changes in image input quality.

[0100] 1. Max-Pooling

[0101] Max pooling is to downsample by selecting the maximum value in the pooling window. For example, if the size of the pooling window is p×p, the max pooling formula is:

[0102] P(i,j)=max{O′(i+m,j+n)|0≤m <p,0≤n<p}

[0103] Where: P(i,j) is the value of the pooled feature map at position (i,j); O′(i+m,j+n) is the value of the feature map after convolution and activation function processing; p×p is the size of the pooling window (for example, 2×2 or 3×3).

[0104] 2. Specific applications

[0105] (1) Select the pooling window size

[0106] Choosing an appropriate pooling window size (such as 2×2) can significantly reduce the spatial dimension of the feature map. For example, a 2×2 maximum pooling window will halve both the width and height of the feature map.

[0107] (2) Stride

[0108] The pooling operation usually uses a stride to determine the step size of the window movement. The stride is generally set to the size of the pooling window (for example, 2) so that each pooling window does not overlap.

[0109] (3) Feature map size after pooling operation

[0110] Assuming the size of the input feature map is W×H, after using a 2×2 pooling window and stride 2, the size of the output feature map will become W / 2×H / 2.

[0111] The key points of the pooling operation in this embodiment are:

[0112] (1) Adaptive Pooling

[0113] In response to changes in different image regions, an adaptive pooling strategy is adopted, such as dynamically adjusting the pooling window based on local statistical information.

[0114] Adaptive pooling formula:

[0115] P(i,j)=adaptive max{O′(i+m,j+n)|0≤m <p(i,j),0≤n<p(i,j)}

[0116] Among them, p(i,j) is the dynamically changing pooling window size.

[0117] (2) Local noise suppression

[0118] It can be combined with local variance pooling to reduce the response to areas with more noise.

[0119] Variance pooling formula:

[0120]

[0121] Among them, μ(i,j) is the average value in the pooling window, which is used to calculate the local variance.

[0122] (3) Robustness to uneven illumination

[0123] Global Average Pooling is used to reduce the impact of uneven illumination and take the average value of the entire feature map:

[0124]

[0125] This approach can help suppress local lighting problems and improve the network's robustness to overall image changes.

[0126] In a specific implementation of this embodiment, the fully connected layer integrates the local features and the global morphological features of the third feature image, including:

[0127] The third feature image after convolution and pooling operations is transmitted to the fully connected layer through feature mapping, and the fully connected layer integrates the local features and global morphological features of the third feature image according to the set weight parameters, and outputs feature comprehensive results of different categories;

[0128] The comprehensive results of the features of different categories are converted into probability distribution through the Softmax function, and the identification and classification results of the pollen types are output according to the probability distribution.

[0129] In specific implementation, the deep feature map generated by the convolutional layer and the pooling layer is finally passed to the fully connected layer, which is a key step for pollen species identification. In this process, the fully connected layer customized in this embodiment integrates the local features of pollen with the global morphological information, and generates the discriminant information of a specific pollen species through carefully adjusted weight parameters.

[0130] In order to adapt to the diversity of different types of pollen, a specific weight initialization scheme is introduced in the structural design of the fully connected layer, so that it can converge to the optimal parameters more quickly. The multi-classification results output by the fully connected layer are then converted into probability distributions through the Softmax function, and the network finally determines the pollen type based on this probability value.

[0131] 1. Fully connected layer formula

[0132] Assume that after convolution and pooling operations, the feature map is flattened into a feature vector x, and the fully connected layer calculates the output through the weight matrix W and the bias vector b:

[0133] y=W·x+b

[0134] Among them: W is the weight matrix, which contains the weight parameters learned by the network; x is the input feature vector, which comes from the output of the convolution layer and the pooling layer; b is the bias term, which is used to fine-tune the output; y is the output of the fully connected layer, which represents the comprehensive results of features of different categories.

[0135] 2. Discriminative information generation and classification

[0136] The output y of the fully connected layer will be converted into a probability distribution through the Softmax function for classification. The formula of the Softmax function is:

[0137]

[0138] Where: P(y i ) is the predicted probability of the i-th class, y i is the activation value of the fully connected layer for the i-th category, and C is the total number of classification categories. Through the probability distribution of the Softmax output, the network will finally determine which pollen type the image belongs to.

[0139] 3. Key points of design and optimization

[0140] Customized initialization of the weight matrix W: In order to adapt to the complex characteristics of pollen images, the weight matrix W is usually initialized specifically to enhance the convergence of the model and ensure that the differences between different pollen types can be captured more quickly.

[0141] Fusion of global and local features: The fully connected layer not only combines the local features extracted by each convolution kernel, but also integrates the overall global morphological information to ensure that the model can comprehensively consider different feature dimensions.

[0142] In this embodiment, a specific weight initialization scheme is introduced in the structural design of the fully connected layer. The purpose of introducing a specific weight initialization scheme in the structural design of the fully connected layer is to help the model converge to the optimal parameters quickly, especially in dealing with the diversity of different types of pollen in the pollen recognition task. For the pollen recognition task, when initializing the weight matrix of W, the randomly selected initial value may cause the gradient to disappear or explode during the training process. This will make the model training very slow and even difficult to converge. In order to solve this problem, a specific weight initialization scheme is introduced, such as:

[0143] Xavier initialization: This method avoids gradient vanishing and exploding by ensuring that the variance of the initial weights is proportional to the size of the input and output layers. The Xavier initialization formula is:

[0144]

[0145] Where nin is the number of input neurons.

[0146] He initialization: This method is suitable for networks using the ReLU activation function. It initializes weights with a larger variance. The formula is:

[0147]

[0148] This initialization can speed up the training of deep networks.

[0149] Due to the variety of pollen types, the specific weight initialization of this embodiment can provide appropriate initial conditions at the beginning of training, so that the model can learn the discrimination of each pollen feature more quickly and avoid the model falling into the local optimum or gradient vanishing problem.

[0150] In a specific implementation of this embodiment, it also includes: reverse optimizing the network model of the convolutional neural network through a cross entropy loss function and a gradient descent optimizer; and fine-tuning the network model of the convolutional neural network through a pre-trained model and transfer learning.

[0151] In the specific implementation, the back propagation algorithm of the CNN module has been specially optimized for the pollen recognition task. The system uses the cross-entropy loss function (Cross-Entropy Loss) combined with a gradient descent optimizer (such as Adam Optimizer) to maximize the network's classification accuracy for pollen types. At the same time, during the training process, the CNN module combines data augmentation technology (Data Augmentation), including rotation, scaling, and random cropping methods, to expand the limited pollen image dataset and improve the generalization ability of the model.

[0152] To cope with the complexity of the pollen dataset, the system introduced a pre-trained model and fine-tuned it on the pollen dataset through transfer learning. This method greatly shortened the model training time and improved the recognition effect with a small amount of data.

[0153] The specific algorithm of this embodiment using the cross entropy loss function combined with the gradient descent optimizer to improve the classification accuracy of pollen types is as follows:

[0154] 1. Cross-Entropy Loss The training process of the CNN module uses the cross-entropy loss function to measure the difference between the model prediction output and the true category label. The specific formula is as follows:

[0155]

[0156] Where: L is the loss value; Y i is the label of the actual category (1 if the sample belongs to the i-th category, otherwise 0), P(y i ) is the probability of the i-th category predicted by the model, and C is the total number of categories. This function optimizes the classification result by maximizing the probability of the correct category.

[0157] 2. Gradient descent optimizer (such as Adam Optimizer) CNN weight parameter update uses a gradient descent optimizer, such as Adam (adaptive moment estimation), to accelerate convergence and ensure stability. Adam combines momentum and adaptive learning rate, and the update formula is:

[0158]

[0159] Where: θ t is the current model parameter, α is the learning rate, m t and v t are the first-order moment and second-order moment estimates of the gradient, and ∈ is a small value to prevent division by zero errors. By adjusting the model weights through the optimizer, the loss function gradually decreases, thereby improving the classification accuracy.

[0160] 3. Data Augmentation Since the pollen dataset is small, the model uses data augmentation techniques during training, such as rotation and scaling, random cropping, etc., to expand the training samples. These techniques help improve the model's generalization ability for pollen of different forms. Although data augmentation does not involve specific formulas, it increases the diversity of data and reduces the risk of model overfitting by performing various transformations on the image during the preprocessing stage.

[0161] The specific operation method of fine-tuning the pre-trained model on the pollen dataset is as follows:

[0162] In this embodiment, the introduction of pre-trained models and transfer learning is an effective strategy to improve model performance in the case of complexity of pollen datasets and small amounts of data.

[0163] 1. Pre-trained Model A pre-trained model is a neural network trained on a large-scale dataset (such as ImageNet). This model has learned rich visual features and can be transferred to specific tasks, such as pollen image recognition.

[0164] In CNN, the pre-trained model can include the following parts:

[0165] f pre (X) = W pre ·X

[0166] Where: f pre (X) is the feature extracted by the pre-trained model for the input X, W pre It is the weight parameter of the pre-trained model trained on a large data set.

[0167] 2. Transfer Learning: On the pollen dataset, the weights of specific layers of the pre-trained model are fine-tuned instead of training the entire model from scratch. This can effectively utilize the existing features in the pre-trained model, avoid overfitting on small datasets, and increase training speed.

[0168] The fine-tuning process can be expressed by the following formula:

[0169] W fine-tune =W pre +ΔW

[0170] Where: W fine-tune is the fine-tuned weight, W pre is the initial weight of the pre-trained model, and ΔW is the weight update obtained by training the pollen dataset.

[0171] 3. Through transfer learning, the general visual features in the pre-trained model are combined with the specific features of pollen images, so that the model can converge faster, especially when data is limited. This not only shortens the training time, but also improves the recognition accuracy of complex pollen.

[0172] This embodiment combines the pre-trained model with transfer learning to enable the model to better handle the complexity and diversity of pollen images, greatly improving the recognition effect in the case of small data sets.

[0173] The present application also provides a convolutional neural network, comprising:

[0174] An input layer, wherein the input layer pre-processes the original pollen image to obtain a processed pollen image;

[0175] A plurality of convolutional layers, wherein the plurality of convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, wherein the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel, and a pollen surface texture extraction convolution kernel, so as to extract pollen local features in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract pollen global morphological features of different sizes in the first feature image, and feature fuse the pollen global morphological features of different sizes to generate a second feature image;

[0176] A pooling layer, wherein the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image;

[0177] A fully connected layer, wherein the fully connected layer integrates local features and global morphological features of the third feature image to generate identification and classification results of pollen types;

[0178] An output layer outputs the identification and classification results of the pollen types.

[0179] The convolutional neural network of the present embodiment is applied to the monitoring of pollen, and the training and adaptive update of the dedicated pollen data set. The CNN module is trained based on a large-scale pollen image data set, which covers a variety of common and rare pollen types, ensuring that the network can be widely adapted to different pollen species and morphological characteristics. In addition, the system also introduces an adaptive update mechanism, that is, when a new type of pollen image data is input, the system automatically updates the model parameters by online learning (Online Learning) to ensure that the efficient recognition ability is still maintained when the pollen distribution and species change. In addition, in order to improve the real-time performance of the system, the CNN module is highly optimized and can process and analyze a large amount of pollen image data in real time. In the final application of the system, the identified pollen species and their concentration distribution are presented by the data analysis module, and a visual report is generated. These reports can help environmental monitoring personnel, allergy patients and public health experts to understand the trend of pollen concentration changes in a timely manner and take corresponding preventive measures.

[0180] like Figure 2 As shown, the present application also provides a pollen monitoring device 200 based on a convolutional neural network, comprising:

[0181] An image acquisition module 201 is used to acquire original pollen images and input the original pollen images into a convolutional neural network;

[0182] The recognition module 202 is used to process the original pollen image data through the convolutional neural network to obtain a feature image, and output the recognition and classification results of the pollen types based on the feature image;

[0183] The convolutional neural network includes: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer; the input layer preprocesses the original pollen image to obtain a processed pollen image, the multiple convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel and a pollen surface texture extraction convolution kernel, so as to extract the local features of pollen in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract the global morphological features of pollen of different sizes in the first feature image, and feature fuse the global morphological features of pollen of different sizes to generate a second feature image; the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; the fully connected layer integrates the local features and the global morphological features of the third feature image to generate the recognition and classification results of the pollen species, and the output layer outputs the recognition and classification results of the pollen species.

[0184] In specific implementation, the input layer preprocesses the original pollen image, including: acquiring multiple pollen images with different focal lengths, performing multi-focus stacking processing on the multiple pollen images with different focal lengths to generate a full-focus pollen image; performing feature fusion on the full-focus pollen images in multiple fields of view to generate a wide-field pollen image; performing integrated denoising, contrast enhancement and normalization processing on the wide-field pollen image to obtain a processed pollen image.

[0185] During specific implementation, the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image, including: using an adaptive pooling strategy to dynamically adjust the size of the pooling window according to the local feature information of the second feature image; using a variance pooling operation to perform local noise suppression on the second feature image; and using a global average pooling operation to perform local illumination suppression on the second feature image.

[0186] In a specific implementation, the fully connected layer integrates the local features and global morphological features of the third feature image, including: the third feature image after convolution and pooling operations is transmitted to the fully connected layer via feature mapping, and the fully connected layer integrates the local features and global morphological features of the third feature image according to the set weight parameters, and outputs the feature synthesis results of different categories; the feature synthesis results of different categories are converted into probability distributions through the Softmax function, and the recognition and classification results of the pollen types are output according to the probability distribution.

[0187] In a specific implementation, the fully connected layer integrates the local features and global morphological features of the third feature image according to the set weight parameters, including: initializing the set weight parameters, and integrating the local features and global morphological features of the third feature image through the following formula: y=W·x+b; wherein W is the initialized weight parameter matrix, x is the feature vector of the input third feature image, b is a bias term used to fine-tune the output, and y is the output of the fully connected layer.

[0188] In a specific implementation, the device also includes: reversely optimizing the network model of the convolutional neural network through a cross entropy loss function and a gradient descent optimizer; and fine-tuning the network model of the convolutional neural network through a pre-training model and transfer learning.

[0189] The CNN machine vision module of this device is completely customized and optimized for pollen identification. It can accurately and efficiently identify a variety of pollen types, significantly improving the accuracy, real-time and automation of pollen monitoring.

[0190] In one embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned pollen monitoring method based on convolutional neural network is implemented.

[0191] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0192] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned convolutional neural network-based pollen monitoring method.

[0193] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0194] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0195] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A pollen monitoring method based on convolutional neural network, characterized in that: include: Collecting original pollen images, and inputting the original pollen images into a convolutional neural network; Processing the original pollen image data by the convolutional neural network to obtain a feature image, and outputting the identification and classification results of the pollen types based on the feature image; The convolutional neural network includes: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer; the input layer preprocesses the original pollen image to obtain a processed pollen image, the multiple convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel and a pollen surface texture extraction convolution kernel, so as to extract the local features of pollen in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract the global morphological features of pollen of different sizes in the first feature image, and feature fuse the global morphological features of pollen of different sizes to generate a second feature image; the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; the fully connected layer integrates the local features and the global morphological features of the third feature image to generate the recognition and classification results of the pollen species, and the output layer outputs the recognition and classification results of the pollen species.

2. The pollen monitoring method based on convolutional neural network according to claim 1, characterized in that: The input layer preprocesses the original pollen image, including: Acquire multiple pollen images with different focal lengths, perform multi-focus stacking on the multiple pollen images with different focal lengths, and generate a full-focus pollen image; Performing feature fusion on the full-focus pollen images in multiple fields of view to generate a wide-field pollen image; The wide-view pollen image is subjected to integrated denoising, contrast enhancement and normalization processing to obtain a processed pollen image.

3. The pollen monitoring method based on convolutional neural network according to claim 1, characterized in that: The pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image, including: Adopting an adaptive pooling strategy to dynamically adjust the size of the pooling window according to the local feature information of the second feature image; Using a variance pooling operation to perform local noise suppression on the second feature image; A global average pooling operation is used to perform local illumination suppression on the second feature image.

4. The pollen monitoring method based on convolutional neural network according to claim 1, characterized in that: The fully connected layer integrates the local features and the global morphological features of the third feature image, including: The third feature image after convolution and pooling operations is transmitted to the fully connected layer through feature mapping, and the fully connected layer integrates the local features and global morphological features of the third feature image according to the set weight parameters, and outputs feature comprehensive results of different categories; The comprehensive results of the features of different categories are converted into probability distribution through the Softmax function, and the identification and classification results of the pollen types are output according to the probability distribution.

5. The pollen monitoring method based on convolutional neural network according to claim 4, characterized in that: The fully connected layer integrates the local features and the global morphological features of the third feature image according to the set weight parameters, including: Initialize the set weight parameters, and integrate the local features and global morphological features of the third feature image using the following formula: y=W·x+b; Wherein, W is the initialized weight parameter matrix, x is the feature vector of the third feature image input, b is the bias term used to fine-tune the output, and y is the output of the fully connected layer.

6. The pollen monitoring method based on convolutional neural network according to claim 1, characterized in that: Also includes: Reversely optimize the network model of the convolutional neural network through a cross entropy loss function and a gradient descent optimizer; The network model of the convolutional neural network is fine-tuned through pre-training models and transfer learning.

7. A convolutional neural network, characterized in that: include: An input layer, wherein the input layer pre-processes the original pollen image to obtain a processed pollen image; A plurality of convolutional layers, wherein the plurality of convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, wherein the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel, and a pollen surface texture extraction convolution kernel, so as to extract pollen local features in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract pollen global morphological features of different sizes in the first feature image, and feature fuse the pollen global morphological features of different sizes to generate a second feature image; A pooling layer, wherein the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; A fully connected layer, wherein the fully connected layer integrates local features and global morphological features of the third feature image to generate identification and classification results of pollen types; An output layer outputs the identification and classification results of the pollen types.

8. A pollen monitoring device based on convolutional neural network, characterized in that: include: An image acquisition module, used for acquiring original pollen images and inputting the original pollen images into a convolutional neural network; An identification module, used for processing the original pollen image data through the convolutional neural network to obtain a feature image, and outputting the identification and classification results of the pollen types based on the feature image; The convolutional neural network includes: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer; the input layer preprocesses the original pollen image to obtain a processed pollen image, the multiple convolutional layers include a pollen low-order visual feature extraction layer and a multi-scale feature mapping layer, the pollen low-order visual feature extraction layer includes a pollen edge detection convolution kernel, a pollen fine line extraction convolution kernel and a pollen surface texture extraction convolution kernel, so as to extract the local features of pollen in the processed pollen image to obtain a first feature image; the multi-scale feature mapping layer includes convolution kernels of multiple different sizes and a feature fusion network, so as to extract the global morphological features of pollen of different sizes in the first feature image, and feature fuse the global morphological features of pollen of different sizes to generate a second feature image; the pooling layer performs a pooling operation on the second feature image to obtain an optimized third feature image; the fully connected layer integrates the local features and the global morphological features of the third feature image to generate the recognition and classification results of the pollen species, and the output layer outputs the recognition and classification results of the pollen species.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the pollen monitoring method based on convolutional neural network described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the pollen monitoring method based on a convolutional neural network as described in any one of claims 1 to 6.

Images