Pollen detection method and system based on pollen scanning film

Abstract

The invention provides a pollen detection method and system based on a pollen scanning film, and relates to the technical field of image processing, and the method comprises the steps: obtaining the pollen scanning film; based on an rtmdet network, establishing a pollen detection model; inputting the pollen scanning film into a pollen detection model, and outputting coordinates, width and height of target pollen; determining a target pollen cutting image according to the coordinates, width and height of the target pollen; establishing a pollen filtering model; inputting the cut image into a pollen filtering model, and outputting coordinates, width and height of clear pollen; determining a clear pollen cutting image according to the coordinates, width and height of the clear pollen; establishing a family classification model; inputting the clear pollen cutting image into a category classification model, and determining the category of the clear pollen; and counting the pollen quantity of each family category and the total pollen quantity of the pollen scanning film to finish pollen detection. The pollen detection efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pollen detection method and system based on pollen scanning films. Background Art

[0002] The pollen detection method based on pollen scans is a technical method that uses digital scanning technology and computer vision algorithms to automatically identify and analyze pollen images. This method first obtains a scanned image of a pollen sample through a microscope or other imaging equipment to generate a pollen scan (Whole Slide Image, WSI). Then, with the help of deep learning, image processing and pattern recognition technology, the pollen in the scan is automatically detected, filtered, classified and statistically analyzed.

[0003] With the intensification of global climate change and air pollution, the increase in pollen concentration has had a significant impact on allergic diseases, agricultural production and the ecological environment. An efficient and accurate pollen detection method can automatically and accurately perform full quantitative detection and family classification counting of pollen to make up for the shortcomings of manual detection and counting, significantly improve the speed and accuracy of pollen monitoring, provide early warning information for allergy patients in a timely manner, and help optimize management in the agricultural and environmental fields.

[0004] However, traditional pollen detection methods mostly rely on manual operation, and are often unable to accurately distinguish subtle pollen samples when processing complex pollen scans, which makes false detections and missed detections prone to occur. Pollen classification tasks are difficult due to the wide variety of species and the high similarity between families and genera. Traditional methods have weak ability to distinguish these similar pollens, resulting in low efficiency and accuracy in family classification. Summary of the invention

[0005] In order to solve the technical problem that most traditional pollen detection methods rely on manual operation, often cannot accurately distinguish subtle pollen samples when processing complex pollen scans, and are prone to false detection and missed detection, and the pollen classification task has a wide variety of species and high similarity between families and genera, and traditional methods have weak discrimination ability for these similar pollens, resulting in low efficiency and accuracy of family classification, the present invention provides a pollen detection method and system based on pollen scans.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect:

[0008] An embodiment of the present invention provides a pollen detection method based on a pollen scanning film, comprising:

[0009] S1: Obtain the pollen scan film;

[0010] S2: Establish a pollen detection model based on the rtmdet network;

[0011] S3: Input the pollen scan film into the pollen detection model, and output the coordinates and width and height of the target pollen;

[0012] S4: Determine a cropped image of the target pollen according to the coordinates and width and height of the target pollen;

[0013] S5: Establish pollen filtration model based on Vit;

[0014] S6: inputting the cropped image into the pollen filtering model, and outputting pollen categories, wherein the pollen categories include clear pollen, blurred pollen, and non-pollen;

[0015] S7: Determine the coordinates and width and height of the clear pollen according to the pollen category;

[0016] S8: Determine a clear pollen cropped image according to the coordinates and width and height of the clear pollen;

[0017] S9: Establish a family classification model based on Vit;

[0018] S10: inputting the clear pollen cropped image into the family classification model to determine the family category of the clear pollen;

[0019] S11: Count the number of pollens of each family and genus and the total number of pollens in the pollen scanning film to complete pollen detection.

[0020] Second aspect:

[0021] An embodiment of the present invention provides a pollen detection system based on a pollen scanning film, comprising:

[0022] processor;

[0023] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the pollen detection method based on pollen scanning film as described in the first aspect is implemented.

[0024] The third aspect:

[0025] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the pollen detection method based on pollen scanning films as described in the first aspect is implemented.

[0026] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0027] In an embodiment of the present invention, by establishing a pollen detection model based on the RTMDET network, it is possible to accurately detect targets in pollen images, enhance the model's ability to recognize pollen samples, reduce false detections and missed detections, and thus improve the accuracy of pollen detection. By utilizing the pollen filtering model, invalid or fuzzy pollen can be further filtered out according to the clarity and classification of the target pollen, thereby improving the accuracy of the final detection result and ensuring that valid data is used for subsequent analysis. By establishing a family classification model, it is possible to accurately classify clear pollen and clarify its family category, thereby meeting the needs of different fields such as allergy research, ecological monitoring, and agricultural production. The present invention can adapt to the pollen detection needs in various actual usage scenarios during pollen detection, and improve the efficiency and accuracy of pollen detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A schematic diagram of a process of a pollen detection method based on a pollen scanning film provided by an embodiment of the present invention;

[0030] Figure 2 A schematic structural diagram of a pollen detection system based on a pollen scanning film provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0032] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0033] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0034] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0035] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0036] Reference Manual Attached Figure 1 , showing a schematic flow chart of a pollen detection method based on pollen scanning film provided in an embodiment of the present invention.

[0037] The embodiment of the present invention provides a pollen detection method based on pollen scanning film, the method comprising:

[0038] S1: Obtain pollen scans.

[0039] Among them, the pollen whole slide image (WSI) refers to the whole microscope slide image of the pollen sample scanned and saved as a high-resolution digital image through digital microscopy technology. This image contains all the detailed information of the sample, can be magnified and viewed, and can be further analyzed by computer.

[0040] It should be noted that the step of obtaining pollen scans is the basis of the pollen detection process. It can obtain high-resolution full-image data, provide more accurate information, avoid the limitations of manual observation under a traditional microscope, support efficient automated detection, and ensure higher detection accuracy and faster processing speed.

[0041] S2: Establish a pollen detection model based on the rtmdet network.

[0042] Among them, the rtmdet (Real-Time Multi-Detection) network is a target detection framework based on deep learning. It usually adopts an efficient convolutional neural network (CNN) structure to extract image features and perform classification and regression tasks. It optimizes the use of computing resources, making target detection faster and suitable for real-time processing tasks.

[0043] It should be noted that establishing a pollen detection model based on the RTMDet network can improve the efficiency and accuracy of target detection. RTMDet combines multi-layer convolutional feature extraction with efficient algorithms, which enables rapid response in real-time detection and accurate identification of the coordinates and size of pollen. It is suitable for large-scale data processing and field applications.

[0044] Specifically, pollen scans usually have a higher resolution. When pollen images are used as input to the target detection network, they must first be cropped or resized according to the hardware memory and model input size. In the present invention, the rtmdet network is improved, such as using a large kernel depth convolution of size 5 × 5 in the Backbone and Neck layers, designing a dynamic soft label allocation strategy and using a cache mechanism to increase the amount of data, introducing a higher resolution shallow feature map in the C2 stage of Backbone, and designing a DSPPF module in the C5 stage to make it more suitable for capturing the receptive field of pollen. The feature map is sent to the csp module through a connection operation at the Neck for convolution. After the feature map is downsampled 32 times, an HDA module is added to further enhance the feature.

[0045] In a possible implementation, the pollen detection model includes a backbone network module and a feature network module.

[0046] Among them, the Backbone Network Module is the core part of the deep learning model responsible for extracting features from the input data. In the pollen detection model, the Backbone Network Module includes specific structural units for efficiently extracting important feature information from pollen scans. The Feature Network Module is used to further process and fuse the features extracted by the Backbone Network to refine pollen detection and adjust the scale, resolution and content of the feature map to provide higher quality detection results.

[0047] The backbone network module includes the CSPNext Block unit and the DSPPF expanded spatial pyramid pooling fast unit.

[0048] Among them, the CSPNext Block unit is a structural unit based on convolutional neural networks. It reduces the computational burden through cross-stage partial connections while maintaining information fluency to improve the efficiency and accuracy of the model. The DSPPF dilated spatial pyramid pooling fast unit is an dilated spatial pyramid pooling technology that can effectively capture a wide range of contextual information in images through multi-scale pooling operations, which is crucial for pollen sample detection.

[0049] The feature network module includes a CSPNextPAFPN unit, a HAD unit, a CSP unit, and a CARAFE upsampling operator unit.

[0050] Among them, the CSPNextPAFP unit is an improved feature fusion network that combines a partial cross-stage convolutional network (CSP) and a feature pyramid network (FPN), effectively fusing features at different levels and improving the expressiveness of multi-scale features. The HAD (Hybrid Attention Domain) unit combines multiple attention mechanisms to enhance the model's ability to focus on features in different regions and scales, and improves the distinctiveness of image features. The CSP (Cross-Stage Partial) unit enhances feature learning efficiency by segmenting the network structure and connecting partially across stages, reduces the amount of calculation, and improves the performance of the model. The CARAFE (Content-Aware ReAssembly of FEatures) upsampling operator can reconstruct low-resolution feature maps into high-resolution images in a content-aware manner, maintain the image's detail information, and avoid the blurring introduced by traditional upsampling methods.

[0051] In a possible implementation, the mathematical calculation formula of the DSPPF expansion space pyramid pooling fast unit is specifically:

[0052]

[0053] Among them, M i represents the maximum distance between two non-zero values ​​in the i-th feature map, max represents maximization, M i+1 represents the maximum distance between two non-zero values ​​in the i+1th feature map, r i represents the expansion rate of the i-th layer convolution, f out1 Indicates passing The feature map of the convolutional layer, f out2 Indicates passing The feature map of the convolutional layer, f out3 Indicates passing The feature map of the convolutional layer, f out4 Indicates passing The feature map of the convolutional layer, f out represents the final output feature map, represents a convolutional layer of size 1×1, represents a convolutional layer of size 1×3 and dilation rate 1, represents a convolutional layer of size 3×3 and dilation rate 2, represents a convolutional layer with a size of 3×3 and a dilation rate of 3, f in Represents the input feature map, SEModule represents the SE attention module, and Concat represents the connection operation.

[0054] In the present invention, a 1×1 ordinary convolution is used in the first layer, and then three 3×3 convolutions with expansion rates of 1, 2, and 3 are applied to extract feature maps of different receptive fields. Finally, the feature maps of the four different receptive fields are concatenated and reduced in dimension, and then the SE attention module provides output.

[0055] The mathematical calculation formula of the CARAFE upsampling operator unit is as follows:

[0056]

[0057]

[0058] in, represents the convolution kernel of the lth feature point after upsampling, φ represents the kernel prediction module, N represents kernel normalization, that is, the Softmax activation function in space, X represents the feature map, represents the feature map of the lth feature point after upsampling, k encoder Indicates that the kernel size is k encoder The convolutional layer, θ represents the content-aware reorganization module, k up represents the size of the reorganization kernel, m represents the column direction of the window, and n represents the row direction of the window. , r represents k up Half of and rounded down, i and j represent the coordinate position of the lth feature point after transformation, σ represents the sampling rate, and Represents the original coordinate position of the lth feature point.

[0059] The mathematical calculation formula of HAD unit is as follows:

[0060]

[0061] Among them, f out represents the output feature map, ECAMoudle represents the ECA attention mechanism, and f in represents the input feature map, represents a convolutional layer of size 1×1, represents a convolutional layer of size 3×3 and dilation rate 1, represents a convolutional layer of size 3×3 and dilation rate 2, Represents a convolutional layer of size 3×3 and dilation rate 4.

[0062] In the present invention, after passing through the feature network module, the output feature maps are {4, 8, 16, 32} times the input downsampling size, and the HDA module is added after the feature map is downsampled to 32 times to enhance the model's ability to interpret contextual information. The model reduces the channel dimension and module parameters by a 1×1 convolution. The output after dimensionality reduction is then fed in parallel to three convolution branches and one residual attention branch. The three parallel convolution expansion rates are 1, 2, and 4, respectively, and the residual attention consists of spatial attention and channel attention.

[0063] Specifically, the backbone network module provides powerful feature extraction capabilities through CSPNext Block and DSPPF expanded spatial pyramid pooling fast units, which can extract multi-scale information under complex backgrounds, while the feature network module combines CSPNextPAFPN, HAD unit, CSP unit and CARAFE upsampling operator, effectively optimizing the feature fusion and upsampling process, and improving the classification accuracy and detection speed of the model.

[0064] In a possible implementation manner, after S2, the following steps are further included:

[0065] The pollen scans are labeled to determine first labeled data.

[0066] The first labeled data is input into the pollen detection model for training until the loss function value is less than the preset loss function value.

[0067] It should be noted that by labeling the pollen scans and generating the first labeled data, a high-quality training data set can be provided for the pollen detection model. By inputting these labeled data into the pollen detection model for training, the model can gradually optimize its recognition ability and improve the detection accuracy by learning the labeled information. During the training process, the loss function value is optimized until the model performance reaches the preset goal, which can ensure that the trained model has good generalization ability and accuracy, thereby achieving efficient detection and classification of pollen.

[0068] The calculation formula of the loss function value is as follows:

[0069]

[0070] Among them, C represents the total loss function, C cls represents the classification loss, C reg represents the regression loss, λ1, λ2 and λ3 all represent weight coefficients, C center represents the regional loss, CE represents the cross entropy loss, P represents the probability predicted by the model, and Y soft represents the soft label, IoU represents the index to measure the overlap between the predicted box and the real box, α and β both represent, pred Indicates the center coordinate of the prediction box, xgt Represents the center coordinates of the ground-truth box.

[0071] In the present invention, α and β are set to 10 and 3 by default.

[0072] In the present invention, during the training of the model, the initial learning rate is set to 0.003, the learning rate update strategy is cosine, the batch size is 32, the optimizer is Adamw, the iteration cycle is set to 300, and the training is completed on two 4080s.

[0073] S3: Input the pollen scan film into the pollen detection model and output the coordinates and width and height of the target pollen.

[0074] In the present invention, the scanned film is scaled to mpp=1, and then the scanned film is windowed with 2024×2024 size and 128×128 overlapping windows, and these sliding windows are respectively input into the detection model. For each window, the model will output the position and score of the pollen on the window, retain the pollen with a score greater than 0.3, and return the coordinates of the pollen to WSI, and remove the pollen in the overlapping part between the windows.

[0075] S4: Determine a cropped image of the target pollen according to the coordinates, width and height of the target pollen.

[0076] It should be noted that by determining the cropped image according to the coordinates and width and height of the target pollen, the pollen area can be accurately extracted and irrelevant background information can be removed, thereby improving the efficiency of subsequent processing and analysis.

[0077] S5: Establish a pollen filtration model based on Vit.

[0078] It should be noted that establishing a pollen filtering model can effectively screen out clear pollen samples and filter out fuzzy or non-pollen targets, thereby improving the accuracy of subsequent classification and analysis, making the pollen detection process more automated, avoiding the limitations of manual intervention, saving time and costs, and enhancing the generalization ability of the model.

[0079] In the present invention, the main framework of the pollen filtering model is the vision transformer (Vit). Since the Vit-based model is usually several times slower than the convolutional network, and in order to solve the problem of limited device resources such as edge devices, the present invention changes the inefficient design of Vit and proposes a dimensionally consistent network with 4D feature implementation and 3D multi-head self-attention to offset the frequent reshaping operations.

[0080] In a possible implementation, the pollen filtering model includes: a MetaFormer module, a Patch Embeding module, a dimension consistency processing module, a multi-head attention module, and a delay-driven lightweight module.

[0081] Among them, the MetaFormer module is a module based on the Transformer architecture, which combines convolutional networks and attention mechanisms and can efficiently process multi-scale features in images. The MetaFormer module effectively improves the feature extraction capability through parallel processing and self-attention mechanism, and performs feature fusion at different scales to enhance important information in the image. The Patch Embedding module is a commonly used technology in image processing and Transformer models. It mainly cuts the input image into several small patches (patches), and then maps each small patch to a feature vector through linear transformation. The dimensional consistency processing module is used to ensure the dimensional alignment of feature maps between different layers or modules in the network. The multi-head attention module is a core component in the Transformer architecture. By calculating the attention mechanism in parallel on multiple "heads", the model can focus on different feature parts from multiple angles, enhancing the model's ability to capture global information. The delay-driven lightweight module reduces latency and improves processing speed by dynamically selecting appropriate network paths.

[0082] In the present invention, the essence of the dimension consistency processing module is to divide the MB module into two networks, 4D and 3D. Linear projection and attention are performed on 3D, so that global modeling capabilities can be enjoyed without sacrificing the efficiency of multi-head attention. 4D is mainly used in the beginning of the network output, mainly implemented in the form of Conv-net. First, the input image is patch embbeded by two 3×3 Convs with stride 2, and then by MB 4D Extract low-level semantics and process all MB 4D Module, execute reshape once again to enter MB 3D .

[0083] In a possible implementation manner, the pollen filtering model performs pollen filtering using the formula:

[0084]

[0085] Where y represents the output of the pollen filtration model, MB i represents the i-th MB module, PatchEmbed represents image block embedding, B represents the batch size, H represents the input image height, W represents the input image width, x0 represents the input image, Represents the accumulation operation from i to m.

[0086] The mathematical calculation formula of the MetaFormer module is as follows:

[0087]

[0088] Among them, x i+1 represents the intermediate features flowing to the i+1th MB module, x i represents the intermediate features flowing to the i-th MB module, MB i represents the i-th MB module, MLP represents a multi-layer perceptron, TokenMixer represents a 4-dimensional multi-head self-attention module, BN represents batch normalization, Conv2D represents a two-dimensional convolution, RELU represents an activation function, Upsample represents upsampling, V represents the value in the input, V local It represents the local feature enhancement operation, which consists of a depthwise separable convolution and a BN layer. TH1 and TH2 both represent convolutions of size 1×1. Softmax represents the activation function. Attention represents the attention operation. DepthwiseConv2D represents the depthwise separable convolution.

[0089] The mathematical calculation formula of the Patch Embedding module is as follows:

[0090]

[0091] in, represents the feature map obtained after the PatchEmbed operation, C j Indicates the number of channels of the jth Stage.

[0092] The mathematical calculation formula of the dimension consistency processing module is as follows:

[0093]

[0094] Among them, I i represents the output of the i-th Stage, Pool represents the pooling operation, Conv B Indicates the convolution followed by the BN layer, Conv B,G Conv B Then connect it to the GeLU activation layer.

[0095] The mathematical calculation formula of the multi-head attention module is as follows:

[0096]

[0097] Among them, Linear represents the linear layer, MHSA represents multi-head self-attention, LN represents the LayerNorm layer, and Linear G represents a linear layer followed by a GeLU activation layer, Q represents query, K represents key, V represents value, and b represents bias. T Represents a transpose operation.

[0098] The mathematical calculation formula of the delay drive lightweight module is as follows:

[0099]

[0100] Among them, MP r represents the meta-path of the rth stage, represents the i-th 4-dimensional MetaFormer module in the meta-path, Represents the i-th 3D MetaFormer module in the meta-path.

[0101] In a possible implementation manner, after S5, the method further includes:

[0102] The target pollen cropped image is labeled to determine second labeled data, wherein the second labeled data specifically includes: clear pollen, fuzzy pollen, and non-pollen.

[0103] The second labeled data is input into the pollen filtration model for training.

[0104] It should be noted that by labeling the target pollen and determining the second labeling data, the pollen filtering model can more accurately distinguish between clear pollen, fuzzy pollen and non-pollen, enabling the model to make more accurate judgments in complex backgrounds and improve the accuracy and robustness of pollen classification. In addition, the introduction of the second labeling data can enhance the learning effect of the model, reduce misidentification, improve the filtering effect, and ultimately optimize the overall performance of the pollen detection process.

[0105] In the present invention, the pollen filtering model adopts CrossEntropyLoss cross entropy loss. During the training process, the initial learning rate is set to 0.003, the learning rate update strategy is cosine, the batch size is 32, the optimizer is Adamw, the iteration cycle is set to 100, and the training is completed on a single block 4080.

[0106] S6: Input the cropped image to a pollen filtering model, and output pollen categories, where the pollen categories include clear pollen, blurred pollen, and non-pollen.

[0107] It should be noted that inputting the cropped image into the pollen filtering model can ensure that the model only processes image areas containing clear pollen, removes irrelevant background and noise, improves the accuracy and efficiency of the model, reduces the waste of computing resources, and focuses on the processing of high-quality data.

[0108] S7: According to the pollen category, determine the coordinates and width and height of the clear pollen.

[0109] In the present invention, after the detection model obtains the coordinates, width and height of the pollen sample, the original image is cropped to obtain the corresponding target, and then the target is scaled to 384×384 size as model input. The model outputs the probabilities of three categories: clear pollen, fuzzy pollen, and impurities, and the category with the highest probability is taken as the category of the pollen.

[0110] S8: Determine a clear pollen cropping image according to the coordinates, width and height of the clear pollen.

[0111] It should be noted that by determining the cropped image based on the coordinates, width and height of the clear pollen, the pollen area can be accurately extracted and background noise and irrelevant information can be removed. The cropped image reduces unnecessary calculations, improves the model processing speed and efficiency, and avoids interference with blurred or erroneous data, further improving the accuracy of pollen detection.

[0112] S9: Based on Vit, establish a family classification model.

[0113] It should be noted that establishing a family classification model can effectively and accurately classify clear pollen and clarify its family category. By in-depth learning of the subtle characteristics of pollen, the accuracy of pollen classification is improved. The family classification model can process complex pollen samples, identify pollen of similar families and avoid errors caused by human intervention.

[0114] In the present invention, the family classification model uses vision transformer as the basic framework of the model. The improvement of the model is consistent with the above-mentioned pollen filtering model and will not be repeated here.

[0115] In a possible implementation manner, after S8, the method further includes:

[0116] The clear pollen cropped image is labeled to determine the third labeled data.

[0117] The third labeled data is input into the family classification model for training.

[0118] In the present invention, the clear pollen obtained by the filtering model is divided into the categories of Fagaceae Quercus, Asteraceae Artemisia, Chenopodiaceae Kochia, Amaranthaceae Amaranth, Chenopodiaceae Chenopodia, Cyperus, Asteraceae Taraxacum, Asteraceae Ambrosia, Pinaceae Picea, Pinaceae Pinus, Betulaceae Corylus, Moraceae Humulus, Taxaceae Taxus, Salicaceae Salix, Salicaceae Salix, and Ulmus. During the training of the model, the initial learning rate is set to 0.003, the learning rate update strategy is cosine, the batchsize is 64, the optimizer is Adamw, the iteration cycle is set to 200, and the training is completed on a single block 4080.

[0119] S10: Inputting the clear pollen cropped image into the family classification model to determine the family category of the clear pollen.

[0120] It should be noted that after inputting the clear pollen cropped image into the family classification model, it can ensure that the model focuses on high-quality pollen samples for classification and avoids interference factors. By accurately identifying the family categories of pollen, the model can significantly improve the accuracy and reliability of classification, make the family analysis of pollen automated, reduce human errors, and improve processing speed. At the same time, it provides more effective data support for pollen research and allergy prevention and control.

[0121] In the present invention, the coordinates, width and height of the clear pollen are cut out from the original image and scaled to 384×384 size as the input of the classification model. The family classification model outputs the probability that the pollen belongs to each family, and takes the category with the highest probability as the family to which the pollen belongs.

[0122] S11: Count the number of pollen in each family and genus category and the total number of pollen in the pollen scanning film to complete the pollen detection.

[0123] It should be noted that by counting the number of pollen in each family and genus category and the total number of pollen scans, comprehensive pollen distribution data can be provided, which helps to accurately assess the concentration of different pollen types.

[0124] In the present invention, clear pollen is obtained through a filtering model, and the family category of the clear pollen is obtained through a family classification model. The number of pollen in each family category is added together to obtain the total number of pollen on the scanned film.

[0125] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0126] In an embodiment of the present invention, by establishing a pollen detection model based on the RTMDET network, it is possible to accurately detect targets in pollen images, enhance the model's ability to recognize pollen samples, reduce false detections and missed detections, and thus improve the accuracy of pollen detection. By utilizing the pollen filtering model, invalid or fuzzy pollen can be further filtered out according to the clarity and classification of the target pollen, thereby improving the accuracy of the final detection result and ensuring that valid data is used for subsequent analysis. By establishing a family classification model, it is possible to accurately classify clear pollen and clarify its family category, thereby meeting the needs of different fields such as allergy research, ecological monitoring, and agricultural production. The present invention can adapt to the pollen detection needs in various actual usage scenarios during pollen detection, and improve the efficiency and accuracy of pollen detection.

[0127] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a pollen detection system based on a pollen scanning film provided by the present invention.

[0128] The present invention further provides a pollen detection system 20 based on a pollen scanning film, which is applied to the above-mentioned pollen detection method based on a pollen scanning film, and comprises:

[0129] Processor 201.

[0130] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, the pollen detection method based on pollen scanning film as in the method embodiment is implemented.

[0131] The pollen detection system 20 based on pollen scanning film provided by the present invention can execute the above-mentioned pollen detection method based on pollen scanning film and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0132] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0133] In an embodiment of the present invention, by establishing a pollen detection model based on the RTMDET network, it is possible to accurately detect targets in pollen images, enhance the model's ability to recognize pollen samples, reduce false detections and missed detections, and thus improve the accuracy of pollen detection. By utilizing the pollen filtering model, invalid or fuzzy pollen can be further filtered out according to the clarity and classification of the target pollen, thereby improving the accuracy of the final detection result and ensuring that valid data is used for subsequent analysis. By establishing a family classification model, it is possible to accurately classify clear pollen and clarify its family category, thereby meeting the needs of different fields such as allergy research, ecological monitoring, and agricultural production. The present invention can adapt to the pollen detection needs in various actual usage scenarios during pollen detection, and improve the efficiency and accuracy of pollen detection.

[0134] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0135] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0136] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0137] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0138] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0139] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0140] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0145] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0146] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a pollen detection method based on pollen scanning films as in the method embodiment is implemented.

[0147] A computer-readable storage medium provided by the present invention can implement the steps and effects of the pollen detection method based on pollen scanning film of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0148] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0149] In an embodiment of the present invention, by establishing a pollen detection model based on the RTMDET network, it is possible to accurately detect targets in pollen images, enhance the model's ability to recognize pollen samples, reduce false detections and missed detections, and thus improve the accuracy of pollen detection. By utilizing the pollen filtering model, invalid or fuzzy pollen can be further filtered out according to the clarity and classification of the target pollen, thereby improving the accuracy of the final detection result and ensuring that valid data is used for subsequent analysis. By establishing a family classification model, it is possible to accurately classify clear pollen and clarify its family category, thereby meeting the needs of different fields such as allergy research, ecological monitoring, and agricultural production. The present invention can adapt to the pollen detection needs in various actual usage scenarios during pollen detection, and improve the efficiency and accuracy of pollen detection.

[0150] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0151] There are a few points to note:

[0152] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to the general design.

[0153] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0154] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0155] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A pollen detection method based on pollen scanning film, characterized in that: include: S1: Obtain the pollen scan film; S2: Establish a pollen detection model based on the rtmdet network; S3: Input the pollen scan film into the pollen detection model, and output the coordinates and width and height of the target pollen; S4: Determine a cropped image of the target pollen according to the coordinates and width and height of the target pollen; S5: Establish pollen filtration model based on Vit; S6: inputting the cropped image into the pollen filtering model, and outputting pollen categories, wherein the pollen categories include clear pollen, blurred pollen, and non-pollen; S7: Determine the coordinates and width and height of the clear pollen according to the pollen category; S8: Determine a clear pollen cropped image according to the coordinates and width and height of the clear pollen; S9: Establish a family classification model based on Vit; S10: inputting the clear pollen cropped image into the family classification model to determine the family category of the clear pollen; S11: Count the number of pollens of each family and genus and the total number of pollens in the pollen scanning film to complete pollen detection.

2. The pollen detection method based on pollen scanning film according to claim 1 is characterized in that: The pollen detection model includes: a backbone network module and a feature network module; The backbone network module includes a CSPNext Block unit and a DSPPF expansion space pyramid pooling fast unit; The feature network module includes a CSPNextPAFPN unit, a HAD unit, a CSP unit and a CARAFE upsampling operator unit.

3. The pollen detection method based on pollen scanning film according to claim 2 is characterized in that: The mathematical calculation formula of the DSPPF expansion space pyramid pooling fast unit is specifically: ； Among them, M i represents the maximum distance between two non-zero values ​​in the i-th feature map, max represents maximization, M i+1 represents the maximum distance between two non-zero values ​​in the i+1th feature map, r i represents the expansion rate of the i-th layer convolution, f out1 Indicates passing The feature map of the convolutional layer, f out2 Indicates passing The feature map of the convolutional layer, f out3 Indicates passing The feature map of the convolutional layer, f out4 Indicates passing The feature map of the convolutional layer, f out represents the final output feature map, represents a convolutional layer of size 1×1, represents a convolutional layer of size 1×3 and dilation rate 1, represents a convolutional layer of size 3×3 and dilation rate 2, represents a convolutional layer with a size of 3×3 and a dilation rate of 3, f in Represents the input feature map, SEModule represents the SE attention module, and Concat represents the connection operation; The mathematical calculation formula of the CARAFE upsampling operator unit is specifically: ； ； in, represents the convolution kernel of the lth feature point after upsampling, φ represents the kernel prediction module, N represents kernel normalization, that is, the Softmax activation function in space, X represents the feature map, represents the feature map of the lth feature point after upsampling, k encoder Indicates that the kernel size is k encoder The convolutional layer, θ represents the content-aware reorganization module, k up represents the size of the reorganization kernel, m represents the column direction of the window, and n represents the row direction of the window. , r represents k up Half of and rounded down, i and j represent the coordinate position of the lth feature point after transformation, σ represents the sampling rate, and Represents the original coordinate position of the lth feature point; The mathematical calculation formula of the HAD unit is specifically: ； Among them, f out represents the output feature map, ECAMoudle represents the ECA attention mechanism, and f in represents the input feature map, represents a convolutional layer of size 1×1, represents a convolutional layer of size 3×3 and dilation rate 1, represents a convolutional layer of size 3×3 and dilation rate 2, Represents a convolutional layer of size 3×3 and dilation rate 4.

4. The pollen detection method based on pollen scanning film according to claim 1, characterized in that: The S2 also includes: Annotating the pollen scan film to determine first annotation data; The first labeled data is input into the pollen detection model for training until the loss function value is less than a preset loss function value.

5. The pollen detection method based on pollen scanning film according to claim 4 is characterized in that: The calculation formula of the loss function value is specifically: ； Among them, C represents the total loss function, C cls represents the classification loss, C reg represents the regression loss, λ1, λ2 and λ3 all represent weight coefficients, C center represents the regional loss, CE represents the cross entropy loss, P represents the probability predicted by the model, and Y soft represents a soft label, IoU represents an indicator to measure the overlap between the predicted box and the true box, α and β represent hyperparameters, and x pred Indicates the center coordinate of the prediction box, x gt Represents the center coordinates of the ground-truth box.

6. The pollen detection method based on pollen scanning film according to claim 1, characterized in that: The pollen filtering model includes: a Meta Former module, a Patch Embeding module, a dimension consistency processing module, a multi-head attention module and a delay-driven lightweight module.

7. The pollen detection method based on pollen scanning film according to claim 6, characterized in that: The specific formula for pollen filtration by the pollen filtration model is: ； Where y represents the output of the pollen filtration model, MB i represents the i-th MB module, PatchEmbed represents image block embedding, B represents the batch size, H represents the input image height, W represents the input image width, x0 represents the input image, Represents the accumulation operation from i to m; The mathematical calculation formula of the MetaFormer module is specifically: ； Among them, x i+1 represents the intermediate features flowing to the i+1th MB module, x i represents the intermediate features flowing to the i-th MB module, MB i represents the i-th MB module, MLP represents a multi-layer perceptron, TokenMixer represents a 4-dimensional multi-head self-attention module, BN represents batch normalization, Conv2D represents a two-dimensional convolution, RELU represents an activation function, Upsample represents upsampling, V represents the value in the input, V local It represents the local feature enhancement operation, which consists of a depthwise separable convolution and a BN layer. TH1 and TH2 both represent convolutions of size 1×1. Softmax represents the activation function. Attention represents the attention operation. DepthwiseConv2D represents the depthwise separable convolution. The mathematical calculation formula of the Patch Embeding module is specifically: ； in, represents the feature map obtained after the PatchEmbed operation, C j Indicates the number of channels of the jth Stage; The mathematical calculation formula of the dimension consistency processing module is specifically: ； Among them, I i represents the output of the i-th Stage, Pool represents the pooling operation, Conv B Indicates the convolution followed by the BN layer, Conv B,G Conv B Then connect to GeLU activation layer; The mathematical calculation formula of the multi-head attention module is specifically: ； Among them, Linear represents the linear layer, MHSA represents multi-head self-attention, LN represents the LayerNorm layer, and Linear G represents a linear layer followed by a GeLU activation layer, Q represents query, K represents key, V represents value, and b represents bias. T Represents a transpose operation; The mathematical calculation formula of the delayed drive lightweight module is specifically: ； Among them, MP r represents the meta-path of the rth stage, represents the i-th 4-dimensional MetaFormer module in the meta-path, Represents the i-th 3D MetaFormer module in the meta-path.

8. The pollen detection method based on pollen scanning film according to claim 1, characterized in that: The S5 also includes: Annotating the target pollen cropped image to determine second annotation data, wherein the second annotation data specifically includes: clear pollen, fuzzy pollen, and non-pollen; The second labeled data is input into the pollen filtration model for training.

9. The pollen detection method based on pollen scanning film according to claim 1, characterized in that: The S9 and later parts further include: Annotating the clear pollen cropped image to determine third annotation data; The third labeled data is input into the family classification model for training.

10. A pollen detection system based on pollen scanning film, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the pollen detection method based on pollen scanning films as described in any one of claims 1 to 9 is implemented.

Images