A pollen image intelligent recognition method and system

Abstract

The embodiment of the present invention relates to the technical field of pollen identification, and discloses a method for intelligent identification of pollen images, including: obtaining a pollen image to be identified; extracting a first pollen image of each pollen to be identified according to the position information of each pollen in the pollen image to be identified; inputting the second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified into a species classification model for identification to determine the species information of each pollen to be identified, and outputting the pollen image of each pollen to be identified and the corresponding species information. The intelligent identification method of pollen images in the embodiment of the present invention first extracts a single pollen high-definition image through a pollen high-definition image extraction model, which can avoid impurity images and extract high-definition feature information of pollen; further, the pollen classification model fused with phenological features is used to accurately classify pollen species, which establishes a high-precision, multi-species pollen image intelligent identification method based on phenological classification.

Description

Technical Field

[0001] The present invention relates to the technical field of pollen recognition, and in particular to an intelligent recognition method and system for pollen images. Background Art

[0002] At present, there is a basic lack of observation and research on airborne allergenic pollen in southern China. Traditional airborne pollen monitoring methods mainly rely on artificial field collection of pollen slides for indoor microscope identification, which results in low data acquisition efficiency and poor data quality. The lack of pollen concentration and type monitoring data has seriously restricted the forecast and early warning of allergenic pollen and limited the prevention and control of pollen allergies. On the other hand, microscope images of pollen slides often contain a lot of dust, impurities and other particles, which cause a lot of interference to manual identification and also bring challenges to the development of automatic monitoring methods. Therefore, designing a solution that can perform efficient monitoring and identification has become a technical problem that needs to be solved urgently by technicians in this field. Summary of the invention

[0003] In view of the above-mentioned defects, an embodiment of the present invention discloses an intelligent identification method for pollen images, which can realize accurate and efficient classification and type identification.

[0004] The first aspect of the embodiment of the present invention discloses an intelligent recognition method for pollen images, comprising:

[0005] Acquire a pollen image to be identified, wherein the pollen image to be identified is a depth-of-field composite image formed based on multiple images with different focus points;

[0006] Inputting the pollen image to be identified into a pollen identification model to perform pollen feature extraction to determine the position information of each pollen in the pollen image to be identified, and extracting a first pollen image of each pollen to be identified according to the position information of each pollen in the pollen image to be identified;

[0007] performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain second pollen images of each to-be-identified pollen;

[0008] The second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified are input into the species classification model for identification to determine the species information of each pollen to be identified, and the pollen image of each pollen to be identified and the corresponding species information are output.

[0009] As an optional implementation, in the first aspect of the embodiment of the present invention, the pollen recognition model is constructed based on a YOLOv7 model based on feature fusion;

[0010] The pollen recognition model is obtained by the following steps:

[0011] Acquire a pollen sample image, and annotate the pollen sample image with information using an image annotation tool to obtain image annotation information for each pollen sample image;

[0012] Constructing a sample training set according to the pollen sample image and the image annotation information; and dividing the sample training set to obtain a training set, a validation set, and a test set;

[0013] Performing a preprocessing operation on the pollen sample image;

[0014] The pollen sample image that has undergone the preprocessing operation is input into the pollen recognition model with completed parameter configuration for training and parameter configuration adjustment until the corresponding pollen recognition model meets the training requirements and the current optimal model parameters are saved.

[0015] As an optional implementation, in the first aspect of the embodiment of the present invention, the image annotation information includes size information of the annotation box; after obtaining the pollen sample image and annotating the pollen sample image with information through the image annotation tool to obtain the image annotation information for each pollen sample image, it also includes:

[0016] Performing data enhancement operation on the pollen sample image to expand the data sample; the data enhancement operation includes performing one or more of horizontal flip operation, vertical flip operation, clockwise flip operation of 90°, clockwise flip operation of 180°, clockwise flip operation of 270°, transposition operation and image mirroring operation on the original data using Python language and OpenCV coding;

[0017] Or, after obtaining the pollen sample image and annotating the pollen sample image with information by an image annotation tool to obtain image annotation information for each pollen sample image, the method further includes:

[0018] Determine a corresponding clustering data set according to the size information of the annotation boxes corresponding to all the pollen sample images, and determine a corresponding number of clusters according to the clustering data set;

[0019] Selecting K data objects from the clustering data set as initial clustering centers;

[0020] For each data object in the clustering data set, the distance between the data object and the initial cluster is calculated, and each data object is assigned to the cluster to which the cluster center closest to the data object belongs;

[0021] Repeat the above steps until the corresponding cluster center no longer changes, and output the final clustering result, which includes the cluster to which each data object belongs and the cluster center of each cluster;

[0022] A priori box is generated according to the clustering result to serve as an initial value of the model prediction box.

[0023] As an optional implementation, in the first aspect of the embodiment of the present invention, after constructing a sample training set according to the pollen sample image and the image annotation information, the method further includes:

[0024] Get the pre-trained weight file based on the VOC2017 dataset;

[0025] Modifying the configuration of the pre-training weight file according to the sample training set; the configuration modification includes changing the number of categories, specifying the data set path and training parameters;

[0026] The pollen recognition model includes a convolution layer and a pooling layer, and a CA attention module is added between the convolution layer and the pooling layer, and the feature map is optimized through the CA attention module.

[0027] As an optional implementation, in the first aspect of the embodiment of the present invention, the step of performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain the second pollen images of each to-be-identified pollen includes:

[0028] The extracted first pollen images of each to-be-identified pollen are padded to 640*640 pixels using a black background to obtain a corresponding second pollen image.

[0029] As an optional implementation, in the first aspect of the embodiment of the present invention, the second pollen image of each to-be-identified pollen and the acquisition time of the to-be-identified pollen image are input into the species classification model for identification to determine the species information of each to-be-identified pollen, including:

[0030] Determining month information corresponding to the pollen image to be identified according to the acquisition time of the pollen image to be identified;

[0031] Determine a species classification model corresponding to the phenological information according to the month information;

[0032] Inputting the second pollen image of each to-be-identified pollen into a species classification model corresponding to the phenological information for identification to determine the species information of each to-be-identified pollen;

[0033] After outputting the pollen images of each pollen to be identified and the corresponding type information, the method further includes:

[0034] The priori frame size information is output according to the priori frame size corresponding to the pollen image to be identified as the pollen size information, wherein the priori frame size information includes the size of the priori frame and the area of ​​the priori frame.

[0035] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the intelligent identification method further includes:

[0036] receiving pollen collection method information input by a user, wherein the pollen collection method information is an air volume method or an air deposition method;

[0037] If the pollen collection method is the air volume method, the first pollen concentration information of the corresponding collection point is determined according to the first concentration calculation formula, and the first concentration calculation formula is: C1=n1×N1 / V, wherein n1 is the number of pollens identified on each picture, N1 is the number of pollen images taken in each collection period, V is the amount of gas collected in the corresponding period, and C1 is the first pollen concentration information;

[0038] If the pollen collection method is the air deposition method, the second pollen concentration information of the corresponding collection point is determined according to the second concentration calculation formula, and the second concentration calculation formula is: C2=n2×N2 / m×1000, wherein m is the area of ​​the glass slide collected in the corresponding period, n2 is the number of pollens identified on each picture, N2 is the number of pollen images taken in each collection period, and C2 is the second pollen concentration information;

[0039] The pollen image to be identified is obtained by the following steps:

[0040] A preset number of slide image information is obtained at the pollen slide by a stereo microscope; the focal distance between each slide image information is 3 μm;

[0041] A preset number of slide image information are synthesized according to a focus image synthesis algorithm to obtain a corresponding pollen image to be identified.

[0042] A second aspect of an embodiment of the present invention discloses an intelligent pollen image recognition system, comprising:

[0043] An acquisition module is used to acquire a pollen image to be identified, wherein the pollen image to be identified is a depth-of-field composite image formed based on multiple images with different focus points;

[0044] Feature extraction module: used for inputting the pollen image to be identified into the pollen recognition model to perform pollen feature extraction to determine the position information of each pollen in the pollen image to be identified, and extracting the first pollen image of each pollen to be identified according to the position information of each pollen in the pollen image to be identified;

[0045] Image processing module: used for performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain the second pollen images of each to-be-identified pollen;

[0046] Species identification module: used to input the second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified into the species classification model for identification to determine the species information of each pollen to be identified, and output the pollen image of each pollen to be identified and the corresponding species information.

[0047] The third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the intelligent recognition method for pollen images disclosed in the first aspect of the embodiment of the present invention.

[0048] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the intelligent identification method for pollen images disclosed in the first aspect of an embodiment of the present invention.

[0049] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0050] The intelligent recognition method for pollen images in the embodiment of the present invention first extracts a single pollen high-definition image through a pollen high-definition image extraction model, which can avoid impurity images and extract high-definition feature information of pollen; further, the pollen classification model fused with phenological features is used to accurately classify pollen types. This method establishes a high-precision, multi-species pollen image intelligent recognition method based on phenological classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flow chart of the intelligent identification method of pollen images disclosed in the embodiment of the present invention;

[0053] Figure 2 It is a schematic diagram of the process of constructing the pollen recognition model disclosed in the embodiment of the present invention;

[0054] Figure 3 It is a schematic diagram of the pollen picture annotation shooting process disclosed in an embodiment of the present invention;

[0055] Figure 4 is a schematic diagram of displaying the annotation operation interface disclosed in an embodiment of the present invention;

[0056] Figure 5This is a pollen exposure display diagram disclosed in an embodiment of the present invention that is collected through field observation;

[0057] Figure 6 It is a display diagram of identified pollen disclosed in an embodiment of the present invention;

[0058] Figure 7 It is a structural schematic diagram of an intelligent identification system for pollen images provided by an embodiment of the present invention;

[0059] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0062] Airborne allergenic pollen is one of the main causes of allergic diseases, especially airway allergic diseases. After atopic patients come into contact with pollen allergens, they develop a series of pathological and physiological processes of hay fever symptoms including sneezing, nasal congestion, tearing, asthma, etc., which bring serious inconvenience to human health and life. Therefore, accurate and efficient monitoring of allergenic pollen is particularly important for the prevention and treatment of pollen allergies. Based on pollen monitoring data, accurate pollen concentration prediction and allergy risk warning are carried out, so that pollen allergy patients can grasp the recent pollen situation in the air in real time, so as to reasonably arrange daily life and treatment plans.

[0063] At present, in the field of pollen monitoring and forecasting, most of the work of airborne pollen identification still relies on manual work, including manual film collection, microscope photography, photo observation and identification. The observation process is slow and not timely. At the same time, the identification of pollen images often requires experts with relevant knowledge to conduct effective identification, which is easily affected by human subjective factors, and inaccurate classification often occurs. In addition, pollen images have the characteristics of low clarity and similar shapes. Maintaining high-precision pollen image recognition is a challenging task. In addition, as the types of pollen images gradually increase, the recognition accuracy of manual recognition methods will also decrease, which brings great difficulties to the effective identification of pollen images. Especially in the field environment, the images of pollen films taken under a microscope often contain more dust, non-biological or biological impurity particles, which brings greater interference and challenges to pollen identification.

[0064] On the other hand, affected by factors such as geographical environment, climate, and vegetation type, the distribution and regularity of the main airborne allergenic pollens in different regions are different. The existing pollen monitoring stations in the world are mainly concentrated in mid- and high-latitude regions, where there are relatively few plant species and only more than ten major allergenic pollen species in the air. In the tropical and subtropical southern China, there are many plant species, high pollen concentrations, and more complex species. The types of allergenic pollens vary greatly from month to month, and there are nearly 100 types of pollen that dominate throughout the year. Traditional manual methods are difficult to achieve accurate and efficient pollen observation. In view of the shortcomings of manual observation, the automatic recognition of pollen images can greatly reduce the time required for image recognition, solving the problem of wasting a lot of manpower, material resources, and financial resources in traditional methods. Therefore, the realization of automatic recognition of pollen images has important practical significance. However, it is worth noting that the image features of common allergenic pollens in spring, such as Gramineae and Moraceae, may be similar to those of some pollen species that appear in summer and autumn in the automatic recognition system. If the system is not combined with seasonal prior knowledge or database for correction, it is very easy to make misjudgments. This cross-season misidentification not only affects the accuracy of pollen monitoring and forecasts, but may also mislead the public's assessment and prevention of allergy risks.

[0065] Based on this, the embodiments of the present invention disclose an intelligent recognition method, system, electronic device and storage medium for pollen images. The method first extracts a single pollen high-definition image through a pollen high-definition image extraction model, which can avoid impurity images and extract high-definition feature information of pollen; further, the pollen classification model fused with phenological features is used to accurately classify pollen types. This method establishes a high-precision, multi-species pollen image intelligent recognition method based on phenological classification.

[0066] Embodiment 1

[0067] See also Figure 1 , Figure 1It is a flow chart of the intelligent identification method of pollen images disclosed in the embodiment of the present invention. Among them, the execution subject of the method described in the embodiment of the present invention is an execution subject composed of software and / or hardware, and the execution subject can receive relevant information by wired or / and wireless means, and can send certain instructions. Of course, it can also have certain processing functions and storage functions. The execution subject can control multiple devices, such as a remote physical server or cloud server and related software, or it can be a local host or server and related software that performs related operations on a device placed somewhere. In some scenarios, multiple storage devices can also be controlled, and the storage devices can be placed in the same place or different places as the devices. For example Figure 1 As shown, the intelligent recognition method based on pollen images includes the following steps:

[0068] S101: Acquire a pollen image to be identified, wherein the pollen image to be identified is a depth-of-field composite image formed based on multiple images with different focus points;

[0069] S102: Inputting the pollen image to be identified into a pollen identification model to perform pollen feature extraction to determine the position information of each pollen in the pollen image to be identified, and extracting a first pollen image of each pollen to be identified according to the position information of each pollen in the pollen image to be identified;

[0070] S103: performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain second pollen images of each to-be-identified pollen;

[0071] S104: Inputting the second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified into the species classification model for identification to determine the species information of each pollen to be identified, and outputting the pollen image of each pollen to be identified and the corresponding species information.

[0072] Specifically, the pollen image to be identified is obtained by the following steps:

[0073] A preset number of slide image information is obtained at the pollen slide by a stereo microscope; the focal distance between each slide image information is 3 μm;

[0074] A preset number of slide image information are synthesized according to a focus image synthesis algorithm to obtain a corresponding pollen image to be identified.

[0075] When acquiring images, you need to make standardized pollen slides first. Take the flower material from the plastic bag and operate under a NIKON stereo microscope. Hold the flower with tweezers in your left hand and use a dissecting needle in your right hand to gently push the anther to expose the pollen. Then use a small piece of gelatin to stick an appropriate amount of pollen and fix the gelatin on the slide. Then heat the bottom of the slide until the gelatin melts and quickly cover it with a coverslip. Standardized pollen image acquisition: The shooting process is as follows, that is, when taking a single depth of field composite picture, 15 pictures need to be taken, using helicon focus for synthesis, with a spacing of about 3μm each time. Convert the image to JPG format for saving, and the image size must not be less than 1280*1280 pixels; specifically, Figure 3 As shown in the figure, laboratory standard pollen slides need to take 50 images as a pollen image dataset of a single species, so a total of 750 images of pollen at different focal planes are required for an image dataset of one pollen species. Historical airborne pollen slides are prepared and photographed in the above manner to obtain a pollen image dataset representing the state of field observation.

[0076] The solution of the embodiment of the present invention uses a depth-of-field synthesis image formed based on multiple different focus points as the pollen image to be identified. This method can effectively enhance the clarity and detail expression of the image, especially for pollen, a tiny and complex object. The depth-of-field synthesis technology can ensure that pollen particles at different depths in the image can be clearly presented, thereby improving the accuracy of subsequent feature extraction and identification.

[0077] The solution of the embodiment of the present invention inputs the pollen image to be identified into the pollen recognition model for feature extraction and position information determination, which realizes the automation and intelligence of pollen recognition. Compared with the traditional manual recognition method, this method can greatly reduce labor costs, improve recognition efficiency, and avoid errors that may be caused by human factors.

[0078] The solution of the embodiment of the present invention further processes the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain a clearer second pollen image with more prominent features. This process helps to further improve the recognition accuracy of the species classification model, because more refined image input can make it easier for the model to capture key feature information.

[0079] The solution of the embodiment of the present invention inputs the second pollen image of each pollen to be identified and additional information such as the collection time into the species classification model for identification. This method makes full use of multi-source information. Information such as the collection time may be particularly important for the identification of certain specific types of pollen because they may be affected by environmental factors such as season and climate. The integration of this information can further improve the recognition and generalization capabilities of the model.

[0080] Finally, the method can output the pollen image and corresponding species information of each pollen to be identified, providing users with intuitive and comprehensive identification results. This is of great significance for research and application in the fields of ecology, botany, agriculture, etc., and can help researchers quickly and accurately obtain pollen species information, and then carry out more in-depth research and analysis.

[0081] More preferably, the pollen recognition model is constructed based on the YOLOv7 model;

[0082] like Figure 2 As shown, the pollen recognition model is obtained through the following steps:

[0083] S102a: Acquire pollen sample images, and annotate the pollen sample images with information using an image annotation tool to obtain image annotation information for each pollen sample image;

[0084] S102b: constructing a sample training set according to the pollen sample image and the image annotation information; and dividing the sample training set to obtain a training set, a validation set and a test set;

[0085] S102c: performing a preprocessing operation on the pollen sample image;

[0086] S102d: Inputting the pollen sample image that has undergone the preprocessing operation into the pollen recognition model with completed parameter configuration for training and parameter configuration adjustment until the corresponding pollen recognition model meets the training requirements, and saving the current optimal model parameters.

[0087] like Figure 4 As shown in the figure, this method uses the Create Circle function in the Labeling tool in python to annotate the pollen species information in the image. The characteristic area of ​​each pollen in the image is annotated in the form of a matrix. Each image generates a corresponding annotation result xml file, which contains information such as the target position of the image, the anchor frame size, and the label of the pollen image performance characteristics. The label of the pollen performance characteristics is represented by the name of the pollen species, that is, each anchor frame corresponds to a pollen species name. After image feature annotation and data enhancement, a training sample set is formed, and a pollen recognition model is constructed using a deep learning-based target detection algorithm.

[0088] More preferably, the image annotation information includes size information of the annotation box; after obtaining the pollen sample image and annotating the pollen sample image with information through an image annotation tool to obtain image annotation information for each pollen sample image, the method further includes:

[0089] Performing data enhancement operation on the pollen sample image to expand the data sample; the data enhancement operation includes performing one or more of horizontal flip operation, vertical flip operation, clockwise flip operation of 90°, clockwise flip operation of 180°, clockwise flip operation of 270°, transposition operation and image mirroring operation on the original data using Python language and OpenCV coding;

[0090] Data enhancement is a method of expanding data samples by randomly transforming the original image, which can make the model better adapt to the application scenario. Use Python language and OpenCV coding to perform horizontal flipping, vertical flipping, clockwise flipping 90°, clockwise flipping 180°, clockwise flipping 270°, transposition, and horizontal operations on the original data, and finally adjust the annotation file corresponding to the image accordingly. In the embodiment of the present invention, the pollen image dataset is split into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0091] Data augmentation is a commonly used technique in deep learning. It generates more training samples by transforming the original data, thereby expanding the data set without increasing the actual data collection cost. Python language and OpenCV library are used to implement a variety of data augmentation operations, such as horizontal flip, vertical flip, rotation (90°, 180°, 270°), transposition and mirroring. These operations can simulate various image transformations that may be encountered in actual scenes, so that the trained model has stronger generalization ability and can accurately identify pollen under different conditions.

[0092] Although the data augmentation operation itself requires certain computing resources, it can accelerate the model training process by increasing the number of training samples. In deep learning, the more training samples there are, the richer the features learned by the model and the faster the convergence speed. Therefore, the data augmentation operation can shorten the model training time and improve the research and development efficiency to a certain extent. The size information of the annotation box is very critical for the pollen recognition model. It not only provides the location information of the pollen in the image, but also gives the approximate size and shape of the pollen particles through the size information, which is of great significance for the model to learn the characteristics of pollen and distinguish different types of pollen. More accurate annotation information can improve the training effect of the model and make it have higher recognition accuracy in practical applications.

[0093] After obtaining the pollen sample image and annotating the pollen sample image with information by an image annotation tool to obtain image annotation information for each pollen sample image, the method further includes:

[0094] Determine a corresponding clustering data set according to the size information of the annotation boxes corresponding to all the pollen sample images, and determine a corresponding number of clusters according to the clustering data set;

[0095] Selecting K data objects from the clustering data set as initial clustering centers;

[0096] For each data object in the clustering data set, the distance between the data object and the initial cluster is calculated, and each data object is assigned to the cluster to which the cluster center closest to the data object belongs;

[0097] Repeat the above steps until the corresponding cluster center no longer changes, and output the final clustering result, which includes the cluster to which each data object belongs and the cluster center of each cluster;

[0098] A priori box is generated according to the clustering result to serve as an initial value of the model prediction box.

[0099] The size of pollen is an important recognition feature. In the target detection algorithm, the classification of image size is achieved through the size of the prior box. In this method, K-means clustering is used to achieve size clustering of image prior boxes to obtain the prior box required by the pollen target detection model.

[0100] The scheme of the embodiment of the present invention can determine the optimal prior frame size and proportion according to the actual size distribution of the pollen sample annotation frame through cluster analysis. These prior frames, as the initial values ​​of the model prediction frame, can more accurately match the actual size of the pollen sample, thereby improving the initial positioning accuracy of the target detection model when detecting pollen. The optimization of initialization can reduce the amount of adjustment during the model training process and accelerate the convergence speed. The prior frames generated by clustering can fit the actual size and shape of the pollen sample more closely, so that the model can generate more accurate bounding boxes when predicting. This helps to improve the accuracy and recall rate of detection, especially when the pollen sample size varies greatly or the morphology is diverse. Cluster analysis is performed based on the entire data set, so the generated prior frames can reflect the overall characteristics of the data set. This enables the model to make more accurate predictions based on these prior frames when processing new and unseen pollen samples, thereby enhancing the generalization ability of the model. Compared with using fixed-size or randomly generated prior boxes, the number of prior boxes obtained through clustering analysis is smaller and closer to reality, which can reduce the number of bounding boxes that the model needs to process during the prediction process, thereby reducing computational complexity and increasing detection speed. The clustering results not only provide optimized initialization parameters for the model, but can also be used for subsequent data analysis and processing. For example, pollen samples can be classified and counted based on the clustering results, or the size distribution characteristics of different types of pollen can be analyzed.

[0101] More preferably, after constructing the sample training set according to the pollen sample image and the image annotation information, the method further includes:

[0102] Get the pre-trained weight file based on the VOC2017 dataset;

[0103] Modifying the configuration of the pre-training weight file according to the sample training set; the configuration modification includes changing the number of categories, specifying the data set path and training parameters;

[0104] Due to the limited size of the pollen image dataset, as well as the training efficiency and accuracy of target detection and image classification, this method uses the feature extraction parameters obtained by training YOLOv7 using the voc2017 dataset as the migration object for transfer learning, which reduces the workload of training and prevents the randomness of the training parameters from being too high, making the model difficult to converge.

[0105] Using pre-trained weight files can significantly speed up the model training process on specific tasks (such as pollen recognition). Pre-trained weights are trained on large-scale datasets (such as VOC2017) and already contain rich visual features and knowledge. These features and knowledge are common to most visual tasks, so they can be used as a good starting point to initialize the model, reducing the time and computing resources required to train from scratch.

[0106] Since the pre-trained weights are trained on a large-scale dataset, they contain a wider range of visual features and patterns. These features are also useful for new tasks (such as pollen recognition) because many basic visual elements (such as edges, textures, shapes, etc.) are shared across tasks. By using pre-trained weights, the model can learn features related to pollen recognition faster, thereby improving the final performance.

[0107] Training a deep learning model from scratch with relatively little training data can easily lead to overfitting. Overfitting is a phenomenon where a model performs well on the training data but performs poorly on unseen data. Using pre-trained weights can help the model have a certain degree of generalization ability in the early stages of training, because the pre-trained weights have learned many common visual features on large-scale datasets. This helps reduce the model's excessive reliance on training data during training, thereby reducing the risk of overfitting.

[0108] By using pre-trained weight files, the model configuration process can be simplified. Users only need to make necessary modifications to the pre-trained weights (such as changing the number of categories, specifying the dataset path, and training parameters) according to specific tasks (such as pollen recognition), without having to define the entire network structure and initialize the weights from scratch. This can save a lot of time and effort, and reduce performance issues caused by configuration errors.

[0109] The pollen recognition model includes a convolution layer and a pooling layer, and a CA attention module is added between the convolution layer and the pooling layer, and the feature map is optimized through the CA attention module.

[0110] The CA attention mechanism is a new method that integrates channel and spatial attention mechanisms. It can effectively improve the recognition regression rate, separate pollen images from complex backgrounds, and reduce the influence of impurities in the image. This method uses the CA (Coordinate attention) attention mechanism to fuse the YOLOv7 model to achieve accurate positioning of pollen images and eliminate the interference of impurities in the image background. By optimizing the feature map, the CA attention module enables the model to more accurately locate the target area when identifying pollen and extract more refined features. This helps to improve the accuracy of recognition, especially when dealing with pollen species that are similar in morphology but have large differences in details.

[0111] The method of the embodiment of the present invention selects the most advanced deep learning model YOLOv7 (You Only Look Once version-7) currently used for target detection to train the automatic recognition model of pollen. The particle size of a single grain of airborne pollen is between 10μm and 200μm. For pollen with a smaller particle size, its image recognition is already a problem in the field of small target recognition. The classic YOLOv7 model has certain deficiencies in small target detection. Therefore, this method optimizes the model through cluster analysis, transfer learning, attention mechanism, and changing the activation function, and realizes the positioning and rough classification of single grain pollen with the feature fusion YOLOv7 model.

[0112] The specific model training method and process are as follows: Based on the Tensorflow2 framework, with a graphics processing unit (graphic processing unit, model NVDIAA40, video memory bandwidth 696GB / S, video memory 48×4GB), the optimized YOLOv7 model is first optimized and built on the Windows operating system by means of Mosaic data enhancement, K-means clustering analysis to adjust the prior frame and transfer learning. The input size is 640×640 pixels, the initial learning rate is set to 0.0001, and the learning rate is adjusted by the cosine annealing decay algorithm, that is, linear increase and cosine function decrease. The number of training iterations is 300 times, 50 rounds of training in the freezing stage, and 250 rounds of training in the thawing stage. The batch size in the freezing stage is set to 64, and the thawing batch size is set to 32.

[0113] More preferably, the step of performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain the second pollen images of each to-be-identified pollen includes:

[0114] The extracted first pollen images of each to-be-identified pollen are padded to 640*640 pixels using a black background to obtain a corresponding second pollen image.

[0115] More preferably, the step of inputting the second pollen image of each to-be-identified pollen and the acquisition time of the to-be-identified pollen image into a species classification model for identification to determine the species information of each to-be-identified pollen includes:

[0116] Determining month information corresponding to the pollen image to be identified according to the acquisition time of the pollen image to be identified;

[0117] Determine a species classification model corresponding to the phenological information according to the month information;

[0118] The second pollen image of each to-be-identified pollen is input into a species classification model corresponding to the phenological information for identification to determine the species information of each to-be-identified pollen;

[0119] Due to equipment limitations, the image size of the detection input is 640*640 pixels. In the actual images of pollen monitoring work, a single pollen grain occupies 5%-10% of the overall image area. The input pixel information is relatively small, and the pixel information of a single pollen grain is not enough to support accurate identification at the species level. In order to improve the accuracy and speed of detection, a classification network is used to further accurately identify single pollen grains. ResNet50 is an image classification algorithm based on CNN. It is a classic model in the field of image classification. It has high accuracy and efficient detection speed, so it is selected as the backbone of the pollen image classification model. The species classification model here mainly uses the ResNet50 model.

[0120] In the specific implementation, the flowering period of each plant is marked, and the time information is put into the model as a feature. The time series module is used to assist in the identification of pollen types based on the time information, thereby improving the accuracy of model detection. Due to the limited size of the single pollen image dataset and the training efficiency and accuracy of image classification, this method uses the feature extraction parameters obtained by training the coco2017 dataset as the migration object for transfer learning, which reduces the workload of training and prevents the randomness of the parameters obtained by training from being too high, making it difficult for the model to converge. Specifically, the types of single pollen images are divided into folders, and the folders are named the Latin name of the pollen species. The method for labeling pollen phenological feature information is to add the flowering month of the plant to the file name, and use the code to read the information as a feature for learning during training.

[0121] The specific model training method and process are as follows: Based on the Tensorflow2 framework, the optimized ResNet50 model is built on the Windows operating system with a graphics processing unit (graphic processing unit, model NVDIAA40, video memory bandwidth 696GB / S, video memory 48×4GB). The input size is 640×640 pixels, the initial learning rate is set to 0.0001, and the learning rate is adjusted by the cross entropy loss function and the cosine annealing decay algorithm, that is, linear increase and cosine function decrease. The number of training iterations is 300 times, 50 rounds of training in the freezing stage, and 250 rounds of training in the thawing stage. The batch size in the freezing stage is set to 64, and the thawing batch size is set to 32.

[0122] The solution of the embodiment of the present invention is based on previous observations, and there are more than 200 species of flowering plants within 100 meters around a single point. The increase in pollen species will greatly reduce the accuracy of detection. In order to improve the accuracy and efficiency of detection, the flowering period in months is marked for each plant, and it is put into the data set as a feature. The time series module is used to assist in distinguishing pollen species based on time information. In summary, this method uses the ResNet50 single pollen high-definition image classification model based on phenological information feature fusion to classify and identify single pollen images. Figure 5 As shown, it is shown that the pollen exposure film collected by field observation is full of impurities; the scheme implemented by the present invention can quickly distinguish pollen from impurities, as shown in 6, which is the corresponding identification result.

[0123] After outputting the pollen images of each pollen to be identified and the corresponding type information, the method further includes:

[0124] The priori frame size information is output according to the priori frame size corresponding to the pollen image to be identified as the pollen size information, wherein the priori frame size information includes the size of the priori frame and the area of ​​the priori frame.

[0125] The image is identified by the pollen classification model. With the help of the size of the prior frame corresponding to the image, the length, width and area of ​​the prior frame are output as the basis for distinguishing the size of the pollen.

[0126] More preferably, the intelligent identification method further includes:

[0127] receiving pollen collection method information input by a user, wherein the pollen collection method information is an air volume method or an air deposition method;

[0128] If the pollen collection method is the air volume method, the first pollen concentration information of the corresponding collection point is determined according to the first concentration calculation formula, and the first concentration calculation formula is: C1=n1×N1 / V, wherein n1 is the number of pollens identified on each picture, N1 is the number of pollen images taken in each collection period, V is the amount of gas collected in the corresponding period, and C1 is the first pollen concentration information;

[0129] If the pollen collection method is the air deposition method, the second pollen concentration information of the corresponding collection point is determined according to the second concentration calculation formula, and the second concentration calculation formula is: C2=n2×N2 / m×1000, wherein m is the area of ​​the glass slide collected in the corresponding period, n2 is the number of pollens identified on each picture, N2 is the number of pollen images taken in each collection period, and C2 is the second pollen concentration information;

[0130] The solution of the embodiment of the present invention can more accurately reflect the pollen concentration in the actual environment by distinguishing and applying pollen concentration calculation formulas suitable for different collection methods. The air volume method directly relates the amount of gas collected to the number of identified pollens, while the air sedimentation method considers the relationship between the slide area and the sedimented pollen, thereby avoiding calculation errors caused by different collection methods.

[0131] This method allows users to choose the pollen collection method according to actual needs. Whether it is a situation that requires continuous monitoring (such as using the air volume method) or monitoring suitable for a specific period of time or area (such as the air deposition method), accurate pollen concentration data can be obtained, enhancing the universal applicability of the monitoring system. Combined with intelligent image recognition technology, this method can automatically identify the amount of pollen from the captured pollen image, and based on the collection method information input by the user, automatically select and apply the corresponding concentration calculation formula, which greatly reduces manual intervention and improves the automation level and efficiency of monitoring work. The above method can accurately provide pollen concentration data, which is convenient for subsequent allergy warnings based on pollen concentration.

[0132] In the embodiment of the present invention, the display module is used to display the observation results of the pollen type and concentration at the observation site, and the identification results can be corrected to further improve the quality of the observation data. The functions are as follows: Site management: Real-time viewing of the equipment operation status and pollen monitoring data status of each observation site. Real-time pollen monitoring: Real-time viewing of the real-time pollen observation data of the selected site. Pollen log: View historical pollen observation data according to the selected site and date range. Data correction: According to the selected site and time period, the pollen recognition results of the pollen pictures taken by the pollen collector and detected by the pollen automatic recognition model are retrieved, and the recognition results are manually corrected. Auxiliary discrimination: According to the pollen type identified by the pollen image, the electron microscope image, light microscope image and pollen feature description of the corresponding type of pollen are retrieved from the pollen feature library. Specific pollen type feature information can also be retrieved according to demand. To help correctly judge whether the recognition result of the pollen automatic recognition model is right or wrong. Pollen Encyclopedia: Pollen Encyclopedia displays the plant morphological characteristics, ecological characteristics and flowering characteristics of the main air pollen types of the day in real time.

[0133]

[0134] Table 1

[0135] Table 1 above is a comparison table of pollen identification results obtained by different methods for meteorological exposure images collected by Wushan Meteorological Station in Guangzhou in March 2024; among them, the pollen identification model is the result of identification using only the YOLOv7 model, and the method of the present invention refers to the identification result by integrating the YOLOv7 model with the ResNet50 classification network; it can be found that the scheme of the present invention can identify the number of more types of pollen, and the identification accuracy is also higher.

[0136] The intelligent recognition method for pollen images in the embodiment of the present invention first extracts a single pollen high-definition image through a pollen high-definition image extraction model, which can avoid impurity images and extract high-definition feature information of pollen; further, the pollen classification model fused with phenological features is used to accurately classify pollen types. This method establishes a high-precision, multi-species pollen image intelligent recognition method based on phenological classification.

[0137] Embodiment 2

[0138] See also Figure 7 , Figure 7 FIG. 1 is a schematic diagram of the structure of the intelligent identification system for pollen images disclosed in the embodiment of the present invention. Figure 7 As shown, the intelligent recognition system for pollen images may include:

[0139] Acquisition module 21: used to acquire a pollen image to be identified, wherein the pollen image to be identified is a depth-of-field composite image formed based on multiple images with different focus points;

[0140] Feature extraction module 22: used for inputting the pollen image to be identified into the pollen identification model to perform pollen feature extraction to determine the position information of each pollen in the pollen image to be identified, and extracting the first pollen image of each pollen to be identified according to the position information of each pollen in the pollen image to be identified;

[0141] Image processing module 23: used for performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain the second pollen images of each to-be-identified pollen;

[0142] Species identification module 24: used to input the second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified into the species classification model for identification to determine the species information of each pollen to be identified, and output the pollen image of each pollen to be identified and the corresponding species information.

[0143] The intelligent recognition method for pollen images in the embodiment of the present invention first extracts a single pollen high-definition image through a pollen high-definition image extraction model, which can avoid impurity images and extract high-definition feature information of pollen; further, the pollen classification model fused with phenological features is used to accurately classify pollen types. This method establishes a high-precision, multi-species pollen image intelligent recognition method based on phenological classification.

[0144] Embodiment 3

[0145] See also Figure 8 , Figure 8 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may be a computer, a server, etc. Of course, in certain circumstances, it may also be a smart device such as a mobile phone, a tablet computer, a monitoring terminal, and an image acquisition device with processing functions. Figure 8 As shown, the electronic device may include:

[0146] A memory 510 storing executable program codes;

[0147] a processor 520 coupled to the memory 510;

[0148] The processor 520 calls the executable program code stored in the memory 510 to execute part or all of the steps in the intelligent identification method of pollen images in the first embodiment.

[0149] The embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in the intelligent identification method of pollen images in the first embodiment.

[0150] The embodiment of the present invention further discloses a computer program product, wherein when the computer program product is run on a computer, the computer is enabled to execute part or all of the steps in the intelligent identification method of pollen images in the first embodiment.

[0151] An embodiment of the present invention further discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in the intelligent identification method of pollen images in embodiment one.

[0152] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0154] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

[0156] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0157] A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0158] The above is a detailed introduction to the intelligent recognition method, system, electronic device and storage medium for pollen images disclosed in the embodiments of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for intelligent recognition of pollen images, characterized in that: include: The pollen image to be identified is obtained, wherein the pollen image to be identified is a depth-of-field composite image formed based on multiple images with different focus points; the pollen image to be identified is obtained by the following steps: A preset number of slide image information is obtained at the pollen slide by a stereo microscope; the focal distance between each slide image information is 3 μm; the pollen slide is made by the following steps: using a small piece of gelatin to stick an appropriate amount of pollen, fixing the gelatin on a slide, then heating the bottom of the slide to melt the gelatin, and quickly covering it with a cover slip; Performing image synthesis on a preset number of slide image information according to a focus image synthesis algorithm to obtain a corresponding pollen image to be identified; The pollen image to be identified is input into a pollen recognition model to perform pollen feature extraction to determine the position information of each pollen in the pollen image to be identified, and the first pollen image of each pollen to be identified is extracted according to the position information of each pollen in the pollen image to be identified; the pollen recognition model includes a convolution layer and a pooling layer, and a CA attention module is added between the convolution layer and the pooling layer, and the feature map is optimized by the CA attention module; the position information of the pollen includes the size and position information of the bounding box obtained after the pollen image to be identified is input into the pollen recognition model; performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain second pollen images of each to-be-identified pollen; The second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified are input into the species classification model integrated with phenological characteristics for identification to determine the type information of each pollen to be identified, and the pollen image of each pollen to be identified and the corresponding type information are output, wherein the time series module is used to assist in distinguishing the pollen type based on the time information.

2. The intelligent identification method of pollen images according to claim 1, characterized in that: The pollen recognition model is constructed based on the YOLOv7 model of feature fusion; The pollen recognition model is obtained by the following steps: Acquire a pollen sample image, and annotate the pollen sample image with information using an image annotation tool to obtain image annotation information for each pollen sample image; Constructing a sample training set according to the pollen sample image and the image annotation information; and dividing the sample training set to obtain a training set, a validation set, and a test set; Performing a preprocessing operation on the pollen sample image; The pollen sample image that has undergone the preprocessing operation is input into the pollen recognition model with completed parameter configuration for training and parameter configuration adjustment until the corresponding pollen recognition model meets the training requirements and the current optimal model parameters are saved.

3. The intelligent identification method of pollen images according to claim 2, characterized in that: The image annotation information includes size information of the annotation box; after obtaining the pollen sample image and annotating the pollen sample image with information through the image annotation tool to obtain the image annotation information for each pollen sample image, it also includes: Performing data enhancement operation on the pollen sample image to expand the data sample; the data enhancement operation includes performing one or more of horizontal flip operation, vertical flip operation, clockwise flip operation of 90°, clockwise flip operation of 180°, clockwise flip operation of 270°, transposition operation and image mirroring operation on the original data using Python language and OpenCV coding; Or, after obtaining the pollen sample image and annotating the pollen sample image with information by an image annotation tool to obtain image annotation information for each pollen sample image, the method further includes: Using a K-means algorithm to cluster the size information of the annotation boxes corresponding to all the pollen sample images; Generate a priori box based on the clustering results as the initial value of the model prediction box.

4. The intelligent identification method of pollen images according to claim 2, characterized in that: After constructing the sample training set according to the pollen sample image and the image annotation information, the method further includes: Get the pre-trained weight file based on the VOC2017 dataset; The pre-training weight file is configured and modified according to the sample training set; the configuration modification includes changing the number of categories, specifying the data set path and training parameters.

5. The intelligent identification method of pollen images according to claim 1, characterized in that: The step of performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain the second pollen images of each to-be-identified pollen includes: The extracted first pollen images of each to-be-identified pollen are padded to 640*640 pixels using a black background to obtain a corresponding second pollen image.

6. The intelligent identification method of pollen images according to claim 1, characterized in that: After outputting the pollen images of each pollen to be identified and the corresponding type information, the method further includes: The priori frame size information is output according to the priori frame size corresponding to the pollen image to be identified as the pollen size information, wherein the priori frame size information includes the size of the priori frame and the area of ​​the priori frame.

7. The intelligent identification method of pollen images according to claim 1, characterized in that: The intelligent identification method further includes: receiving pollen collection method information input by a user, wherein the pollen collection method information is an air volume method or an air deposition method; If the pollen collection method is the air volume method, the first pollen concentration information of the corresponding collection point is determined according to the first concentration calculation formula, and the first concentration calculation formula is: C1=n1×N1 / V, wherein n1 is the number of pollens identified on each picture, N1 is the number of pollen images taken in each collection period, V is the amount of gas collected in the corresponding period, and C1 is the first pollen concentration information; If the pollen collection method is the air deposition method, the second pollen concentration information of the corresponding collection point is determined according to the second concentration calculation formula. The second concentration calculation formula is: C2=n2×N2 / m×1000, wherein m is the slide area collected in the corresponding time period, n2 is the number of pollen identified on each picture, N2 is the number of pollen images taken in each collection period, and C2 is the second pollen concentration information.

8. An intelligent recognition system for pollen images, characterized in that: include: Acquisition module: used to acquire the pollen image to be identified, wherein the pollen image to be identified is a depth of field composite image formed based on multiple images with different focus points; the pollen image to be identified is acquired by the following steps: A preset number of slide image information is obtained at the pollen slide by a stereo microscope; the focal distance between each slide image information is 3 μm; the pollen slide is made by the following steps: using a small piece of gelatin to stick an appropriate amount of pollen, fixing the gelatin on a slide, then heating the bottom of the slide to melt the gelatin, and quickly covering it with a cover slip; Performing image synthesis on a preset number of slide image information according to a focus image synthesis algorithm to obtain a corresponding pollen image to be identified; Feature extraction module: used for inputting the pollen image to be identified into the pollen recognition model to perform pollen feature extraction to determine the position information of each pollen in the pollen image to be identified, and extracting the first pollen image of each pollen to be identified according to the position information of each pollen in the pollen image to be identified; the pollen recognition model includes a convolution layer and a pooling layer, and a CA attention module is added between the convolution layer and the pooling layer, and the feature map is optimized by the CA attention module; the position information of the pollen includes the size and position information of the bounding box obtained after the pollen image to be identified is input into the pollen recognition model; Image processing module: used for performing image processing on the extracted first pollen images of each to-be-identified pollen according to the image setting conditions to obtain the second pollen images of each to-be-identified pollen; Species identification module: used to input the second pollen image of each pollen to be identified and the acquisition time of the pollen image to be identified into the species classification model integrated with phenological characteristics for identification to determine the species information of each pollen to be identified, and output the pollen image of each pollen to be identified and the corresponding species information, wherein the time series module is used to assist in the identification of pollen species based on time information.

9. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the intelligent recognition method for pollen images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the intelligent identification method for pollen images according to any one of claims 1 to 7.

Images