A method for intelligently identifying allergenic plants based on YOLO network

Abstract

The present application relates to the technical field of plant intelligent identification, and particularly relates to a method for intelligently identifying pollen-allergic plants based on a YOLO network. The method comprises the following steps: collecting video data of pollen-allergic plants in the air in urban streets, and constructing an image data set of pollen-allergic plants in the air; using LabelImg software to label the categories of pollen-allergic plants in the air in the images in the image data set of pollen-allergic plants in the air, and automatically generating an.xml file; training and optimizing a YOLOv8 model, saving the best weight file after training, and obtaining a YOLO intelligent identification model for pollen-allergic plants in the air; inputting video data to be detected or pictures to be detected, detecting the video data, and outputting corresponding target detection results. The method of the present application is used for identifying pollen-allergic plants in urban streets, and provides a reference for urban community layout, selection of greening tree species, and community configuration, and has the characteristics of high identification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of plant intelligent identification, and particularly relates to a YOLO network-based intelligent identification method for pollen-allergic plants. BACKGROUND

[0002] Pollen-allergic plants refer to plants that can produce a large amount of pollen during their growth and the pollen can be transmitted through the air to cause allergic reactions in human bodies. According to the literature materials such as Chinese Airborne Allergenic Pollen Survey, Chinese Airborne Pollen and Plant Color Atlas and Chinese Flora, pollen-allergic plants usually have a high incidence in spring and summer, and their pollen is usually small and light, which can be blown to a very far place with wind, thus easily causing airborne allergic reactions such as allergic rhinitis and asthma. In the process of urban construction, unreasonable urban community layout, improper selection of greening tree species and community configuration aggravate the allergenicity of pollen of allergic plants. It is reported that about 30% of the urban population worldwide is troubled by allergenic pollen, and the allergenic pollen plants that have a significant impact on human health have reached more than 400 species. Therefore, detection and identification of airborne pollen-allergic plants are helpful to the protection of allergic population and the development of risk control of allergic plants in urban community management.

[0003] Traditional detection and identification methods of airborne pollen-allergic plants mainly rely on manual identification, which has a large workload and low efficiency, and the staff need to have rich botanical knowledge. In addition, the traditional method mainly relies on subjective judgment of people, which greatly affects the accuracy of identification and classification. With the continuous progress of science and technology, the application of deep learning target detection algorithm (RCNN, Fast RCNN, YOLO, etc.) in the field of computer can greatly improve the work efficiency. Although in recent years, target detection algorithms have been used in plant detection and identification, but in the existing research, the detection of flowers, leaves, fruits and diseases of plants is mainly used, and there are problems such as insufficient data set, complex background, poor shooting angle and low recognition accuracy in the overall detection of plants, especially the overall detection of pollen-allergic plants. SUMMARY

[0004] The purpose of the present application is to provide a YOLO network-based intelligent identification method for airborne pollen-allergic plants, which is used for identifying pollen-allergic plants on urban streets, providing a reference for urban community layout, selection of greening tree species and community configuration, and has the characteristics of high identification accuracy.

[0005] To achieve this purpose, the present application adopts the following technical solutions:

[0006] A YOLO network-based intelligent identification method for airborne pollen-allergic plants comprises the following steps:

[0007] S1, video data of airborne pollen allergenic plants in urban streets is collected, images in each video are intercepted by frame number to form a picture sequence, and an airborne pollen allergenic plant image dataset is constructed in this way;

[0008] S2, LabelImg software is used to label the categories of airborne pollen allergenic plants in images in the airborne pollen allergenic plant image dataset and automatically generate.xml files;

[0009] S3, the YOLO v8 model is trained and optimized using the image dataset in S2, the best weight file after training is saved, and a YOLO airborne pollen allergenic plant intelligent recognition model is obtained;

[0010] S4, input video data or pictures to be tested into the YOLO airborne pollen allergenic plant intelligent recognition model, detect the video data and output the corresponding target detection result.

[0011] Further, in step S1, video data of airborne pollen allergenic plants in urban streets is collected under different weather conditions in different seasons;

[0012] The images in each video are intercepted by frame number to form a picture sequence, and after removing the pictures in which airborne pollen allergenic plants do not appear, an airborne pollen allergenic plant image dataset is constructed.

[0013] Further, in step S2, LabelImg software is used to manually label the names of various airborne pollen allergenic plants in the pictures, and an.xml file conforming to the YOLO network labeling format is generated after the labeling is completed;

[0014] The.xml file includes the following information: file name, image size, label name, and coordinate information of each labeled target.

[0015] Further, in step S3, the backbone network of the YOLO v8 model is a Darknet53 structure, and the Darknet53 structure includes a Backbone module, a Neck module and a Head module connected to each other;

[0016] The Backbone module is used to extract and convert the input image into a multi-layer feature map, the Neck module is used to fuse the feature map output by the Backbone module, and the Head module is used to couple the feature map output by the Neck module.

[0017] The Backbone module and the Neck module each include a C2F unit for processing a feature map to output a one-dimensional feature vector, and the calculation formula is:

[0018] text(C2F)(X) = text(FC)(text(Conv)(X)) (1)

[0019] where X is an input feature map, text(Conv) represents a convolution operation, and text(FC) represents a full connection operation.

[0020] Further, in the step S3, the FCOS algorithm is used to predict the coupled feature map of the Head module, and the prediction calculation formula is:

[0021] x(i,j) = text(Conv)(x)(F(i,j)) + j (2)

[0022] y(i,j) = text(Conv)(y)(F(i,j)) + i (3)

[0023] w(i,j) = text(Conv)(w)(F(i,j)) (4)

[0024] h(i,j) = text(Conv)(h)(F(i,j)) (5)

[0025] where F(i,j) is a pixel feature vector at the (i,j) position of the feature map, text(Conv)(x) represents the x coordinate of the center point of the bounding box predicted by the convolution operation, text(Conv)(y) represents the y coordinate of the center point of the bounding box predicted by the convolution operation, text(Conv)(w) represents the width of the bounding box predicted by the convolution operation, and text(Conv)(h) represents the height of the bounding box predicted by the convolution operation.

[0026] The calculation formula of the pixel point prediction classification is as follows:

[0027] P(i,j) c = text(Conv)(F(i,j)) (6)

[0028] where P(i,j) c represents the probability of the pixel point belonging to each class, and text(Conv) represents a convolution operation.

[0029] The pixel point prediction classification calculation result and the prediction calculation result are used as the input of the loss function, and the loss function formula is as follows:

[0030] L = L (cls) + L (ctr) + L (box) + L (IoU) (7)

[0031] wherein L (cls) represents a classification loss, L (ctr) represents a center point coordinate loss, L (box) represents a width and height information loss, and L (IoU) represents an IoU Loss.

[0032] When the L value is greater than a set threshold value, the coupled feature map of the Head module is regarded as a positive sample block.

[0033] Further, in the step S3, the YOLOv8 model is trained and optimized by using the image data set in S2 in a manner of obtaining a positive sample block, and after training, the best weight file best.pt is saved and a YOLO airborne pollen allergenic plant intelligent recognition model and a training result are obtained.

[0034] Further, in the step S2, the images and corresponding.xml files in the airborne pollen allergenic plant image data set are divided into a training set, a validation set and a test set.

[0035] The training set is used to train the YOLO v8 model, the validation set is used to verify the YOLO v8 model after training is completed, so as to evaluate the training result of the YOLO v8 model, and the test set is used to test the YOLO v8 model, so as to judge the recognition accuracy of the YOLO v8 model.

[0036] Further, in the step S3, the evaluation indexes of the YOLO airborne pollen allergenic plant intelligent recognition model include precision (P), recall (R), mean average precision (mAP) and F1 score (F1 Score), and the calculation formula is:

[0037] Precision:

[0038] Recall:

[0039] Average precision of a single class:

[0040] Mean average precision:

[0041] F1 score:

[0042] Wherein, TP represents the number of samples that are correctly detected, FP represents the number of samples that are incorrectly detected, FN represents the number of samples that are not detected, r represents a value from 0-1, i represents the number of average precision values of a single class, 1, 2, 3...n; class represents a class, and N(class) represents the number of classes.

[0043] Further, in the step S4:

[0044] The to-be-tested video data is input into the YOLO air-borne pollen allergenic plant intelligent recognition model.

[0045] The YOLO air-borne pollen allergenic plant intelligent recognition model processes the to-be-tested video data and outputs a processed video; if a target air-borne pollen allergenic plant appears in a picture of the processed video, the target air-borne pollen allergenic plant is identified by a prediction frame, and a confidence score is displayed for each prediction frame.

[0046] Further, the calculation formula of the confidence score is as follows:

[0047] Confidence Score

[0048] = Objectness Score x Class Probability x IoU Score (13)

[0049]

[0050]

[0051]

[0052] Wherein, Confidence Score represents the confidence score; a represents the output of a class prediction branch; Objectness Score represents the probability that an object exists in a detection window; Class Probability represents the probability that a detected object belongs to a specific class; and IoU Score represents the degree of overlap between a prediction bounding box and a real bounding box.

[0053] The technical scheme provided by the application can include the following beneficial effects:

[0054] The present application collects video data of air-borne pollen allergenic plants against the background of urban streets to train and optimize a YOLO v8 model, and obtains a YOLO air-borne pollen allergenic plant intelligent recognition model, which can recognize air-borne pollen allergenic plants in to-be-tested video data, realizes recognition of air-borne pollen allergenic plants in urban streets, provides reference for urban community layout, greening tree species selection and community configuration, and has the characteristics of high recognition accuracy.

[0055] Moreover, the YOLO air-borne pollen allergenic plant intelligent recognition model based on the YOLO v8 as the main network model has high prediction accuracy without data enhancement, and can detect and recognize pictures and videos, and can balance detection speed and detection accuracy, and has good robustness in the recognition of target allergenic plants in the complex environment background of community streets. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a flow chart of the YOLO network-based air-borne pollen allergenic plant intelligent recognition method of an embodiment of the present application;

[0057] Figure 2 is a network structure schematic diagram of the YOLO v8 model;

[0058] Figure 3 is a network structure schematic diagram of the C2F module of the YOLO v8 model shown in Figure 2

[0059] Figure 4 is a network structure schematic diagram of the Bottleneck of the C2F module shown in Figure 3

[0060] Figure 5 is a network structure schematic diagram of the SPPF unit of the YOLO v8 model shown in Figure 2

[0061] Figure 6 is a network structure schematic diagram of the CBS module in the YOLO v8 model shown in Figure 2

[0062] Figure 7 is a model training evaluation index curve schematic diagram in an embodiment of the present application;

[0063] Figure 8 is a recognition effect diagram with a prediction frame in an embodiment of the present application. DETAILED DESCRIPTION

[0064] ​​​​Embodiments of the present application are described below in detail with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0065] In the description of the present application, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features, which are used to distinguish the described features, and have no order or importance.

[0066] In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0067] In the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0068] The embodiments of the present application are described below in combination with Figures 1 to 8 A YOLO network-based intelligent identification method for airborne pollen allergenic plants.

[0069] A YOLO network-based intelligent identification method for airborne pollen allergenic plants, comprising the following steps:

[0070] S1, collecting video data of airborne pollen allergenic plants in urban streets, and cutting images in each video by frame number to form a picture sequence, thereby constructing an airborne pollen allergenic plant image dataset;

[0071] S2, using LabelImg software to label the categories of airborne pollen allergenic plants in the images in the airborne pollen allergenic plant image dataset and automatically generate.xml files;

[0072] S3, training and optimizing the YOLO v8 model by using the image data set in S2, saving the best weight file after training, and obtaining a YOLO airborne pollen allergenic plant intelligent recognition model;

[0073] S4, inputting the to-be-detected video data or to-be-detected picture into the YOLO airborne pollen allergenic plant intelligent recognition model, detecting the video data, and outputting a corresponding target detection result.

[0074] In the field of target detection and recognition, most of the existing plant data sets (PlantCLEF2015, Ecust104, Oxford102, etc.) are mainly local images of plant organs, the image background is single, and the herbaceous plants are mostly used, and the airborne pollen allergenic plants of woody plants are lacking. Meanwhile, based on the defects of the airborne pollen allergenic plant detection and recognition in the prior art, the present application provides a YOLO network-based airborne pollen allergenic plant intelligent recognition method. The present application collects video data of airborne pollen allergenic plants with city streets as the background to train and optimize the YOLO v8 model, and obtains a YOLO airborne pollen allergenic plant intelligent recognition model. The YOLO airborne pollen allergenic plant intelligent recognition model can recognize airborne pollen allergenic plants in to-be-detected video data, realize the recognition of airborne pollen allergenic plants in city streets, provide reference for city community layout, greening tree species selection and community configuration, and has the characteristics of high recognition accuracy.

[0075] It is worth noting that in the present application, the video data of airborne pollen allergenic plants in city streets is used to train the YOLOv8 model. The background of the video data is a city street, and the background is single. Moreover, the video data has images of airborne pollen allergenic plants from multiple angles, which can effectively improve the recognition accuracy of the YOLO airborne pollen allergenic plant intelligent recognition model. Moreover, in step S1 of the present application, the images in each video are intercepted according to the frame number to form a picture sequence, and the airborne pollen allergenic plant image data set is constructed in this way. There are a large number of images in the image data set, which can fully train the YOLO v8 model to further obtain a YOLO airborne pollen allergenic plant intelligent recognition model with high recognition accuracy.

[0076] In addition, the YOLO airborne pollen allergenic plant intelligent recognition model detects the video or picture taken in the city street, and has low requirements for the shooting angle of the to-be-detected plant in the video or picture, thereby reducing the difficulty of collecting the to-be-detected video.

[0077] Further, in step S1, the video data of airborne pollen allergenic plants in city streets is collected under different weather conditions in different seasons;

[0078] The images in each video are intercepted by frame number to form a picture sequence, and after pictures in which the wind-borne pollen allergenic plants do not appear are removed, a wind-borne pollen allergenic plant image dataset is constructed.

[0079] In some embodiments of the present application, the YOLO v8 model is trained by collecting video data of wind-borne pollen allergenic plants in urban streets under different weather conditions in different seasons, and the obtained YOLO wind-borne pollen allergenic plant intelligent recognition model can recognize wind-borne pollen allergenic plants in different states, such as flowering state, non-flowering state, leaf falling state, and new leaf sprouting state, and still achieves high recognition accuracy for videos to be tested under different light conditions.

[0080] Specifically, in an embodiment of the present application, a motion camera is used to shoot wind-borne pollen allergenic plants along the street in Guangzhou. The two most widely distributed wind-borne pollen allergenic plants in Guangzhou are two woody plants, namely the genus Bauhinia and the genus Archontophoenix. The camera lens parameters are 4K30FPS, and other settings are default parameters. Considering the variability of light intensity and the morphology of the detection object, which may affect the detection speed and accuracy, video data is collected under different seasons and weather conditions. The video data is split into pictures by frame number to form a picture sequence through a Python program. Specifically, the video frame number is 30, there are 1800 frames of video per minute, and the video data format is MP4. Now the images are intercepted by intercepting a picture every 30 frames, the image format is JPG, and they are saved to the corresponding folder. Then the intercepted pictures are screened, and after the pictures in which the wind-borne pollen allergenic plants do not appear are removed, 1560 pictures are left to construct a wind-borne pollen allergenic plant image dataset.

[0081] Further, in step S2, the LabelImg software is used to manually annotate the names of the various wind-borne pollen allergenic plants in the pictures, and after the annotation is completed, an.xml file in the YOLO network annotation format is generated accordingly;

[0082] The.xml file includes the following information: file name, image size, annotation name, and coordinate information of each annotation target.

[0083] In some embodiments of the present application, the LabelImg software is used to process the image dataset of wind-pollinated allergenic plants, which has the characteristics of low difficulty and simple operation. Specifically, in an embodiment of the present application, the LabelImg software is used to manually annotate the pictures of Bauhinia plants and Roystonea plants. If the whole area of the target plant is labeled as Bauhinia (Bauhinia) and Roystonea (king palm), respectively. After labeling, an.xml file in the YOLO network labeling format is automatically generated, which includes the following fields: file name, image size, label name, and coordinate information of the four corner points of each labeled target.

[0084] Further, in step S2, the images and corresponding.xml files in the image dataset of wind-pollinated allergenic plants are divided into a training set, a validation set, and a test set.

[0085] The training set is used to train the YOLO v8 model, the validation set is used to validate the trained YOLO v8 model to evaluate the training result of the YOLO v8 model, and the test set is used to test the YOLO v8 model to determine the recognition accuracy of the YOLO v8 model.

[0086] Specifically, the number of images in each dataset is divided into training set: validation set: test set = 7:2:1. The role of the validation set is as follows: 1. Model selection and optimization: the validation set is used to evaluate the performance of different models or different hyperparameter settings, and the best model or hyperparameter is selected; by evaluating the performance of the model on the validation set, the performance of different models can be compared, and the optimal model is selected for further application; in the present application, the validation set is used to validate the trained YOLO v8 model to determine the selection of the YOLO v8 model.

[0087] 2. Prevent overfitting: by monitoring the performance of the model on the validation set, overfitting can be detected in time, and appropriate measures such as regularization, adjustment of model complexity, etc. can be taken to improve the generalization ability of the model;

[0088] 3. Hyperparameter optimization: hyperparameters are parameters that need to be manually set during model training, such as learning rate, regularization parameter, etc. By evaluating the performance of different hyperparameter settings on the validation set, the best hyperparameter configuration can be selected to improve the performance and robustness of the model;

[0089] 4. Monitor the model training process: during the model training process, the performance of the validation set can be used to monitor the training progress of the model; observing the accuracy or loss value on the validation set can help determine whether the model has problems such as underfitting or overfitting, and adjust the training strategy.

[0090] Further, in the step S3, the backbone network of the YOLO v8 model is a Darknet53 structure, the Darknet53 structure includes a Backbone module, a Neck module and a Head module connected with each other;

[0091] The Backbone module is used for feature extraction of the input image to convert into a multi-layer feature map, the Neck module is used for fusing the feature map output by the Backbone module, and the Head module is used for coupling the feature map output by the Neck module;

[0092] The Backbone module and the Neck module each include a C2F unit, the C2F unit is used for processing the feature map to output a one-dimensional feature vector, and the calculation formula is:

[0093] text(C2F)(X)=text(FC)(text(Conv)(X)) (1)

[0094] Wherein, X is the input feature map, text(Conv) represents convolution operation, and text(FC) represents full connection layer.

[0095] Specifically, the Darknet53 structure is used for feature extraction of the image of the airborne pollen allergenic plant, the main body is composed of Backbone, Neck and Head, and the YOLO v8 network structure is as shown in Figure 2 The C2F unit, i.e. CSPLayer_2Conv, adds more gradient flow information and further lightens the model, optimizes the network structure, improves the robustness of the model, and can reduce the requirement for hardware in actual application, and can be conveniently connected to the device for detecting and identifying the airborne pollen allergenic plant.

[0096] The C2F module includes a convolution layer and a full connection layer, and the input of the C2F unit is a feature map preprocessed by a convolutional neural network, and the output is a one-dimensional feature vector. In the C2F unit, the input feature map X is input into the convolution layer text(Conv) to perform convolution operation, and after obtaining an output feature map y, the output feature map y is input into the full connection layer text(FC) to perform a series of linear transformation and activation function processing, and finally a one-dimensional vector is obtained.

[0097] Specifically, CSPLayer_2Conv: this refers to a custom layer in the network architecture, usually called C2F unit. 1. Convolution layer 1: the C2F unit starts from a convolution layer. This layer performs a set of convolution operations on the input feature map. Convolution is a basic operation in deep learning that extracts spatial patterns by applying filters to the input data.

[0098] 2. Center Path: The center path is a branch of the C2F unit that focuses on capturing the central information of the input feature map. It typically consists of a series of convolutional layers, which can vary depending on the specific architecture. The purpose of the center path is to extract and preserve the central details and features of the input.

[0099] 3. Surround Path: The surround path is another branch of the C2F unit, designed to capture contextual information surrounding the central region. It also includes a set of convolutional layers, which can have different configurations compared to the center path. The surround path captures more extensive environmental and background information around the central region.

[0100] 4. Fusion: Combining or fusing the center path and surround path to integrate the extracted information. The fusion process can take different forms, such as element-wise addition, concatenation, or other fusion operations. The goal is to effectively merge the central and surrounding features, taking advantage of their complementarity.

[0101] 5. Convolutional Layer 2: The fused features are further processed through another convolutional layer. This additional layer refines the fused features, extracting more abstract representations that are better suited for subsequent tasks or network layers.

[0102] In summary, the C2F unit (CSPLayer_2Conv) is designed to capture both the central information and contextual information of the input feature map. It uses a fusion technique to combine these two types of features and then refines them with another convolutional layer. This unit helps the network learn more discriminative and contextual representations, improving the performance of various computer vision tasks.

[0103] It is worth mentioning that the Backbone module has an SPPF unit, which serves to enhance spatial information. The SPPF unit fuses multi-scale spatial information through a spatial pyramid pooling (SPP) and combines it with the original features, enabling the model to capture fine-grained details and contextual information, thereby improving target detection and recognition performance.

[0104] Further, in step S3, the coupled feature map of the Head module is predicted using the FCOS algorithm, and the prediction calculation formula is:

[0105] x(i, j) = text(Conv)(x)(F(i, j)) + j (2)

[0106] y(i, j) = text(Conv)(y)(F(i, j)) + i (3)

[0107] w(i, j) = text(Conv)(w)(F(i, j)) (4)

[0108] h(i, j) = text(Conv)(h)(F(i, j)) + j (5)

[0109] wherein, F(i,j) is the feature vector of the pixel point at the (i,j) position on the feature map, text(Conv)(x) represents the x coordinate of the center point of the bounding box predicted by the convolution operation, that is, the horizontal position of the bounding box in the input image; text(Conv)(y) represents the y coordinate of the center point of the bounding box predicted by the convolution operation, that is, the vertical position of the bounding box in the input image; text(Conv)(w) represents the width of the bounding box predicted by the convolution operation, that is, the horizontal span of the bounding box in the input image; text(Conv)(h) represents the height of the bounding box predicted by the convolution operation, that is, the vertical span of the bounding box in the input image.

[0110] The calculation formula of the pixel point prediction classification is as follows:

[0111] P(i, j) c = text(Conv)(F(i, j)) + j (6)

[0112] wherein, P(i,j) c represents the probability of the pixel point belonging to each category, and text(Conv) represents the convolution operation.

[0113] The pixel point prediction classification calculation result and the prediction calculation result are used as the input of the loss function, and the loss function formula is as follows:

[0114] L = L(cls) + L(ctr) + L(box) + L(IoU) (7)

[0115] wherein, L(cls) represents the classification loss, L(ctr) represents the center point coordinate loss, L(box) represents the width and height information loss, and L(IoU) represents the IoU Loss.

[0116] When the L value is greater than the set threshold value, the coupled feature map of the Head module is regarded as a positive sample block.

[0117] In some embodiments of the present application, the FCOS (Fully Convolutional One-Stage Object) algorithm is used to take each pixel point on the feature map as the center point to directly predict the center point coordinates and width-height information of the target, and to predict the category of the target for each pixel point without predefining anchor boxes. Specifically, in the prediction calculation formula, x(i,j) and y(i,j) are the offsets relative to the pixel point (i,j) on the feature map. FCOS obtains a plurality of positive sample blocks by calculating the feature map, and then uses these positive sample blocks for regression operation, which is performed in the prediction stage to obtain better performance improvement.

[0118] In the loss function of the FCOS algorithm, the classification loss is the cross-entropy loss of the classification branch, and the center point coordinate loss and the width-height information loss are the L1 distance loss between the predicted value and the true value. At the same time, the FCOS algorithm also introduces an IoU loss to constrain the IoU value between the predicted box and the true box.

[0119] Further, in the step S3, the YOLOv8 model is trained and optimized using the image data set in S2 in a manner of obtaining positive sample blocks, and after training, the best weight file best.pt is saved and the YOLO airborne pollen allergenic plant intelligent recognition model and the training result are obtained.

[0120] Further, in the step S3, the evaluation indexes of the YOLO airborne pollen allergenic plant intelligent recognition model include precision (P), recall (R), mean average precision (mAP), and F1 score (F1 Score), and the calculation formula is:

[0121] Precision:

[0122] Recall:

[0123] Average precision of a single class:

[0124] Mean average precision:

[0125] F1 score:

[0126] Wherein, TP represents the number of samples correctly detected, FP represents the number of samples incorrectly detected, FN represents the number of samples not detected, and r represents a value from 0 to 1, so as to derive the formula. The letter itself has no specific meaning; i represents the number of average precision values of a single class, 1, 2, 3, …, n; class represents a category, and N(class) represents the number of categories.

[0127] In a preferred scheme of the present application, the hardware environment of the training process: the CPU is Intel(R) Xeon(R), 20 cores, model Platinum 8255C, main frequency 2.5G Hz, memory 64G. The GPU acceleration training can speed up the model to identify the airborne pollen allergenic plants, the GPU model is NVIDIA GeForce RTX3090, and the video memory size is 24G. The software environment of the training process: the operating system is Windows 10, the Python version is 3.8.10, the deep learning framework is Pytorch1.11.0, and Cuda 11.3.

[0128] The optimized YOLO v8 network model is trained by using the airborne pollen allergenic plant image data set, and the network model training parameter settings are as follows: the input image size is 640*640*3, the multi-thread data loading is 8, the optimizer is selected as SGD, the initial learning rate is 0.01, the training batch size is modified from the default value 16 to 8, and other training parameters are default. It is proved by many experiments that the training model has converged in the first 300 rounds, so the number of model training rounds is 300, and the best weight file best.pt is saved after training to obtain the YOLO airborne pollen allergenic plant intelligent recognition model and the training result.

[0129] In the present application, the YOLO airborne pollen allergenic plant intelligent recognition model is evaluated by using precision (P), recall (R), mean average precision (mAP) and F1 score (F1 Score) as evaluation indexes. Specifically, TP represents a true positive, that is, the number of samples correctly detected, FP represents a false positive, that is, the number of samples incorrectly detected, and FN represents a false negative, that is, the number of samples not detected. AP represents the average precision of a single class, which is composed of the area surrounded by the P-R curve generated by the recall and the precision and the horizontal and vertical coordinate axes. The size of the enclosed area can be calculated by integration. Referring to Figure 6 In a preferred scheme of the present application, after the model is trained, the P, R, mAP and F1 score are 91.9%, 98.0%, 95.1% and 90.0% respectively, which indicates that the model has good detection accuracy for detecting and identifying airborne pollen allergenic plants of woody plants.

[0130] Further, in the step S4:

[0131] The video data to be tested is input into the YOLO airborne pollen allergenic plant intelligent recognition model.

[0132] The YOLO airborne pollen allergenic plant intelligent recognition model processes the to-be-tested video data and outputs processed video. If a target airborne pollen allergenic plant appears in the picture of the processed video, the target airborne pollen allergenic plant is identified by a prediction box, and the confidence is displayed for each prediction box.

[0133] In an embodiment of the present application, the prepared video data is input into the YOLO airborne pollen allergenic plant intelligent recognition model in a terminal window, and the input video data format is MP4. The YOLO airborne pollen allergenic plant intelligent recognition model decomposes the video data into a picture sequence of one frame of image, then performs reasoning and identification on the image to determine whether it is an airborne pollen allergenic plant, and simultaneously performs detection and classification. After the video is processed by the YOLO airborne pollen allergenic plant intelligent recognition model, if a target airborne pollen allergenic plant appears in the video picture, a prediction box appears and the confidence is displayed. The confidence is calculated for each boundary box predicted by the model, which is used to rank the detection and determine which detection to keep and which detection to discard.

[0134] Further, the calculation formula of the confidence is as follows:

[0135] Confidence Score

[0136] =Objectness Score×Class Probability×IoU Score (13)

[0137]

[0138]

[0139]

[0140] wherein, Confidence Score represents the confidence; a represents the output of the class prediction branch; Objectness Score represents the probability that the object exists in the detection window; Class Probability represents the probability that the detected object belongs to a specific class; and IoU Score represents the degree of overlap between the predicted boundary box and the real boundary box.

[0141] In the calculation formula of the confidence, the Objectness Score represents the probability of the object existing in the detection window, which is calculated by using a logistic regression, and the value is between 0 and 1. The Class Probability represents the probability of the detected object belonging to a specific class, and the probability of the specific class is calculated by using a logistic regression, and the value is between 0 and 1. The IoU Score represents the degree of overlap between the predicted bounding box and the real bounding box, and is calculated by using the standard union intersection (IoU) index. The IoU Score is a value between 0 and 1, wherein the value is 1, indicating perfect overlap between the predicted bounding box and the real bounding box. The video detection classification effect and the prediction box diagram after the YOLO air-borne pollen allergenic plant intelligent recognition model processing are shown in Figure 8 Figure 8 Referring to Figure 8 , the YOLO air-borne pollen allergenic plant intelligent recognition model can identify all the air-borne pollen allergenic plants in the video to be detected, and the user can not only know the types of the air-borne pollen allergenic plants, but also know the community configuration of the air-borne pollen allergenic plants in the urban street within the video range according to the recognition result.

[0142] Other configurations and operations of the YOLO network-based air-borne pollen allergenic plant intelligent recognition method according to the embodiments of the present application are known to those skilled in the art, and will not be described in detail here.

[0143] In the description of the present specification, the description referring to the terms "embodiment", "example" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0144] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for intelligent identification of airborne pollen-sensitized plants based on YOLO networks, characterized in that, Includes the following steps: S1. Collect video data of airborne pollen-sensitizing plants in urban streets, extract images from each video according to frame number to form an image sequence, and construct an image dataset of airborne pollen-sensitizing plants. S2. Use LabelImg software to label the categories of airborne pollen-sensitized plants in the images of the airborne pollen-sensitized plant image dataset and automatically generate .xml files; S3. Use the image dataset in S2 to train and optimize the YOLO v8 model, save the best weight file after training, and obtain the YOLO intelligent recognition model for airborne pollen-sensitized plants. S4. Input the video data or image to be tested into the YOLO intelligent identification model for airborne pollen-sensitized plants, detect the video data, and output the corresponding target detection results. The backbone network of the YOLO v8 model is a Darknet53 structure, which includes interconnected Backbone, Neck and Head modules. The Backbone module is used to extract features from the input image and convert them into multi-layer feature maps. The Neck module is used to fuse the feature maps output by the Backbone module. The Head module is used to couple the feature map output of the Neck module. Both the Backbone and Neck modules include C2F units, which process the feature map and output a one-dimensional feature vector. The calculation formula is as follows: Where X is the input feature map, text(Conv) represents the convolution operation, and text(FC) represents the fully connected operation; In step S3, the coupled feature map of the Head module is predicted using the FCOS algorithm, and the prediction calculation formula is as follows: Where F(i,j) is the feature vector of the pixel at position (i,j) on the feature map, text(Conv)(x) represents the x-coordinate of the center point of the bounding box predicted by the convolution operation; text(Conv)(y) represents the y-coordinate of the center point of the bounding box predicted by the convolution operation; text(Conv)(w) represents the width of the bounding box predicted by the convolution operation; and text(Conv)(h) represents the height of the bounding box predicted by the convolution operation. The formula for calculating pixel prediction classification is as follows: Where, P(i,j) c This represents the probability that a pixel belongs to each category, and text(Conv) represents the convolution operation.

2. The method according to claim 1, characterized in that, In step S1, video data of airborne pollen-sensitizing plants in urban streets are collected under various weather conditions in different seasons. Images from each video are extracted by frame number to form an image sequence. After removing images that do not contain airborne pollen-sensitizing plants, an image dataset of airborne pollen-sensitizing plants is constructed.

3. The method according to claim 1, characterized in that, In step S2, LabelImg software is used to manually label the names of various airborne pollen-sensitizing plants in the image, and after labeling, a .xml file conforming to the YOLO network labeling format is generated accordingly. The .xml file includes the following information: file name, image size, annotation name, and coordinate information for each annotation target.

4. The method according to claim 1, characterized in that, In step S3, the coupled feature map of the Head module is predicted using the FCOS algorithm, and the prediction calculation formula is as follows: The pixel-level prediction classification results and the prediction calculation results are used as inputs to the loss function, and the loss function formula is as follows: Where L(cls) represents the classification loss, L(ctr) represents the center point coordinate loss, L(box) represents the width and height information loss, and L(IoU) represents the IoU Loss; When the L value is greater than the set threshold, the coupled feature map of the Head module is regarded as a positive sample block.

5. The method according to claim 1, characterized in that, In step S3, the YOLO v8 model is trained and optimized using the image dataset in S2 in the manner of obtaining positive sample blocks. After training, the best weight file best.pt is saved and the YOLO smart identification model for airborne pollen-sensitized plants and the training results are obtained.

6. The method according to claim 1, characterized in that, In step S2, the images and corresponding .xml files in the airborne pollen-sensitized plant image dataset are divided into a training set, a validation set, and a test set. The training set is used to train the YOLO v8 model, and the validation set is used to validate the YOLO v8 model after training in order to evaluate the training results of the YOLO v8 model. The test set is used to test the YOLOv8 model in order to determine the recognition accuracy of the YOLOv8 model.

7. The method according to claim 1, characterized in that, In step S3, the evaluation metrics of the YOLO airborne pollen-sensitized plant intelligent identification model include precision (P), recall (R), mean precision (mAP), and F1 score (F1Score), calculated as follows: Accuracy: Recall rate: Average precision for a single category: Mean, average precision: F1 score: Where TP represents the number of correctly detected samples, FP represents the number of incorrectly detected samples, FN represents the number of undetected samples, r represents a value from 0 to 1, i represents the number of average precision values ​​for a single class, 1, 2, 3...n; class represents the class, and N(class) represents the number of classes.

8. The method according to claim 1, characterized in that, In step S4: Input the video data to be tested into the YOLO intelligent identification model for airborne pollen-sensitized plants. The YOLO airborne pollen allergen intelligent identification model processes the test video data and outputs the processed video. If the target airborne pollen allergen appears in the processed video, the target airborne pollen allergen will be marked with a prediction box, and the confidence level will be displayed for each prediction box.

9. The method according to claim 8, characterized in that, The confidence level is calculated using the following formula: Wherein, Confidence Score represents the confidence level; 'a' represents the output of the class prediction branch; Objectness Score represents the probability that an object exists in the detection window; Class Probability represents the probability that the detected object belongs to a specific class; and IoU Score represents the degree of overlap between the predicted bounding box and the ground truth bounding box.

Images