A real-time multifunctional intelligent identification and early warning system for pests and diseases

Abstract

The present invention relates to the field of smart agricultural technology, and specifically to a real-time multifunctional intelligent pest and disease identification and early warning system, which includes a crop disease identification module, a crop disease early warning module, a crop pest image detection module, a crop pest image batch detection module, and a crop pest video detection module. The present invention utilizes a trained disease identification network to realize the identification and early warning of crop diseases, and utilizes a trained pest detection network to realize the identification and real-time detection of crop pests. A GUI is developed based on PyQt5, and the user only needs to perform a simple mouse click on the interface to turn on and off identification and detection and obtain identification and detection results in real time. The disease identification network and the pest detection network can be obtained by training a RepVGG convolutional neural network and an improved YOLOV5 network, respectively. The present invention improves the accuracy and speed of crop disease and pest identification and detection.

Description

Technical Field

[0001] The present invention relates to the field of smart agricultural technology, and in particular to a real-time multifunctional intelligent pest and disease identification and early warning system. Background Art

[0002] Crop pest and disease detection and prevention are mainly divided into pest detection and identification and disease detection and identification. Considering that today's agricultural producers not only need to solve the diseases that have already occurred on crop leaves, but also need early identification and prevention, so as to eradicate crop pests and diseases and reduce the impact of pests and diseases on agricultural production.

[0003] Over the past few decades, many researchers have attempted to use deep learning techniques for crop disease identification. Xu Jinghui used deep learning to identify corn leaf blight and rust. Shi Xinxin used a dense convolutional network based on transfer learning, applied digital image processing techniques, and the Softmax regression algorithm to automatically extract image features and accurately identify diseases. Fan Jie proposed a single-species crop disease identification method and a fine-grained multi-species crop disease identification method based on a deep residual neural network. Zhang Yi and others used the ResNet-18 architecture and the SGD optimization algorithm to develop a deep learning model for distinguishing eight varieties of green tea. Zhang Mengyu used ResNet and the attention mechanism to identify flowers.

[0004] However, these methods are lacking in recognition accuracy, speed and number of parameters, so it is necessary to adopt a faster, simpler and more accurate recognition method.

[0005] Identifying crop pests is a new hot topic today. Lu Liujiang and others used the AdaBoost classifier to identify crop pests, Zeng Meng used YOLO to identify rapeseed pest images, and Ma Jiajia used the SVM method to study insect identification. These methods need to be improved in target detection speed and prediction accuracy. Summary of the Invention

[0006] The Summary of the Present Invention is intended to briefly introduce concepts that will be described in detail in the Detailed Description of the Present Invention. The Summary of the Present Invention is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] In order to solve the technical problems of low accuracy and speed in crop disease and insect pest identification and detection, the present invention proposes a real-time multifunctional disease and insect pest intelligent identification and early warning system.

[0008] The present invention provides a real-time multifunctional intelligent pest identification and early warning system, which includes a crop disease identification module, a crop disease early warning module, a crop pest image detection module, a crop pest image batch detection module and a crop pest video detection module;

[0009] The crop disease recognition module is used to obtain a disease image, perform disease recognition on the disease image according to the trained disease recognition network, and obtain the disease category corresponding to the disease image;

[0010] The crop disease early warning module is used to obtain a disease image folder and a disease threshold, perform disease recognition on each disease image in the disease image folder according to the disease recognition network, obtain the disease category corresponding to the disease image, determine the total disease rate according to the disease category corresponding to the disease images in the disease image folder, generate disease information corresponding to the disease image folder according to the total disease rate and the disease threshold, and save the disease information to a preset text file;

[0011] The crop pest image detection module is used to obtain pest images, detect and identify the pest images based on the trained pest detection network, and obtain and display pest information corresponding to the pest images, including: pest category, pest location, and the number of pests in each pest category;

[0012] The crop pest image batch detection module is used to obtain image directory information, detect and identify images corresponding to the image directory information according to the pest detection network, obtain pest information corresponding to the images corresponding to the image directory information, save the images corresponding to the image directory information and the pest information corresponding to the images corresponding to the image directory information to a pre-set folder, and display the save path;

[0013] The crop pest video detection module is used to obtain pest videos, detect and identify images constituting the pest videos according to the pest detection network, obtain pest information corresponding to the images constituting the pest videos, and display the pest information in real time.

[0014] Furthermore, the training process of the disease recognition network includes:

[0015] Build a disease identification network;

[0016] Acquire a crop disease image set, wherein the training labels corresponding to the crop disease images in the crop disease image set include: disease categories;

[0017] The disease recognition network is trained using the crop disease image set and the training labels corresponding to the respective crop disease images in the crop disease image set to obtain a trained disease recognition network.

[0018] Furthermore, the training process of the pest detection network includes:

[0019] Building a pest detection network;

[0020] Acquire a crop pest image set, wherein the training labels corresponding to the crop pest images in the crop pest image set include: pest category and pest location;

[0021] The pest detection network is trained using the crop pest image set and the training labels corresponding to the respective crop pest images in the crop pest image set to obtain a trained pest detection network.

[0022] Furthermore, the disease recognition network is a RepVGG convolutional neural network, and the loss function in the disease recognition network training process is a Loss Function.

[0023] Furthermore, the pest detection network is an improved network of the YOLOV5 network. An attention mechanism is added to the SPPF layer in the Backbone module of the pest detection network. The 3x3 convolutional layer in the Backbone module of the pest detection network is replaced by the RepVGGBlock layer in the RepVGG network. During the training process, the pest detection network adopts the Efficient-IOU loss function, which is better than the Complete-IOU loss function.

[0024] The present invention has the following beneficial effects:

[0025] The present invention provides a real-time, multifunctional, intelligent pest and disease identification and early warning system that can automatically, efficiently, and accurately identify, detect, and warn of crop pests and diseases, has good scalability and robustness, and features a user-friendly interface for easy user use. The real-time, multifunctional, intelligent pest and disease identification and early warning system can include a crop disease identification module, a crop disease early warning module, a crop pest image detection module, a crop pest image batch detection module, and a crop pest video detection module. First, the crop disease identification module is used to acquire disease images, perform disease identification on the disease images based on a trained disease recognition network, and obtain the disease category and cause of the disease corresponding to the disease image. Next, the crop disease early warning module is configured to obtain a disease image folder and a disease threshold. Using the disease recognition network, it performs disease recognition on each disease image in the disease image folder to obtain the disease category corresponding to the disease image. Based on the disease category and disease cause corresponding to the disease images in the disease image folder, it determines the total disease rate. Based on the total disease rate and the disease threshold, it generates disease information corresponding to the disease image folder and saves the disease information to a pre-set text file. The disease recognition network can be a RepVGG convolutional neural network, which can improve the accuracy and speed of disease recognition and, consequently, the accuracy and speed of disease information generation. Next, the crop pest image detection module is configured to obtain pest images and, using the trained pest detection network, detect and identify the pest images. It then obtains and displays the pest information corresponding to the pest images. The pest information includes: pest category, pest location, and the number of pests within the pest category. Next, the crop pest image batch detection module is used to obtain image directory information. Using the pest detection network, it detects and identifies the images corresponding to the image directory information, obtaining pest information corresponding to the images in the directory information. The images and pest information corresponding to the images in the directory information are saved to a pre-set folder, and the save path is displayed. Finally, the crop pest video detection module is used to obtain pest videos. Using the pest detection network, it detects and identifies the images comprising the pest video, obtaining pest information corresponding to the images in the video, and displays the pest information in real time. Since the pest detection network can be a modified YOLOV5 network, the SPPF layer in the Backbone module of the pest detection network can be enhanced with a CoordAtt (Coordinate Attention) mechanism. The 3×3Conv layer in the Backbone module of the pest detection network can be replaced with the RepVGGBlock layer from the RepVGG network.During the training process, the pest detection network can employ the Efficient-IOU (EIOU) loss function, which is more effective than the Complete-IOU (CIOU) loss function. Therefore, using the pest detection network to detect and identify pest images can improve the speed and accuracy of detection and identification. Therefore, the present invention can automatically, efficiently, and accurately identify, detect, and provide early warnings for crop pests and diseases, exhibits excellent scalability and robustness, and provides a user-friendly interface for ease of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 Schematic diagram of the structure of some embodiments of a real-time multifunctional intelligent disease and insect pest identification and early warning system according to the present invention;

[0028] Figure 2 Flowchart of some embodiments of the steps implemented by a real-time multifunctional intelligent disease and insect pest identification and early warning system according to the present invention;

[0029] Figure 3 This is a schematic diagram of the display interface of the crop disease identification module according to the present invention;

[0030] Figure 4 This is a schematic diagram of the display interface of the crop disease early warning module according to the present invention;

[0031] Figure 5 This is a schematic diagram of the display interface of the crop pest image detection module according to the present invention;

[0032] Figure 6 This is a schematic diagram of the display interface of the crop pest image batch detection module according to the present invention;

[0033] Figure 7 Schematic diagram of the display interface of the crop pest video detection module according to the present invention. DETAILED DESCRIPTION

[0034] To further illustrate the technical means and effects employed by the present invention to achieve its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementations, structures, features, and effects of the technical solutions proposed by the present invention. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0035] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0036] This embodiment provides a real-time multifunctional intelligent pest and disease identification and early warning system, which includes a crop disease identification module, a crop disease early warning module, a crop pest image detection module, a crop pest image batch detection module, and a crop pest video detection module;

[0037] The crop disease recognition module is used to obtain disease images, perform disease recognition on the disease images according to the trained disease recognition network, and obtain the disease category corresponding to the disease images;

[0038] The crop disease early warning module is used to obtain a disease image folder and a disease threshold, perform disease recognition on each disease image in the disease image folder according to the disease recognition network, obtain the disease category corresponding to the disease image, determine the total disease rate based on the disease category corresponding to the disease images in the disease image folder, generate disease information corresponding to the disease image folder based on the total disease rate and the disease threshold, and save the disease information to a pre-set text file;

[0039] The crop pest image detection module is used to obtain pest images, detect and identify the pest images based on the trained pest detection network, and obtain and display the pest information corresponding to the pest images. The pest information includes: pest category, pest location, and the number of pests in each pest category;

[0040] The crop pest image batch detection module is used to obtain image directory information, detect and identify images corresponding to the image directory information based on the pest detection network, obtain pest information corresponding to the images corresponding to the image directory information, save the images corresponding to the image directory information and the pest information corresponding to the images corresponding to the image directory information to a pre-set folder, and display the save path;

[0041] The crop pest video detection module is used to obtain pest videos, detect and identify the images that constitute the pest videos according to the pest detection network, obtain pest information corresponding to the images that constitute the pest videos, and display the pest information in real time.

[0042] refer to Figure 1 , which shows a schematic diagram of some embodiments of a real-time, multifunctional, intelligent pest identification and warning system according to the present invention. The real-time, multifunctional, intelligent pest identification and warning system includes a crop disease identification module 101, a crop disease warning module 102, a crop pest image detection module 103, a crop pest image batch detection module 104, and a crop pest video detection module 105.

[0043] refer to Figure 2 , which shows a flowchart of some embodiments of the steps implemented by a real-time multifunctional pest and disease intelligent identification and warning system according to the present invention. The real-time multifunctional pest and disease intelligent identification and warning system implements the following steps:

[0044] In step 201 , the crop disease recognition module is used to obtain a disease image, perform disease recognition on the disease image based on the trained disease recognition network, and obtain a disease category corresponding to the disease image.

[0045] In some embodiments, the crop disease recognition module may be used to obtain disease images, perform disease recognition on the disease images according to a trained disease recognition network, and obtain disease categories corresponding to the disease images.

[0046] The disease image may be an image of a crop to be detected for a disease. The disease recognition network may perform disease recognition on the disease image. The disease category may be a category of a crop disease. For example, the disease category may be apple rust. If the crop to be detected for a disease is not diseased, the disease category corresponding to the disease image may be that the crop to be detected for a disease is healthy.

[0047] As an example, first, the disease image can be input into the above-mentioned disease recognition network, and the disease category corresponding to the above-mentioned disease image can be obtained through the above-mentioned disease recognition network. Then, based on the disease category corresponding to the above-mentioned disease image, the disease cause corresponding to the disease category can be obtained, and the disease cause corresponding to the disease category is also the disease cause corresponding to the above-mentioned disease image. The disease cause can be an introduction to the disease category or the reason why the disease category occurs. For example, the disease cause corresponding to the disease category can be obtained in advance, so that if the disease category is known, the disease cause corresponding to the disease category can be obtained. Figure 3As shown, on the display interface 301 of the crop disease identification module, the user can click the Upload Image button 303 to upload a disease image 302. Then, click the Start Identification button 304. The disease identification network then performs disease identification on the disease image 302, obtaining the corresponding disease category 305 and disease cause 306. The user can click the Copy Result button 307 to copy the disease category 305 and disease cause 306. These users may be users of the real-time, multifunctional intelligent pest and disease identification and early warning system of the present invention. The display interface of the crop disease identification module may be a GUI (Graphical User Interface) developed using PyQt5.

[0048] Optionally, the training process of the disease recognition network may include the following steps:

[0049] The first step is to build a disease identification network.

[0050] The disease recognition network may be a RepVGG convolutional neural network, and the loss function during the disease recognition network training process may be a Loss Function.

[0051] The construction of the disease identification network can be achieved by existing methods and will not be described in detail here.

[0052] The second step is to obtain a collection of crop disease images.

[0053] The crop disease images in the crop disease image set may be images of crops with different disease conditions. The training labels corresponding to the crop disease images in the crop disease image set may include: disease categories.

[0054] For example, the disease categories corresponding to the crop disease images in the aforementioned crop disease image set may be known. The disease categories corresponding to the crop disease images may be the categories of the diseases occurring in the crops captured by the crop disease images. The disease categories corresponding to the crop disease images may be used as training labels for the crop disease images.

[0055] In the third step, the disease recognition network is trained using the crop disease image set and the training labels corresponding to the respective crop disease images in the crop disease image set to obtain a trained disease recognition network.

[0056] When the loss function value of the disease recognition network converges to the minimum, the disease recognition network training is completed.

[0057] In step 202, the crop disease early warning module is used to obtain a disease image folder and a disease threshold, perform disease identification on each disease image in the disease image folder according to the disease identification network, obtain the disease category corresponding to the disease image, determine the total disease rate based on the disease category corresponding to the disease images in the disease image folder, generate disease information corresponding to the disease image folder based on the total disease rate and the disease threshold, and save the disease information to a pre-set text file.

[0058] In some embodiments, the above-mentioned crop disease early warning module can be used to obtain a disease image folder and a disease threshold, perform disease identification on each disease image in the above-mentioned disease image folder according to the above-mentioned disease identification network, obtain the disease category corresponding to the above-mentioned disease image, determine the total disease rate according to the disease category corresponding to the disease image in the above-mentioned disease image folder, generate the disease information corresponding to the above-mentioned disease image folder according to the above-mentioned total disease rate and the above-mentioned disease threshold, and save the above-mentioned disease information to a pre-set text file.

[0059] The disease images in the disease image folder may be images of crops with diseases to be detected at different locations within a target area. The disease threshold may be a pre-set value. The target area may be an area where the crops with the diseases to be detected are grown. The disease threshold may be the maximum value at which the crops are disease-free. The total disease rate may be the total probability of multiple diseases occurring in the crops. The disease information corresponding to the disease image folder may indicate whether the crops with diseases to be detected within the target area are diseased. For example, the number of disease images in the disease image folder may be 6. The disease categories corresponding to the first and second disease images may be apple powdery mildew. The disease category corresponding to the third disease image may be apple mosaic. The disease categories corresponding to the fourth, fifth, and sixth disease images may be healthy. The disease rate for apple powdery mildew may be 2 / 6. The disease rate for apple mosaic may be 1 / 6. The total disease rate may be 3 / 6.

[0060] For example, when the total disease rate is greater than the disease threshold, the disease information may indicate that the crops to be detected for diseases in the target area have been diseased. When the total disease rate is less than or equal to the disease threshold, the disease information may indicate that the crops to be detected for diseases in the target area have not been diseased.

[0061] As another example, the disease images in the disease image folder can be input into the above-mentioned disease recognition network to obtain the disease categories corresponding to the disease images in the above-mentioned disease image folder, and generate the number of disease images in the above-mentioned disease image folder, disease categories, the number of disease images corresponding to the disease categories, disease rates, whether the disease occurs, and the save path. The save path can be the path of a text file that stores the above-mentioned disease information. Figure 4As shown, on the display interface 401 of the crop disease early warning module, the user can enter the disease threshold at the setting warning value 402, can click the select folder button 403 to upload the disease image folder, click the start detection button 404, and use the disease recognition network to perform disease recognition on the disease images in the uploaded disease image folder to obtain the disease category corresponding to the disease image, and can generate the detection result 406. The detection result 406 may include: the number of disease images in the disease image folder is 209; the disease categories may be apple powdery mildew, apple leaf spot, apple brown spot, apple mosaic, apple gray spot, apple health and apple rust; the number of disease images corresponding to the disease categories may be 3, 36, 0, 12, 47, 95 and 16 respectively; the disease rates corresponding to the disease categories may be 1.44%, 17.22%, 0.00%, 5.74%, 22.49%, 45.45% and 7.66% respectively; whether a disease has occurred: "The total disease rate is 0.55, which is greater than the preset value 0.4. A disease may have occurred. Please deal with it in time"; the save path may be: "C: / Users / DELL / Desktop / app / 1 / result.txt".

[0062] Since 1.44%+17.22%+0.00%+5.74%+22.49%+7.66%=54.55%, 54.55% can be rounded to two decimal places, so 54.55% can be approximated to 0.55, and therefore the total disease rate can be 0.55.

[0063] The display interface of the crop disease early warning module can be a GUI developed using PyQt5.

[0064] In step 203, the crop pest image detection module is used to obtain pest images, detect and identify the pest images based on the trained pest detection network, and obtain and display pest information corresponding to the pest images. The pest information includes: pest category, pest location, and the number of pests in each pest category.

[0065] In some embodiments, the above-mentioned crop pest image detection module can be used to obtain pest images, detect and identify the above-mentioned pest images according to the trained pest detection network, and obtain and display the pest information corresponding to the above-mentioned pest images. The pest information includes: pest category, pest location and the number of pests in the pest category.

[0066] The pest image may be an image of a crop to be detected for pests. The pest detection network may detect pest conditions in the pest image. The pest category may be a pest category. For example, the pest category may include, but is not limited to, the following: pine bark beetle, red pine bark beetle, pine bark beetle, Chinese pine bark beetle, coleopteran borer, and pine twig borer. The pest location may be the location of the pest in the pest image. The number of pests in a pest category may be the number of pests belonging to that pest category in the pest image.

[0067] As an example, Figure 5 As shown, on the display interface 501 of the crop pest image detection module, the user can upload a pest image 503 by clicking the Upload Image button 504. Without clicking the Batch Identification button 505, the user can click the Start Detection button 506 to detect and identify the pest image 503 through the pest detection network, obtaining a detection result 508. The detection result 508 can be copied by clicking the Copy Result button 507. Among them, Select Identification Category 502 indicates the pest category that the pest detection network can detect and identify. At Select Identification Category 502, the user can select the pest category to be identified for the pest image 503. The pest category checked at Select Identification Category 502 may be the pest category selected by the user. Statistics 509 may count the number of pests in each pest category selected by the user for identification on the pest image 503. For example, the pest categories selected by the user for identification may be armandi (pine beetle), acuminatus (red pine beetle), boerner (six-toothed pine beetle), coleoptera (Arshan pine beetle), linnaeus (coleoptera borer), and lecoute (pine twig borer). In statistical result 509, armandi: 1, acuminatus: 1, boerner: 1, coleoptera: 1, linnaeus: 1, and lecoute: 2 may respectively indicate that, in pest image 503, the number of twelve-foot pine beetles is 1, the number of red pine beetles is 1, the number of six-toothed pine beetles is 1, the number of Armand pine beetles is 1, the number of coleoptera borers is 1, and the number of pine twig borers is 2. The display interface of the crop pest image detection module may be a GUI developed using PyQt5. When the pest category and pest location of the pests in the pest category are known, the number of pests in the pest category can naturally be counted.

[0068] Optionally, the training process of the above-mentioned pest detection network may include the following steps:

[0069] The first step is to build a pest detection network.

[0070] The pest detection network can be an improved version of the YOLOV5 network. The SPPF layer in the Backbone module of the aforementioned pest detection network can be added with the CoordAtt (Coordinate Attention) mechanism. The 3×3Conv (3x3 convolutional layer) in the Backbone module of the aforementioned pest detection network can be replaced with the RepVGGBlock layer from the RepVGG network. During training, the pest detection network can use the Efficient-IOU (EIOU) loss function, which is more effective than the Complete-IOU (CIOU) loss function. By improving the YOLOV5 network, the accuracy and speed of crop pest detection and identification can be improved.

[0071] For example, the YOLOV5 network can be pre-trained using the COCO dataset, and the pre-trained YOLOV5 network can be determined as the initial pest detection network. Subsequently, the initial pest detection network can be trained to obtain a trained pest detection network.

[0072] The second step is to obtain a collection of crop pest images.

[0073] The training labels corresponding to the crop pest images in the above-mentioned crop pest image set may include: pest category and pest location. The training labels corresponding to the crop pest images in the above-mentioned crop pest image set may be known. The crop pest images in the crop pest image set may be images captured by a camera. The camera may be mounted on the top of the trap. A glass container containing alcohol may be placed at the bottom of the trap. The trap may be placed in the crop area. The camera may capture a crop pest image every 10 minutes. The format of the crop pest images in the crop pest image set may be JPEG format. The image resolution of the crop pest images may be 1286×1286 pixels. The number of crop pest images in the crop pest image set may be 2000.

[0074] For example, the tag file corresponding to the above crop pest image set may be saved, wherein the tag file corresponding to the crop pest image set may be an XML (Extensible Markup Language) file of the crop pest image set.

[0075] In the third step, the pest detection network is trained using the crop pest image set and the training labels corresponding to the crop pest images in the crop pest image set to obtain a trained pest detection network.

[0076] For example, the pest detection network can be trained using each crop pest image in the crop pest image set and the training labels corresponding to each crop pest image in the crop pest image set to obtain a trained pest detection network.

[0077] For another example, using the above crop pest image set and the training labels corresponding to the crop pest images in the above crop pest image set to train a pest detection network to obtain a trained pest detection network may include the following steps:

[0078] In the first step, feature extraction and bounding box prediction are performed on each crop pest image in the above crop pest image set to obtain prediction information corresponding to the above crop pest image, and the prediction information is saved in a label file corresponding to the crop pest image set.

[0079] The prediction information may include: a plurality of prediction boxes and categories corresponding to the prediction boxes. The prediction box may be a rectangular box that is predicted to enclose the pest. The category corresponding to the prediction box may be the category of the pest enclosed by the prediction box.

[0080] For example, this step may include the following sub-steps:

[0081] The first sub-step is to adjust the image size of the crop pest images in the crop pest image set according to a preset target image resolution.

[0082] The target image resolution may be the same as the image resolution of the pest image.

[0083] For example, the image resolution of the crop pest images in the crop pest image set may be adjusted to the target image resolution, so as to adjust the image size of the crop pest images.

[0084] In the second sub-step, the YOLOV5 network structure pre-trained model on the COCO dataset and the custom crop pest image training set are used as the source domain I s and target domain I d .

[0085] For example, a YOLO V5 network structure can be pre-trained on the COCO dataset to obtain a pre-trained model, which can be used as the source domain. A custom crop pest image training set can be used as the target domain. The obtained pre-trained model can be the pest detection network constructed in the first step, which is optionally included in step 203.

[0086] The third sub-step is to use the Backbone module in the pest detection network to detect I s with I dThe feature data is mixed and the common features in the two domains are extracted. The FPN (feature pyramid) structure is used to extract features from the bottom up, and finally three feature maps of different scales are obtained.

[0087] In the fourth sub-step, the K-means clustering algorithm is used to perform adaptive anchor frame calculation on the prior frame of the pests in the crop pest image to determine the size of the final prior frame.

[0088] The prior frame may be a rectangular frame that frames the pests in the crop pest image.

[0089] In the fifth sub-step, a multi-scale feature map detection method in the Head module in the pest detection network is used to perform a Conv2d convolution operation on the three feature maps of different scales obtained in the third sub-step included in the third step of step 203, for example, the first step, and finally obtain three feature maps of sizes bs×80×80×[na×(nc+1+4)], bs×40×40×[na×(nc+1+4)], and bs×20×20×[na×(nc+1+4)].

[0090] Where bs is the batch size of crop pest images in the image collection. na is the number of anchor boxes in each feature map. The number of anchor boxes in each feature map can be the same. nc is the total number of pest categories that the pest detection network needs to identify. 1 represents the confidence score of the foreground and background. 4 represents the center point coordinates, width, and height, a total of four elements.

[0091] The sixth sub-step is to design a feature extractor to extract features through domain adaptation method, and perform feature conversion on the extracted features so that I s and I d The characteristic distribution is closer.

[0092] The seventh sub-step is to apply the anchor box (anchor_box) to the feature map output by the fifth sub-step included in the third step of step 203, and generate a final output vector with category probability, confidence and bounding box, and use the cross-grid matching rule to select the positive sample anchor box as the positive sample prediction box.

[0093] The anchor box can be a priori box. The positive sample prediction box can be the prediction box included in the prediction information. Based on the generated final output vector with class probability, confidence, and bounding box, the category corresponding to the positive sample prediction box, that is, the category corresponding to the prediction box, can be determined.

[0094] This sub-step may include the following steps:

[0095] First, compare the width and height of the predicted box and the prior box. If the width and height are greater than the preset threshold, it is a negative sample (background), otherwise it is a positive sample (with an object).

[0096] Then, after the predicted box matches the prior box, we check which grid the center of the predicted box falls on, and find the two grids closest to the center of the predicted box from the four grids above, below, left, and right of the current grid. Together with the current grid, a total of three grids are matched, thereby increasing the number of positive samples and accelerating convergence.

[0097] The current grid may be the grid where the center point of the prediction box falls. The grid may be a square with a preset side length. The preset length may be a pre-set length. For example, each crop pest image in a crop pest image collection may be divided into a plurality of grids.

[0098] In the second step, the pest detection network is trained using a collection of crop pest images, and the loss function value of the pest detection network is calculated for each iteration until the loss function value converges to the minimum, thereby obtaining a trained pest detection network.

[0099] For example, you can use a Nidia GeForce RTX3090 GPU with 16GB of video memory to train a pest detection network for 24 hours. The specific steps include:

[0100] First, set the number of iterations (Epoch) to 200 and the number of crop pest images read in at a time (Batch-size) to 32.

[0101] Then, after label assignment, the target box and the positive sample prediction box can be matched, and the error between the two can be calculated.

[0102] The target box can be the final rectangular box enclosing the pest. The positive sample prediction box can be the predicted rectangular box enclosing the pest. The traditional YOLOV5 uses the CIOU Loss loss function. While CIOU Loss considers the overlap area, center point distance, and aspect ratio for bounding box regression, it reflects the difference in aspect ratio between the predicted and actual boxes, rather than the actual difference in width and height and their confidence. This can sometimes hinder the model's effective optimization of similarity. To address this issue, we replaced CIOU Loss with EIOU Loss.

[0103] L EIOU =L IOU +L dis +L asp

[0104] L IOU =1-IOU

[0105]

[0106] Among them, A and B are the areas of the target box and the positive sample prediction box respectively. EIOU is the EIOU loss function. L IOU is the IOU loss. dis is the distance loss. L asp is the edge length loss.

[0107]

[0108] Where Cw and Ch are the width and height of the minimum bounding box covering the two boxes. b The center coordinates of the predicted box. b gt is the center coordinate of the true box. c is the Euclidean distance between the two diagonal vertices of the minimum rectangular box. The minimum rectangular box can be the minimum bounding rectangle of the true box and the predicted box. w gt is the width of the ground truth box. gt is the height of the groundtruth box. w is the width of the bounding box. h is the height of the bounding box. ρ is the distance between the center of the groundtruth box and the predicted box. is the square of the minimum rectangular frame width. is the square of the minimum rectangular length.

[0109] The coordinates of the center point of the frame that frames the pest can represent the location of the pest.

[0110] In step 204, the crop pest image batch detection module is used to obtain image directory information, detect and identify images corresponding to the image directory information based on the pest detection network, obtain pest information corresponding to the images corresponding to the image directory information, save the images corresponding to the image directory information and the pest information corresponding to the images corresponding to the image directory information to a pre-set folder, and display the save path.

[0111] In some embodiments, the above-mentioned crop pest image batch detection module can be used to obtain image directory information, detect and identify the images corresponding to the above-mentioned image directory information according to the above-mentioned pest detection network, obtain the pest information corresponding to the images corresponding to the above-mentioned image directory information, save the images corresponding to the above-mentioned image directory information and the pest information corresponding to the images corresponding to the above-mentioned image directory information to a pre-set folder, and display the save path.

[0112] The image directory information may be the path of a folder containing insect pest images, and the save path may be the path of the pre-set folder.

[0113] As an example, the display interface of the crop pest image batch detection module can be as follows: Figure 6 As shown in FIG. The display interface of the crop pest image batch detection module can be a GUI developed using PyQt5. Through the pest detection network, each pest image in the image directory information can be detected and identified.

[0114] In step 205, the crop pest video detection module is used to obtain pest videos, detect and identify images constituting the pest videos according to the pest detection network, obtain pest information corresponding to the images constituting the pest videos, and display the pest information in real time.

[0115] In some embodiments, the crop pest video detection module is used to obtain pest videos. It can detect and identify the images that constitute the pest videos based on the pest detection network, obtain pest information corresponding to the images that constitute the pest videos, and display the pest information in real time.

[0116] The pest video may be a video of crops to be detected for pests. The pest information may include: pest category, pest location, and the number of pests in each pest category.

[0117] As an example, the display interface of the crop pest video detection module can be as follows: Figure 7 As shown in the figure, the display interface of the crop pest video detection module can be a GUI developed using PyQt5. The pest detection network can detect and identify pests in each frame of the pest video.

[0118] The present invention provides a real-time, multifunctional, intelligent pest and disease identification and early warning system that can automatically, efficiently, and accurately identify, detect, and warn of crop pests and diseases, has good scalability and robustness, and features a user-friendly interface for easy user use. The real-time, multifunctional, intelligent pest and disease identification and early warning system can include a crop disease identification module, a crop disease early warning module, a crop pest image detection module, a crop pest image batch detection module, and a crop pest video detection module. First, the crop disease identification module is used to acquire disease images, perform disease identification on the disease images based on a trained disease recognition network, and obtain the disease category and cause of the disease corresponding to the disease image. Next, the crop disease early warning module is configured to obtain a disease image folder and a disease threshold. Using the disease recognition network, it performs disease recognition on each disease image in the disease image folder to obtain the disease category corresponding to the disease image. Based on the disease category and disease cause corresponding to the disease images in the disease image folder, it determines the total disease rate. Based on the total disease rate and the disease threshold, it generates disease information corresponding to the disease image folder and saves the disease information to a pre-set text file. The disease recognition network can be a RepVGG convolutional neural network, which can improve the accuracy and speed of disease recognition and, consequently, the accuracy and speed of disease information generation. Next, the crop pest image detection module is configured to obtain pest images and, using the trained pest detection network, detect and identify the pest images. It then obtains and displays the pest information corresponding to the pest images. The pest information includes: pest category, pest location, and the number of pests within the pest category. Next, the crop pest image batch detection module is used to obtain image directory information. Using the pest detection network, it detects and identifies the images corresponding to the image directory information, obtaining pest information corresponding to the images in the directory information. The images and pest information corresponding to the images in the directory information are saved to a pre-set folder, and the save path is displayed. Finally, the crop pest video detection module is used to obtain pest videos. Using the pest detection network, it detects and identifies the images comprising the pest video, obtaining pest information corresponding to the images in the video, and displays the pest information in real time. Since the pest detection network can be a modified YOLOV5 network, the SPPF layer in the Backbone module of the pest detection network can be enhanced with a CoordAtt (Coordinate Attention) mechanism. The 3×3Conv layer in the Backbone module of the pest detection network can be replaced with the RepVGGBlock layer from the RepVGG network.During the training process, the pest detection network can employ the Efficient-IOU (EIOU) loss function, which is more effective than the Complete-IOU (CIOU) loss function. Therefore, using the pest detection network to detect and identify pest images can improve the speed and accuracy of detection and identification. Therefore, the present invention can automatically, efficiently, and accurately identify, detect, and provide early warnings for crop pests and diseases, exhibits excellent scalability and robustness, and provides a user-friendly interface for ease of use.

[0119] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A real-time multifunctional intelligent pest identification and early warning system, characterized by: It includes crop disease recognition module, crop disease early warning module, crop pest image detection module, crop pest image batch detection module and crop pest video detection module; The crop disease recognition module is used to obtain a disease image, perform disease recognition on the disease image according to the trained disease recognition network, and obtain the disease category corresponding to the disease image; The crop disease early warning module is used to obtain a disease image folder and a disease threshold, perform disease recognition on each disease image in the disease image folder according to the disease recognition network, obtain the disease category corresponding to the disease image, determine the total disease rate according to the disease category corresponding to the disease images in the disease image folder, generate disease information corresponding to the disease image folder according to the total disease rate and the disease threshold, and save the disease information to a preset text file; The crop pest image detection module is used to obtain pest images, detect and identify the pest images based on the trained pest detection network, and obtain and display pest information corresponding to the pest images, including: pest category, pest location, and the number of pests in each pest category; The crop pest image batch detection module is used to obtain image directory information, detect and identify images corresponding to the image directory information according to the pest detection network, obtain pest information corresponding to the images corresponding to the image directory information, save the images corresponding to the image directory information and the pest information corresponding to the images corresponding to the image directory information to a pre-set folder, and display the save path; The crop pest video detection module is used to obtain pest videos, detect and identify images constituting the pest videos according to the pest detection network, obtain pest information corresponding to the images constituting the pest videos, and display the pest information in real time.

2. The system according to claim 1, wherein: The training process of the disease recognition network includes: Build a disease identification network; Acquire a crop disease image set, wherein the training labels corresponding to the crop disease images in the crop disease image set include: disease categories; The disease recognition network is trained using the crop disease image set and the training labels corresponding to the respective crop disease images in the crop disease image set to obtain a trained disease recognition network.

3. The system according to claim 1, wherein: The training process of the pest detection network includes: Building a pest detection network; Acquire a crop pest image set, wherein the training labels corresponding to the crop pest images in the crop pest image set include: pest category and pest location; The pest detection network is trained using the crop pest image set and the training labels corresponding to the respective crop pest images in the crop pest image set to obtain a trained pest detection network.

4. The system according to claim 1, wherein: The disease recognition network is a RepVGG convolutional neural network, and the loss function in the disease recognition network training process is a Loss Function.

5. The system according to claim 1, wherein: The pest detection network is an improved YOLOV5 network. An attention mechanism is added to the SPPF layer in the Backbone module of the pest detection network. The 3x3 convolutional layer in the Backbone module of the pest detection network is replaced by the RepVGGBlock layer in the RepVGG network. During the training process, the pest detection network adopts the Efficient-IOU loss function, which is more effective than the Complete-IOU loss function.

Images