A mulberry tree disease and insect pest monitoring system and method based on Internet of Things image processing

Abstract

The present invention relates to a mulberry pest monitoring system and method based on Internet of Things image processing, which includes obtaining mulberry pest image data, preprocessing and labeling, creating and training a pest analysis model, evaluating the performance of the trained pest analysis model, performing preliminary image acquisition on the monitored mulberry planting area through a first image acquisition module, transmitting the image to the pest analysis model for preliminary pest situation identification and analysis, and locating each pest area, expanding each pest area, and performing infrared image acquisition on the expanded pest areas through a second image acquisition module, identifying and analyzing the pest situation of each expanded pest area, and a pest drawing module drawing the pest area of ​​the monitored mulberry planting area based on the output analysis result, rendering the pest severity, and executing pest control measures. The system realizes all-round pest monitoring of mulberry trees and improves the accuracy of pest identification and analysis results.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a mulberry tree disease and insect pest monitoring system and method based on Internet of Things image processing. Background Art

[0002] The mulberry tree is a deciduous tree or shrub that has many uses. It can be used as livestock feed or made into products such as mulberry leaf tea. The fruit of the mulberry tree is called mulberry, which can be eaten raw or used to make juice, wine, etc. In addition, the leaves, roots, fruits and branches of the mulberry tree can be used as medicine. The bark fiber is fine and can be used as a raw material for textiles and papermaking. The wood is hard and suitable for making furniture, musical instruments and carvings. my country is the largest mulberry planting country. Mulberry leaves occupy a major position in the sericulture industry, so the most important use of mulberry leaves is as the main feed for silkworms.

[0003] During the planting process of mulberry trees, managers need to pay special attention to the pests and diseases of mulberry trees, because the pests and diseases of mulberry trees are closely related to the safety of silkworms during the sericulture process. If mulberry trees are affected by pests and diseases, when the mulberry leaves of the mulberry trees affected by pests and diseases are picked and fed to silkworms, the silkworms will become sick after eating the pest-infected mulberry leaves, which will cause the silkworms to be unable to grow normally, and even die in large numbers, causing huge losses to silkworm farmers. Therefore, during the planting process of mulberry trees, the pest and disease monitoring of mulberry trees is extremely important. It is necessary to dynamically monitor the entire growth process of mulberry trees, dynamically obtain the pest and disease conditions of mulberry trees, and promptly carry out corresponding pest and disease treatment. At the same time, when picking mulberry leaves, healthy mulberry leaves can be picked, thereby ensuring that the silkworms fed with mulberry leaves can eat healthy mulberry leaves and grow normally.

[0004] However, traditional pest and disease monitoring is generally based on the experience of managers. Later, with the development of drone technology, drones were used to take images and perform pest and disease monitoring based on image analysis. However, most of the images acquired were of the upper surface of the leaves. In fact, more pest and disease processes of mulberry trees occur on the lower surface of the leaves. Therefore, the accuracy of the existing image processing and analysis process for identifying and analyzing pests is very low.

[0005] Therefore, how to improve the existing mulberry pest and disease monitoring methods to achieve comprehensive pest and disease monitoring of mulberry trees and improve the accuracy of pest and disease identification and analysis results is a technical problem that urgently needs to be solved. Summary of the invention

[0006] The object of the present invention is to provide a mulberry tree disease and pest monitoring system and method based on Internet of Things image processing, so as to improve the existing mulberry tree disease and pest monitoring method, so as to realize all-round disease and pest monitoring of mulberry trees and improve the accuracy of disease and pest identification and analysis results.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0008] In a first aspect, a mulberry tree pest and disease monitoring method based on Internet of Things image processing is provided, comprising the following steps:

[0009] S1: Create a pest and disease analysis model, obtain pest and disease image data of mulberry trees at various growth stages based on the Internet of Things, preprocess and annotate the pest and disease image data, and divide the annotated pest and disease data into two parts according to a specified ratio, one part of which is transmitted to the pest and disease analysis model to train the pest and disease model;

[0010] S2: The other part is transmitted to the trained pest and disease analysis model, the performance of the trained pest and disease analysis model is evaluated, and it is determined whether the evaluation result meets the specified requirements. If yes, step S3 is executed. If no, the hyperparameters of the pest and disease analysis model are optimized and adjusted until its performance evaluation result meets the specified requirements.

[0011] S3: Performing preliminary image acquisition of the monitored mulberry tree planting area through the first image acquisition module, and inputting the preliminary acquired image data into the pest and disease analysis model, wherein the pest and disease analysis model performs preliminary pest and disease situation identification and analysis on the monitored mulberry tree planting area based on the preliminary acquired image data, and locates each pest and disease area;

[0012] S4: Enlarging each pest and disease area located in step S3 according to a specified ratio, acquiring infrared images of each enlarged pest and disease area through a second image acquisition module, and inputting the acquired infrared image data into the pest and disease analysis model;

[0013] S5: converting the infrared image data into a common image, and further identifying and analyzing the pest conditions of each expanded pest area by a pest analysis model, and outputting analysis results, wherein the analysis results include the pest area and the severity of the corresponding pests in the pest area;

[0014] S6: The pest and disease drawing module draws the pest and disease area of ​​the monitored mulberry tree planting area based on the output analysis results, and renders the severity of the pest and disease in the drawn pest and disease area. The management personnel implement corresponding pest and disease control measures based on the drawn pest and disease area and the corresponding severity of the pest and disease;

[0015] S7: After the specified time period, execute steps S3-S6 again to determine whether the pests and diseases are effectively controlled. If so, continue to execute the current pest and disease control measures until the pests and diseases are completely controlled; if not, adjust the pest and disease control measures and execute the specified time period, and execute steps S3-S6 again.

[0016] Preferably, the specific process of data preprocessing in step S1 is as follows:

[0017] S11: removing duplicate images, images with resolutions lower than a specified threshold, and invalid images from the acquired pest and disease image data, performing cloud and shadow processing on the remaining pest and disease images, and converting them into grayscale images of a specified size;

[0018] S12: Smoothing the specified image, calculating a weight matrix for a specified pixel of the specified image, calculating the grayscale values ​​of a specified number of pixels around the specified pixel, and multiplying the grayscale value of each pixel by the corresponding weight value in the weight matrix to obtain a Gaussian blur value of the specified pixel;

[0019] S13: adjusting the contrast of the image, and using the edge detection algorithm of the Laplace operator to enhance the edge information of the designated target in the image.

[0020] Preferably, the specific process of data labeling in step S1 is as follows:

[0021] S14: identifying the pest-related image features in the preprocessed pest image, and extracting the corresponding image features of the pest image based on the identification result;

[0022] S15: classifying the extracted image features according to the corresponding pest categories, setting the corresponding pest severity level, and dividing the pest image features of each pest category according to the pest severity level;

[0023] S16: Mark the pests and diseases image according to the pests and diseases category and the corresponding pests and diseases severity level.

[0024] Preferably, the pest analysis model is a neural network model, comprising an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer;

[0025] The input layer is a single channel, and the input layer inputs a 28×28 grayscale image;

[0026] The convolution layer is provided with two layers, including a convolution layer with 6 5×5 convolution kernels and a convolution layer with 16 5×5 convolution kernels;

[0027] The pooling layer has two layers, both of which are 2×2 maximum pooling;

[0028] The fully connected layer is provided with two layers. The input of the first fully connected layer is 16×4×4 image features and the output is 120 nodes. The input of the second fully connected layer is 120 nodes and the output is 84 nodes.

[0029] Preferably, the training process of the pest analysis model is:

[0030] S17: Initializing various parameters of the pest and disease analysis model, wherein the parameters include weights and biases of convolution kernels of a convolution layer of the model;

[0031] S18: Perform convolution operation, activation function, pooling operation and calculation of the fully connected layer on the input image through the convolution layer, pooling layer and fully connected layer, so as to extract the feature vector by convolution and pooling processing of the input image;

[0032] S19: Input the extracted feature vector into the fully connected layer to obtain the classification and recognition results of the corresponding pest and disease features;

[0033] S20: Measure the difference between the output result of the pest and disease analysis model and the true label by specifying the loss function, and update the parameters of the pest and disease analysis model by the gradient descent algorithm based on the difference between the output result and the true label to minimize the loss function, and iterate steps S18-S20 until the preset number of iterations is reached or the loss function reaches the specified value.

[0034] Preferably, the second image acquisition module is an infrared image acquisition module, and the specific process in step S4 is as follows:

[0035] S41: Determine the center point and radius of each pest and disease area, expand the radius by a specified multiple, and expand each pest and disease area according to the original center point and the radius expanded by the specified multiple;

[0036] S42: Collect infrared images of each expanded pest and disease area through the infrared image acquisition module, and input the collected infrared image data into the pest and disease analysis model.

[0037] Preferably, the specific process of converting the infrared image data into a common image in step S5 is as follows:

[0038] S51: converting the acquired infrared image into a digital image, and extracting temperature distribution information in the digital image;

[0039] S52: defining a temperature-color mapping relationship table, and calling the temperature-color mapping relationship table to convert temperature distribution information in the digital image into corresponding color information;

[0040] S53: The color information is integrated with the original infrared image to generate a common image, and the pest analysis model performs pest identification and analysis based on the common image.

[0041] In a second aspect, a mulberry pest and disease monitoring system based on Internet of Things image processing is provided, which is used to implement any one of the mulberry pest and disease monitoring methods based on Internet of Things image processing, including a data acquisition module, a data preprocessing module, a data annotation module, a pest and disease analysis model, a first image acquisition module, a second image acquisition module, an image conversion module and a pest and disease drawing module, wherein the data acquisition module is connected to the data preprocessing module, the data preprocessing module is connected to the data annotation module, the data annotation module is connected to the pest and disease analysis model, the first image acquisition module is connected to the pest and disease analysis model, the second image acquisition module is connected to the pest and disease analysis model through the image conversion module, and the pest and disease drawing module is connected to the pest and disease analysis model;

[0042] The data acquisition module is used to acquire image data of pests and diseases at various growth stages of mulberry trees;

[0043] The data preprocessing module is used to preprocess the pest image data;

[0044] The data annotation module is used to perform data annotation processing on the pre-processed pest and disease image data;

[0045] The pest and disease analysis model is used to perform preliminary pest and disease identification and analysis on the monitored mulberry tree planting area based on the initially collected image data, locate each pest and disease area, further identify and analyze the pest and disease situation of each expanded pest and disease area based on the infrared image data, and output the analysis results;

[0046] The first image acquisition module is used to collect preliminary images of the mulberry tree planting area to be monitored and transmit them to the pest and disease analysis model;

[0047] The second image acquisition module is used to acquire infrared images of each expanded pest and disease area, and input the acquired infrared image data into the pest and disease analysis model;

[0048] The image conversion module is used to convert the infrared image into a normal image;

[0049] The pest and disease drawing module is used to draw the pest and disease area of ​​the mulberry tree planting area to be monitored based on the output analysis results, and to render the severity of the pest and disease in the drawn pest and disease area.

[0050] The beneficial effects of the present invention include:

[0051] The mulberry pest and disease monitoring system and method based on Internet of Things image processing provided by the present invention obtains mulberry pest and disease image data, pre-processes and annotates it, creates and trains a pest and disease analysis model; evaluates the performance of the trained pest and disease analysis model; performs preliminary image acquisition of the monitored mulberry planting area through the first image acquisition module, transmits it to the pest and disease analysis model for preliminary pest and disease situation identification and analysis, and locates each pest and disease area; expands each pest and disease area, and the second image acquisition module performs infrared image acquisition of each expanded pest and disease area; identifies and analyzes the pest and disease situation of each expanded pest and disease area; the pest and disease drawing module draws the pest and disease area of ​​the monitored mulberry planting area based on the output analysis result and renders the severity of the pest and disease, and the management personnel execute corresponding pest and disease control measures based on the drawn pest and disease area and the corresponding pest and disease severity. All-round pest and disease monitoring of mulberry trees is realized, and the accuracy of pest and disease identification and analysis results is improved.

[0052] Firstly, by removing duplicate images, images with resolution lower than the specified threshold, and invalid images from the acquired pest and disease image data, processing clouds and shadows and converting them into specified sizes, smoothing the specified images, adjusting the image contrast, and enhancing the edge information, the influence of noise data on the model training process is avoided, the training efficiency of the model is improved, and the performance of the model meets the specified requirements.

[0053] Secondly, by identifying the image features of pests and diseases in the preprocessed pest and disease images and extracting the corresponding image features, classifying and grading the pest and disease categories, and marking them according to the pest and disease categories and severity levels, the trained pest and disease analysis model can realize the recognition and analysis of pest and disease categories and severity levels.

[0054] Again, the pest and disease analysis model sets up an input layer, a convolution layer with 6 5×5 convolution kernels and a convolution layer with 16 5×5 convolution kernels, two pooling layers with 2×2 maximum pooling, the first fully connected layer has an input of 16×4×4 image features and an output of 120 nodes, the second fully connected layer has an input of 120 nodes and an output of 84 nodes. This structure enables the model to efficiently extract target image features, learn complex features, and improve the accuracy of the output results.

[0055] Furthermore, the images collected by the first image acquisition module are used to locate the pest area and preliminarily determine the pest area. However, since the first image acquisition module can only obtain images of the upper layer of the mulberry tree, the pests and diseases in the lower layer are ignored. Therefore, the second image acquisition module is used to collect infrared images of each expanded pest area. On the one hand, it can ensure that the pest area is not missed, and on the other hand, it can realize accurate identification and analysis of the pest situation in the pest area. The reason why infrared images are not collected for all the areas to be monitored is to reduce the processing volume of image conversion and improve the efficiency of pest monitoring.

[0056] Finally, by extracting the temperature distribution information in the digital image; defining the temperature color mapping relationship table, calling the temperature color mapping relationship table to convert the temperature distribution information in the digital image into the corresponding color information; and fusing the color information with the original infrared image to obtain a common image, the model can identify the image features related to pests and diseases in the infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The present invention is a flow chart of the mulberry tree disease and insect pest monitoring method based on Internet of Things image processing.

[0058] Figure 2 The figure is a schematic diagram of the process of converting infrared image data into ordinary images according to the present invention.

[0059] Figure 3 This is a schematic diagram of the architecture of the mulberry tree disease and insect pest monitoring system based on Internet of Things image processing of the present invention. DETAILED DESCRIPTION

[0060] The following is combined with Figure 1~Figure 3 The present invention is further described in detail:

[0061] Example 1

[0062] See attached Figure 1 As shown, a mulberry tree disease and insect pest monitoring method based on Internet of Things image processing includes the following steps:

[0063] S1: Create a pest and disease analysis model, obtain pest and disease image data of mulberry trees at various growth stages based on the Internet of Things, preprocess and annotate the pest and disease image data, and divide the annotated pest and disease data into two parts according to a specified ratio, one part of which is transmitted to the pest and disease analysis model to train the pest and disease model;

[0064] S2: The other part is transmitted to the trained pest and disease analysis model, the performance of the trained pest and disease analysis model is evaluated, and it is determined whether the evaluation result meets the specified requirements. If yes, step S3 is executed. If no, the hyperparameters of the pest and disease analysis model are optimized and adjusted until its performance evaluation result meets the specified requirements.

[0065] S3: Performing preliminary image acquisition of the monitored mulberry tree planting area through the first image acquisition module, and inputting the preliminary acquired image data into the pest and disease analysis model, wherein the pest and disease analysis model performs preliminary pest and disease situation identification and analysis on the monitored mulberry tree planting area based on the preliminary acquired image data, and locates each pest and disease area;

[0066] S4: Enlarging each pest and disease area located in step S3 according to a specified ratio, acquiring infrared images of each enlarged pest and disease area through a second image acquisition module, and inputting the acquired infrared image data into the pest and disease analysis model;

[0067] S5: converting the infrared image data into a common image, and further identifying and analyzing the pest conditions of each expanded pest area by a pest analysis model, and outputting analysis results, wherein the analysis results include the pest area and the severity of the corresponding pests in the pest area;

[0068] S6: The pest and disease drawing module draws the pest and disease area of ​​the monitored mulberry tree planting area based on the output analysis results, and renders the severity of the pest and disease in the drawn pest and disease area. The management personnel implement corresponding pest and disease control measures based on the drawn pest and disease area and the corresponding severity of the pest and disease;

[0069] S7: After the specified time period, execute steps S3-S6 again to determine whether the pests and diseases are effectively controlled. If so, continue to execute the current pest and disease control measures until the pests and diseases are completely controlled; if not, adjust the pest and disease control measures and execute the specified time period, and execute steps S3-S6 again.

[0070] Since the process of pest and disease monitoring based on image analysis in the prior art is all performed by image acquisition through drones, the image data collected is only for the upper layer image of the plant, that is, by analyzing the upper layer image of the plant, only the pest and disease data of the upper layer of the plant can be obtained, but in many cases, the self-protection characteristics of the pests and diseases themselves, the pest and disease positions are all located in the lower layer of the plant leaves or branches, so the prior art cannot achieve accurate analysis of pests and diseases. The present invention collects infrared images by setting a second image acquisition module. Since infrared rays have a certain degree of penetration, the pest and disease data located below the plant can be captured. In addition, for the pest and disease position, there are generally insect bodies and microorganisms, so the temperature of these positions is higher than that of the plant. The infrared image can capture its temperature information, and then accurately capture the pest and disease data, making up for the deficiency that only the upper layer pest and disease data can be captured in the general image, and can achieve all-round and accurate monitoring of the pest and disease of mulberry trees.

[0071] Example 2

[0072] Based on Example 1, the specific process of data preprocessing in step S1 is as follows:

[0073] S11: removing duplicate images, images with resolutions lower than a specified threshold, and invalid images from the acquired pest and disease image data, performing cloud and shadow processing on the remaining pest and disease images, and converting them into grayscale images of a specified size;

[0074] S12: Smoothing the specified image, calculating a weight matrix for a specified pixel of the specified image, calculating the grayscale values ​​of a specified number of pixels around the specified pixel, and multiplying the grayscale value of each pixel by the corresponding weight value in the weight matrix to obtain a Gaussian blur value of the specified pixel;

[0075] S13: adjusting the contrast of the image, and using the edge detection algorithm of the Laplace operator to enhance the edge information of the designated target in the image.

[0076] By removing duplicate images, images with resolutions below a specified threshold, and invalid images from the acquired pest and disease image data, processing clouds and shadows and converting them into specified sizes, smoothing the specified images, adjusting the image contrast, and enhancing the edge information, the influence of noise data on the model training process is avoided, the training efficiency of the model is improved, and the performance of the model meets the specified requirements.

[0077] In this embodiment, the specific process of data labeling in step S1 is as follows:

[0078] S14: identifying the pest-related image features in the preprocessed pest image, and extracting the corresponding image features of the pest image based on the identification result;

[0079] S15: classifying the extracted image features according to the corresponding pest categories, setting the corresponding pest severity level, and dividing the pest image features of each pest category according to the pest severity level;

[0080] S16: Marking the pest image according to the pest category and the corresponding pest severity level. By identifying the pest image features in the pre-processed pest image and extracting the corresponding image features, classifying and grading the pest categories, and marking according to the pest category and severity level, the trained pest analysis model can realize the recognition and analysis of the pest category and severity level.

[0081] Example 3

[0082] On the basis of Example 1 or Example 2, the pest analysis model is a neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer;

[0083] The input layer is a single channel, and the input layer inputs a 28×28 grayscale image.

[0084] The convolution layer has two layers, including a convolution layer with 6 5×5 convolution kernels and a convolution layer with 16 5×5 convolution kernels; the features of the input data are extracted through convolution operation. The convolution layer is composed of multiple convolution kernels, and each convolution kernel is responsible for extracting a feature of the input data.

[0085] The pooling layer is provided with two layers, both of which are 2×2 maximum pooling; the pooling layer can reduce the dimension of the feature map, reduce the amount of calculation, and retain important image feature information.

[0086] The fully connected layer has two layers. The first fully connected layer has an input of 16×4×4 image features and an output of 120 nodes. The second fully connected layer has an input of 120 nodes and an output of 84 nodes. The features extracted by the previous convolutional layer and pooling layer are integrated, linearly transformed through weights and biases, and then nonlinearly transformed through activation functions to finally obtain the classification result.

[0087] Therefore, the pest and disease analysis model sets up an input layer, a convolution layer with 6 5×5 convolution kernels and a convolution layer with 16 5×5 convolution kernels, two 2×2 maximum pooling layers, the first fully connected layer with an input of 16×4×4 image features and an output of 120 nodes, the second fully connected layer with an input of 120 nodes and an output of 84 nodes. This structure enables the model to efficiently extract target image features, learn complex features, and improve the accuracy of the output results.

[0088] In this embodiment, the training process of the pest analysis model is as follows:

[0089] S17: Initializing various parameters of the pest and disease analysis model, wherein the parameters include weights and biases of convolution kernels of a convolution layer of the model;

[0090] S18: Perform convolution operation, activation function, pooling operation and calculation of the fully connected layer on the input image through the convolution layer, pooling layer and fully connected layer, so as to extract the feature vector by convolution and pooling processing of the input image;

[0091] S19: Input the extracted feature vector into the fully connected layer to obtain the classification and recognition results of the corresponding pest and disease features;

[0092] S20: Measure the difference between the output result of the pest and disease analysis model and the true label by specifying the loss function, and update the parameters of the pest and disease analysis model by the gradient descent algorithm based on the difference between the output result and the true label to minimize the loss function, and iterate steps S18-S20 until the preset number of iterations is reached or the loss function reaches the specified value.

[0093] Example 4

[0094] On the basis of Embodiment 1 or Embodiment 2 or Embodiment 3, the second image acquisition module is an infrared image acquisition module, and the specific process in step S4 is as follows:

[0095] S41: Determine the center point and radius of each pest and disease area, expand the radius by a specified multiple, and expand each pest and disease area according to the original center point and the radius expanded by the specified multiple;

[0096] S42: Collect infrared images of each expanded pest and disease area through the infrared image acquisition module, and input the collected infrared image data into the pest and disease analysis model.

[0097] The images collected by the first image acquisition module are used to locate the pest area and preliminarily determine the pest area. However, since the first image acquisition module can only obtain images of the upper layer of the mulberry tree, the pests and diseases in the lower layer are ignored. Therefore, the second image acquisition module is used to collect infrared images of the expanded pest and disease areas. On the one hand, it can ensure that the pest and disease areas are not missed, and on the other hand, it can realize accurate identification and analysis of the pest and disease conditions in the pest and disease areas. The reason why infrared images are not collected for all the areas to be monitored is to reduce the processing volume of image conversion and improve the efficiency of pest and disease monitoring.

[0098] In this embodiment, see Figure 2 As shown, the specific process of converting the infrared image data into a common image in step S5 is as follows:

[0099] S51: converting the acquired infrared image into a digital image, and extracting temperature distribution information in the digital image;

[0100] S52: defining a temperature-color mapping relationship table, and calling the temperature-color mapping relationship table to convert temperature distribution information in the digital image into corresponding color information;

[0101] S53: The color information is integrated with the original infrared image to generate a common image, and the pest analysis model performs pest identification and analysis based on the common image.

[0102] Finally, by extracting the temperature distribution information in the digital image; defining the temperature color mapping relationship table, calling the temperature color mapping relationship table to convert the temperature distribution information in the digital image into the corresponding color information; and fusing the color information with the original infrared image to obtain a common image, the model can identify the image features related to pests and diseases in the infrared image.

[0103] See also Figure 3 As shown, a mulberry pest and disease monitoring system based on Internet of Things image processing is used to implement any one of the mulberry pest and disease monitoring methods based on Internet of Things image processing, including a data acquisition module, a data preprocessing module, a data annotation module, a pest and disease analysis model, a first image acquisition module, a second image acquisition module, an image conversion module and a pest and disease drawing module, the data acquisition module is connected to the data preprocessing module, the data preprocessing module is connected to the data annotation module, the data annotation module is connected to the pest and disease analysis model, the first image acquisition module is connected to the pest and disease analysis model, the second image acquisition module is connected to the pest and disease analysis model through the image conversion module, and the pest and disease drawing module is connected to the pest and disease analysis model.

[0104] The data acquisition module is used to acquire the image data of pests and diseases at various growth stages of mulberry trees; the data preprocessing module is used to preprocess the image data of pests and diseases; the data annotation module is used to perform data annotation processing on the preprocessed image data of pests and diseases; the pest and disease analysis model is used to perform preliminary pest and disease identification and analysis on the monitored mulberry tree planting area based on the initially collected image data, locate each pest and disease area, further identify and analyze the pest and disease situation of each expanded pest and disease area based on the infrared image data, and output the analysis result; the first image acquisition module is used to perform preliminary image acquisition on the monitored mulberry tree planting area and transmit it to the pest and disease analysis model; the second image acquisition module is used to perform infrared image acquisition on each expanded pest and disease area, and input the collected infrared image data into the pest and disease analysis model; the image conversion module is used to convert the infrared image into a normal image; the pest and disease drawing module is used to draw the pest and disease area of ​​the monitored mulberry tree planting area based on the output analysis result, and render the severity of the pest and disease in the drawn pest and disease area.

[0105] In summary, the mulberry pest monitoring system and method based on Internet of Things image processing provided by the present invention, by removing duplicate images, images with resolutions lower than a specified threshold, and invalid images in the acquired pest image data, and performing cloud and shadow processing and converting them into a specified size, smoothing the specified image, adjusting the image contrast, and enhancing the edge information, avoids the influence of noise data on the model training process, improves the training efficiency of the model, and enables the performance of the model to meet the specified requirements. By identifying the pest image features in the pre-processed pest image and extracting the corresponding image features, classifying and grading the pest categories, and marking them according to the pest categories and severity levels, the trained pest analysis model can realize the recognition and analysis of the pest categories and severity levels.

[0106] The pest analysis model sets up an input layer, a convolution layer with 6 5×5 convolution kernels and a convolution layer with 16 5×5 convolution kernels, two 2×2 maximum pooling layers, the first fully connected layer has an input of 16×4×4 image features and an output of 120 nodes, and the second fully connected layer has an input of 120 nodes and an output of 84 nodes. The structure of the fully connected layer enables the model to efficiently extract target image features and learn complex features to improve the accuracy of the output results. The images collected by the first image acquisition module are used to locate the pest area and preliminarily determine the pest area. However, since the first image acquisition module can only obtain the upper layer image of the mulberry tree, the pest situation in the lower layer is ignored. Therefore, the infrared image is collected by the second image acquisition module to collect infrared images of each expanded pest area. On the one hand, it can ensure that the pest area will not be missed, and on the other hand, it can realize the accurate identification and analysis of the pest situation in the pest area. The reason why infrared images are not collected for all the areas to be monitored is to reduce the processing volume of image conversion and improve the efficiency of pest and disease monitoring. The model can identify the image features related to pests and diseases in infrared images by extracting the temperature distribution information in the digital image; defining the temperature color mapping relationship table, calling the temperature color mapping relationship table to convert the temperature distribution information in the digital image into the corresponding color information; and fusing the color information with the original infrared image to obtain a common image.

Claims

1. A mulberry tree disease and insect pest monitoring method based on Internet of Things image processing, characterized in that: The following steps are involved: S1: Create a pest and disease analysis model, obtain pest and disease image data of mulberry trees at various growth stages, preprocess and annotate the pest and disease image data, and divide the annotated pest and disease data into two parts according to a specified ratio, one part of which is transmitted to the pest and disease analysis model to train the pest and disease model; S2: transmitting the other part to the trained pest and disease analysis model, evaluating the performance of the trained pest and disease analysis model, and determining whether the evaluation result meets the specified requirements. If so, executing step S3; if not, optimizing and adjusting the hyperparameters of the pest and disease analysis model until its performance evaluation result meets the specified requirements; S3: Performing preliminary image acquisition of the monitored mulberry tree planting area through the first image acquisition module, and inputting the preliminary acquired image data into the pest and disease analysis model, wherein the pest and disease analysis model performs preliminary pest and disease situation identification and analysis on the monitored mulberry tree planting area based on the preliminary acquired image data, and locates each pest and disease area; S4: Enlarging each pest and disease area located in step S3 according to a specified ratio, acquiring infrared images of each enlarged pest and disease area through a second image acquisition module, and inputting the acquired infrared image data into the pest and disease analysis model; S5: converting the infrared image data into a common image, and further identifying and analyzing the pest conditions of each expanded pest area by a pest analysis model, and outputting analysis results, wherein the analysis results include the pest area and the severity of the corresponding pests in the pest area; S6: The pest and disease drawing module draws the pest and disease area of ​​the monitored mulberry tree planting area based on the output analysis results, and renders the severity of the pest and disease in the drawn pest and disease area. The management personnel implement corresponding pest and disease control measures based on the drawn pest and disease area and the corresponding severity of the pest and disease; S7: After the specified time period, execute steps S3-S6 again to determine whether the pests and diseases are effectively controlled. If so, continue to execute the current pest and disease control measures until the pests and diseases are completely controlled; if not, adjust the pest and disease control measures and execute the specified time period, and execute steps S3-S6 again; The specific process of converting the infrared image data into a common image in step S5 is as follows: S51: converting the acquired infrared image into a digital image, and extracting temperature distribution information in the digital image; S52: defining a temperature-color mapping relationship table, and calling the temperature-color mapping relationship table to convert temperature distribution information in the digital image into corresponding color information; S53: The color information is integrated with the original infrared image to generate a common image, and the pest analysis model performs pest identification and analysis based on the common image.

2. A mulberry tree pest and disease monitoring method based on Internet of Things image processing according to claim 1, characterized in that: The specific process of data preprocessing in step S1 is as follows: S11: removing duplicate images, images with resolutions lower than a specified threshold, and invalid images from the acquired pest and disease image data, performing cloud and shadow processing on the remaining pest and disease images, and converting them into grayscale images of a specified size; S12: Smoothing the specified image, calculating a weight matrix for a specified pixel of the specified image, calculating the grayscale values ​​of a specified number of pixels around the specified pixel, and multiplying the grayscale value of each pixel by the corresponding weight value in the weight matrix to obtain a Gaussian blur value of the specified pixel; S13: adjusting the contrast of the image, and using the edge detection algorithm of the Laplace operator to enhance the edge information of the designated target in the image.

3. A mulberry tree pest and disease monitoring method based on Internet of Things image processing according to claim 2, characterized in that: The specific process of data labeling in step S1 is as follows: S14: identifying the pest-related image features in the preprocessed pest image, and extracting the corresponding image features of the pest image based on the identification result; S15: classifying the extracted image features according to the corresponding pest categories, setting the corresponding pest severity level, and dividing the pest image features of each pest category according to the pest severity level; S16: Mark the pests and diseases image according to the pests and diseases category and the corresponding pests and diseases severity level.

4. A mulberry tree pest and disease monitoring method based on Internet of Things image processing according to claim 1, characterized in that: The pest analysis model is a neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; The input layer is a single channel, and the input layer inputs a 28×28 grayscale image; The convolution layer is provided with two layers, including a convolution layer with 6 5×5 convolution kernels and a convolution layer with 16 5×5 convolution kernels; The pooling layer has two layers, both of which are 2×2 maximum pooling; The fully connected layer is provided with two layers. The input of the first fully connected layer is 16×4×4 image features and the output is 120 nodes. The input of the second fully connected layer is 120 nodes and the output is 84 nodes.

5. A mulberry pest monitoring method based on Internet of Things image processing according to claim 4, characterized in that: The training process of the pest analysis model is as follows: S17: Initializing various parameters of the pest and disease analysis model, wherein the parameters include weights and biases of convolution kernels of a convolution layer of the model; S18: Perform convolution operation, activation function, pooling operation and calculation of the fully connected layer on the input image through the convolution layer, pooling layer and fully connected layer, so as to extract the feature vector by convolution and pooling processing of the input image; S19: Input the extracted feature vector into the fully connected layer to obtain the classification and recognition results of the corresponding pest and disease features; S20: Measure the difference between the output result of the pest and disease analysis model and the true label by specifying the loss function, and update the parameters of the pest and disease analysis model by the gradient descent algorithm based on the difference between the output result and the true label to minimize the loss function, and iterate steps S18-S20 until the preset number of iterations is reached or the loss function reaches the specified value.

6. A mulberry pest monitoring method based on Internet of Things image processing according to claim 1, characterized in that: The second image acquisition module is an infrared image acquisition module, and the specific process in step S4 is as follows: S41: Determine the center point and radius of each pest and disease area, expand the radius by a specified multiple, and expand each pest and disease area according to the original center point and the radius expanded by the specified multiple; S42: Collect infrared images of each expanded pest and disease area through the infrared image acquisition module, and input the collected infrared image data into the pest and disease analysis model.

7. A mulberry pest and disease monitoring system based on Internet of Things image processing, used to implement a mulberry pest and disease monitoring method based on Internet of Things image processing according to any one of claims 1 to 6, characterized in that: It includes a data acquisition module, a data preprocessing module, a data annotation module, a pest and disease analysis model, a first image acquisition module, a second image acquisition module, an image conversion module and a pest and disease drawing module, wherein the data acquisition module is connected to the data preprocessing module, the data preprocessing module is connected to the data annotation module, the data annotation module is connected to the pest and disease analysis model, the first image acquisition module is connected to the pest and disease analysis model, the second image acquisition module is connected to the pest and disease analysis model via the image conversion module, and the pest and disease drawing module is connected to the pest and disease analysis model; The data acquisition module is used to acquire image data of pests and diseases at various growth stages of mulberry trees; The data preprocessing module is used to preprocess the pest image data; The data annotation module is used to perform data annotation processing on the pre-processed pest and disease image data; The pest and disease analysis model is used to perform preliminary pest and disease identification and analysis on the monitored mulberry tree planting area based on the initially collected image data, locate each pest and disease area, further identify and analyze the pest and disease situation of each expanded pest and disease area based on the infrared image data, and output the analysis results; The first image acquisition module is used to collect preliminary images of the mulberry tree planting area to be monitored and transmit them to the pest and disease analysis model; The second image acquisition module is used to acquire infrared images of each expanded pest and disease area, and input the acquired infrared image data into the pest and disease analysis model; The image conversion module is used to convert the infrared image into a normal image; The pest and disease drawing module is used to draw the pest and disease area of ​​the mulberry tree planting area to be monitored based on the output analysis results, and to render the severity of the pest and disease in the drawn pest and disease area.

Images