Grapholitha molesta monitoring and predicting method based on deep learning and BP neural network

Abstract

The invention is suitable for the technical field of pest control, and provides a grapholitha molesta monitoring and predicting method based on deep learning and a BP neural network, and the method can effectively monitor the insect attack condition of grapholitha molesta and predict the insect attack development condition through the combination of a deep learning algorithm and the BP neural network. The monitoring and predicting method is easy to operate, the prediction result basically coincides with the actual result, data are reliable, and a basis can be provided for scientific research and pest control in agricultural planting.

Description

Technical Field

[0001] The invention belongs to the technical field of pest control, and in particular relates to a pear borer monitoring and prediction method based on deep learning and BP neural network. Background Art

[0002] Grapholitha molesta, also known as the oriental fruit borer or peach borer, is a small moth insect that is a serious threat to fruit trees. It mainly attacks fruits of the genera Prunus and Malus, including pears, peaches, and apples. The larvae of Grapholitha molesta burrow into the fruit to feed, causing the fruit quality to deteriorate, seriously affecting the yield and economic value.

[0003] With the development of technology, the use of image recognition technology to automatically analyze captured images or video materials to detect the presence of pests and the extent of their damage has become a new method for monitoring pear borer. However, how to establish a comprehensive monitoring system and predict possible pest situations based on monitoring results to provide data support for subsequent pest control work is an important direction worthy of research. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a pear borer monitoring and prediction method based on deep learning and BP neural network, aiming to solve the problems raised in the above background technology.

[0005] The embodiment of the present invention is implemented in this way: a pear borer monitoring and prediction method based on deep learning and BP neural network comprises the following steps:

[0006] Step 1: Place the target pest sex attractant automatic monitoring device in the area to be monitored to collect pest image data;

[0007] Step 2: Collect meteorological data of the area to be monitored, wherein the meteorological data includes temperature, rainfall, and relative humidity;

[0008] Step 3: Use the deep learning-based pear borer image monitoring model to automatically identify pear borer images in the pest image data and calculate the number of pear borer this week;

[0009] Step 4: Use the short-term prediction model of pear borer based on BP neural network to predict the number of pear borer in the next week and transmit it to the remote display device.

[0010] According to a further technical solution, step 1 comprises the following specific steps:

[0011] Step 1.1: Data collection;

[0012] By using the target pest sex attractant automatic monitoring device to collect photos of pear borer pests, the data set required for subsequent model training is used;

[0013] Step 1.2: Data processing;

[0014] Step 1.2.1: Data annotation;

[0015] Data annotation of the collected photos of pear borer pests;

[0016] Step 1.2.2: Dataset division;

[0017] The collected pictures with target pests were selected as the data set, and randomly divided into training set, validation set and test set in the ratio of 7:2:1. The training set was mainly used to train the model, the validation set was used to evaluate the model performance, and the test set was mainly used to test the final model effect.

[0018] Step 1.2.3: Data enhancement;

[0019] Used to increase model training data when there is little data.

[0020] As a further technical solution, in step 1.2.3, the data enhancement used includes hue enhancement, saturation enhancement, brightness enhancement, image translation, image scaling, image flipping and multi-image stitching.

[0021] A further technical solution is that in step 3, the method for constructing a pear borer image monitoring model based on deep learning is as follows:

[0022] Step a: Construct model evaluation indicators;

[0023] The accuracy rate P represents the accuracy of the model, that is, the proportion of true samples in the detected positive class samples. The calculation formula is as follows:

[0024]

[0025] Among them, TP is a true positive, indicating that the model successfully identifies the target pest; FP is a false positive, indicating that the model identifies a non-target pest as a target pest;

[0026] The recall rate R represents the ratio of the number of positive samples detected by the model to the number of all positive samples. The calculation formula is as follows:

[0027]

[0028] Among them, FN is true negative, which means that the model did not identify the target pest; in addition, TN is used to represent true negative, that is, the model correctly excludes non-target pests;

[0029] The average precision AP, that is, with the recall rate R as the X-axis and the accuracy rate P as the Y-axis, calculate the area under the PR curve:

[0030]

[0031] F1-Score is the harmonic mean of precision and recall, which is used to balance the relationship between the two. Its calculation formula is as follows:

[0032]

[0033] The higher the F1-Score value, the better the overall performance of the model in terms of accuracy and recall;

[0034] Step b: Test environment and parameter settings;

[0035] Based on the constructed test environment, the parameters are set as follows: training rounds are 500, the input image resolution is 640×640, and the stochastic gradient descent method is used for optimization, the momentum is 0.937, and the learning rate is 0.01.

[0036] Step c: model training;

[0037] Use the YOLOv8n model for training, use the Hardswish activation function to replace the original SiLU in the baseline model, and use WIoUv3 to replace the original CIoU loss function of the model;

[0038] At the same time, the CoT attention mechanism is added to the detection head of the baseline model.

[0039] In a further technical solution, in step 4, the short-term BP prediction model of pear borer is:

[0040] There are 7 input layer factors, namely: the average temperature of the previous week (x1, ℃), the precipitation of the previous week (x2, mm), the average temperature of this week (x3, ℃), the precipitation of this week (x4, mm), the number of insects in the previous two weeks (x5, insects / trap), the number of insects in the previous week (x6, insects / trap) and the number of insects in this week (x7, insects / trap);

[0041] The weight of the network connection from the input layer to the hidden layer is W1, and the node threshold is b1; the logarithmic S-type logsig is used as the transfer function tansig;

[0042] The hidden layer contains 11 nodes, the network connection weight from the hidden layer to the output layer is W2, the node threshold is b2, and the linear transfer function purelin is used as the transfer function;

[0043] Set the maximum number of training times to 100,000, the training target accuracy to 0.01, the learning rate to 0.1, the training function to the Levenberg-Marquardt (LM) algorithm, and the output layer to the population value of the pear borer in the next week.

[0044] The embodiment of the present invention provides a method for monitoring and predicting pear borer based on deep learning and BP neural network. The method can effectively monitor the pest situation of pear borer and predict the development of pests by combining deep learning algorithm and BP neural network. The monitoring and prediction method is simple to operate, the prediction results are basically consistent with the actual results, the data is reliable, and it can provide a basis for scientific research and pest control in agricultural planting. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is the effect diagram of labeling target pests using labelimg software;

[0046] Figure 2 Quantify label data in pest datasets and model the correlation between labels during target training;

[0047] Figure 3 This is the YOLOv8n model training graph (Grapholita molesta Busck in the figure represents the pear borer);

[0048] Figure 4 The detection results of the target pests by the optimized YOLOv8n_HSW3CO model;

[0049] Figure 5 This is the YOLOv8 structure diagram;

[0050] Figure 6 Compare the actual and predicted numbers of pear borer in apple orchards in Liquan in 2024;

[0051] Figure 7 Comparison between the actual and predicted numbers of pear borer in apple orchards in Yangling area in 2024. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0054] A method for monitoring and predicting pear borer based on deep learning and BP neural network provided by one embodiment of the present invention comprises the following steps:

[0055] Step 1: Place the target pest sex attractant automatic monitoring device (the sex attractant is the sex attractant of pear borer) in the area to be monitored to collect pest image data;

[0056] Step 2: Collect meteorological data of the area to be monitored, wherein the meteorological data includes temperature, rainfall, and relative humidity;

[0057] Step 3: Use the deep learning-based pear borer image monitoring model to automatically identify pear borer images in the pest image data and calculate the number of pear borer this week;

[0058] Step 4: Use the short-term prediction model of pear borer based on BP neural network to predict the number of pear borer in the next week and transmit it to the remote display device.

[0059] As a preferred embodiment of the present invention, step 1 includes the following specific steps:

[0060] Step 1.1: Data collection;

[0061] By using the target pest sex attractant automatic monitoring device to collect photos of pear borer pests, a total of 368 images with a pixel size of 3072×4096 were collected, which were used as the data set for subsequent model training.

[0062] Step 1.2: Data processing;

[0063] Step 1.2.1: Data annotation;

[0064] Data annotation is performed on the photos of pear borer pests to provide rich training samples for the learning model, so that it can learn the features, patterns and relationships in the images. This data annotation information can help the model better understand the image content, thereby achieving more accurate object detection tasks. The specific annotation effect is as follows: Figure 1 shown.

[0065] Step 1.2.2: Dataset division;

[0066] The 368 images of target pests collected were selected as the data set (as shown in Table 1), and randomly divided into training set, validation set and test set in a ratio of 7:2:1, including 258 training sets, 73 validation sets and 37 test sets. The training set is mainly used to train the model, the validation set is used to evaluate the model performance, and the test set is mainly used to test the final model effect.

[0067] Table 1. The division of training set, validation set and test set after the dataset is annotated

[0068]

[0069] Step 1.2.3: Data enhancement;

[0070] Data augmentation is an important means to increase the model training data when the model has a small amount of data. In this method, the data augmentation used includes hsv_h: 0.015 (hue), hsv_s: 0.7 (saturation), hsv_v: 0.4 (brightness), translate: 0.1 (image translation), scale: 0.5 (image scaling), fliplr: 0.5 (image flipping), mosaic: 1.0 (multi-image stitching).

[0071] As a preferred embodiment of the present invention, in step 3, the method for constructing a pear borer image monitoring model based on deep learning is as follows:

[0072] Step a: Construct model evaluation indicators;

[0073] The accuracy P (Precision) indicates the accuracy of the model, that is, the proportion of true samples in the detected positive class samples. The calculation formula is as follows:

[0074]

[0075] Among them, TP is a true positive, indicating that the model successfully identifies the target pest; FP is a false positive, indicating that the model identifies a non-target pest as a target pest;

[0076] Recall rate R (Recall) indicates the ratio of the number of positive samples detected by the model to the number of all positive samples. The calculation formula is as follows:

[0077]

[0078] Among them, FN is true negative, which means that the model did not identify the target pest; in addition, TN is used to represent true negative, that is, the model correctly excludes non-target pests;

[0079] Average precision AP (Average Precision), that is, with recall rate R as the X-axis and accuracy rate P as the Y-axis, calculate the area under the PR curve:

[0080]

[0081] F1-Score is the harmonic mean of accuracy and recall. It takes into account both precision and recall and is used to balance the relationship between the two. The calculation formula is as follows:

[0082]

[0083] The higher the F1-Score value, the better the overall performance of the model in terms of accuracy and recall.

[0084] Step b: Test environment and parameter settings;

[0085] The configuration of the test environment is shown in Table 2 below;

[0086] Table 2

[0087]

[0088] Parameter settings: the number of training rounds is 500, the input image resolution is 640×640, and the stochastic gradient descent (SGD) method is used for optimization, the momentum is 0.937, and the learning rate is 0.01.

[0089] Step c: model training;

[0090] The quantification of label data in the pest dataset and the modeling of the correlation between labels during target training are shown in the figure below. Figure 2 shown. Figure 2 In a, they are arranged in order from left to right: Grid 1: the amount of data in the training set, showing the number of samples contained in each category; Grid 2: the size and number of boxes, showing the size distribution and corresponding number of bounding boxes in the training set; Grid 3: the position of the center point relative to the entire image, describing the position distribution of the center point of the bounding box in the image; Grid 4: the height-to-width ratio of the target in the image relative to the entire image, reflecting the distribution of the height-to-width ratio of the target in the training set.

[0091] Figure 2 In b, each matrix cell represents the label used when training the model, and the color depth of the cell reflects the correlation between the corresponding labels. Dark cells indicate that the model has learned the correlation between the two labels more strongly. Light cells indicate weaker correlation. The colors on the diagonal represent the correlation of each label itself, which is usually the deepest because it is easier for the model to learn the relationship between the label and itself. It is intuitively possible to identify which labels have strong correlations, which is crucial for optimizing training and prediction results. If you find that the correlation between some labels is too strong, you may want to consider merging them to simplify the model and improve efficiency.

[0092] The relevant parameter changes of YOLOv8n model training are as follows Figure 3 As shown, Figure 3 a is the change process of various indicators generated by the model during the training process. During the training process, with the increase in the number of training times, various loss values ​​continue to decrease and tend to be stable, while precision, recall, and AP values ​​gradually increase with the continuous improvement of model performance and finally converge. Figure 3 b is the Precision-Confidence curve of the model. It can be seen that the confidence of the model increases with the increase of accuracy. Figure 3 c is the Recall-Confidence curve of the model. When the curve shows a higher recall rate at a higher confidence level, it means that the algorithm can accurately predict the existence of the target during target detection and can still maintain a high recall rate after filtering out the low-confidence prediction boxes. This reflects the good performance of the algorithm in the target detection task. Figure 3 d shows the PR curve of the model, with the horizontal axis representing the recall rate and the vertical axis representing the precision. The closer the curve is to the upper right corner, the more the model can guarantee both high precision and high recall rate when predicting, that is, the prediction result is more accurate. Figure 3 e is the F1-Confidence curve, which shows the change of F1 score value under different confidence levels.

[0093] In the embodiment of the present invention, the comparison of the parameters of different versions of YOLOv8 is shown in Table 3 below:

[0094] Table 3. Comparison of parameters of different versions of YOLOv8

[0095]

[0096] Among them, Depth multiple and Width multiple represent the depth and width of the network respectively, layers is the number of network layers, GFLOPs is the number of floating-point operations per second, which indicates the amount of computing required for the computer to run the model, and Params indicates the size of the model parameters.

[0097] The comparison of the training results of different models is shown in Table 4 below;

[0098] Table 4 Comparison of training results of different models

[0099]

[0100] From the training results of different models (Table 4), YOLOv8n has significantly better comprehensive performance on the constructed pest dataset. While its recall rate and F1 value remain at a high level, the training time is significantly shorter, which can save a lot of computer resources. At the same time, from Table 3, YOLOv8n, as the lightest version of the series, is more suitable for deployment on some mobile terminals with low computing performance, which further expands the application scope of the model. Therefore, from all aspects, YOLOv8n is selected as the baseline model for further optimization of subsequent models. The final training results of the baseline model are shown in Table 5 below.

[0101] Table 5 Final training results of the baseline model

[0102]

[0103] Using activation functions can introduce nonlinear factors to neurons, allowing neural networks to arbitrarily approximate any nonlinear function, making deep neural networks more expressive. By replacing the original SiLU in the baseline model with different activation functions, it was found that the Hardswish activation function had the most significant improvement in model performance, with its AP value and recall rate increased by 3.8% and 1% respectively (Table 6).

[0104] Table 6 Training effect of baseline model YOLOv8n under different activation functions

[0105]

[0106] The error between the detection box and the target box (i.e., the bounding box loss IoU) is a key indicator for judging the training effect of the model. By using different loss functions to obtain the best evaluation criteria required by the model under the self-built dataset, it is found that after replacing the original CIoU loss function of the model with WIoUv3, its AP value increased by 1.8%, and the detection speed also improved to a certain extent (Table 7).

[0107] Table 7 Training effect of baseline model YOLOv8n under different loss functions

[0108]

[0109] The addition of the attention mechanism allows the model to focus on some more critical target pest features and reduce the extraction of redundant features. After adding different attention mechanisms to the YOLOv8n backbone network, it was found that the overall performance of the model was not improved, but decreased compared with the original model. However, adding the CoT attention mechanism to the detection head of the baseline model can increase the AP value of the model by 0.1% (Table 8).

[0110] Table 8 Adding attention mechanism to the backbone network of the model

[0111]

[0112]

[0113] The ablation test results of the models under different modules are shown in Table 9 below;

[0114] Table 9 Ablation experiments of models under different modules

[0115]

[0116] As can be seen from Table 9, by combining different modules, it is found that the YOLOv8n_HSW3CO model that uses the Hardswish activation function, the WIoUv3 loss function, adds the CoT attention mechanism in the detection head of the baseline model, and data enhancement performs best on the self-built pear borer pest dataset. Compared with the original YOLOv8n model, its average precision and recall rate are improved by 7.4% and 1.0%, respectively.

[0117] The constructed model is used to detect pest images, and the results are as follows: Figure 4 As shown in the figure, from the detection results, the optimized model is better than the original YOLOv8n model in both detection accuracy (false detection) and recall rate (missed detection). Even when detecting pest images of different densities, the model can still show good performance, and the generalization and robustness of the model are significantly improved compared with the original model.

[0118] The final model structure is as follows Figure 5 shown.

[0119] As a preferred embodiment of the present invention, in step 4, for the short-term prediction model of pear borer using BP neural network, environmental conditions such as field temperature and precipitation and insect population base significantly affect biological characteristics such as insect development and reproduction, and are important factors for predicting insect population. The number of pear borer surveyed weekly was used as the dependent variable, and seven factors were used as independent variables to construct the model through Pearson correlation analysis (see Table 10).

[0120] Table 10 Factors of the short-term BP prediction model for the occurrence of pear borer

[0121]

[0122] In 2023, 30 surveys were conducted in Liquan and Yangling test sites, i.e., a total of 60 data. 10 data were randomly selected as the validation set for debugging the model and the test set for detecting the prediction effect of the model, and the remaining 50 data were used as the training set for training the model. Taking the occurrence of pear borer as the research object, using the relevant factor data in Table 10, the Matlab neural network algorithm program was written to establish the BP network prediction model

[0123] The weight of the network connection from the input layer to the hidden layer is W1, and the node threshold is b1; the logarithmic S-type logsig is used as the transfer function tansig. The weight of the network connection from the hidden layer to the output layer is W2, the node threshold is b2, and the linear transfer function purelin is used as the transfer function. The maximum number of training times is set to 100 000. In the BP neural network model, other parameters are adjusted by trial and error during the debugging process: different target accuracy (10-9-0.9), learning functions [standard BP function traingd, BP function traingdm with momentum, elastic BP function trainrp, BP function traindx with variable learning rate, conjugate gradient BP function traincg and LM function trainlm], learning rate (0.01-0.9) and number of hidden layer nodes are used for training, and the best parameters are selected to set the training model. Mean absolute error (MAE) and mean square error (MSE) are selected to evaluate the model performance during the debugging process.

[0124]

[0125] In the above formula, l is the number of hidden layer nodes, m is the number of input layer nodes, n is the number of output layer nodes, and α is a constant between [1-10]. i is the actual value of the insect level, is the predicted value of the BP network model, and n is the number of training samples. The smaller the MAE value, the smaller the error of the model; the smaller the MSE value, the better the stability of the model.

[0126] In the embodiment of the present invention, the results of the short-term BP model parameter screening for the occurrence of pear borer are shown in Table 11;

[0127] Table 11 Results of screening parameters of short-term BP model for pear borer occurrence

[0128]

[0129]

[0130] Through the model parameter screening results in Table 11, the short-term BP prediction model of pear borer can be summarized as follows: there are 7 input layer factors, namely: the average temperature of the previous week (x1, ℃), the weekly precipitation of the previous week (x2, mm), the average temperature of this week (x3, ℃), the precipitation of this week (x4, mm), the number of insects in the previous two weeks (x5, per trap), the number of insects in the previous week (x6, per trap) and the number of insects in this week (x7, per trap). The weight of the network connection from the input layer to the hidden layer is W1, and the node threshold is b1; the logarithmic S-type logsig is used as the transfer function tansig. The hidden layer contains 11 nodes. The network connection weight from the hidden layer to the output layer is W2, the node threshold is b2, and the linear transfer function purelin is used as the transfer function. Set the maximum number of training times to 100,000, the training target accuracy to 0.01, the learning rate to 0.1, the training function to the Levenberg-Marquardt (LM) algorithm, and the output layer to the population value of the pear borer in the next week.

[0131] As a preferred embodiment of the present invention, the above method is applied to the apple orchard in Liquan area for testing in 2024, and the prediction results are as follows: Figure 6 As shown. Simultaneously, it was applied to apple orchards in Yangling area for testing in 2024, and the predicted results are as follows Figure 7 As shown. Figure 6 and Figure 7 It can be seen that the predicted results are basically consistent with the actual results, the data is reliable, and can be applied to apple orchards to provide a basis for scientific research and pest and disease control.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring and predicting pear borer based on deep learning and BP neural network, characterized in that: The following steps are involved: Step 1: Place the target pest sex attractant automatic monitoring device in the area to be monitored to collect pest image data; Step 2: Collect meteorological data of the area to be monitored, wherein the meteorological data includes temperature, rainfall, and relative humidity; Step 3: Use the deep learning-based pear borer image monitoring model to automatically identify pear borer images in the pest image data and calculate the number of pear borer this week; Step 4: Use the short-term prediction model of pear borer based on BP neural network to predict the number of pear borer in the next week and transmit it to the remote display device.

2. The method for monitoring and predicting pear borer based on deep learning and BP neural network according to claim 1, characterized in that: The step 1 comprises the following specific steps: Step 1.1: Data collection; By using the target pest sex attractant automatic monitoring device to collect photos of pear borer pests, the data set required for subsequent model training is used; Step 1.2: Data processing; Step 1.2.1: Data annotation; Data annotation of the collected photos of pear borer pests; Step 1.2.2: Dataset division; The collected pictures with target pests were selected as the data set, and randomly divided into training set, validation set and test set in the ratio of 7:2:

1. The training set was mainly used to train the model, the validation set was used to evaluate the model performance, and the test set was mainly used to test the final model effect. Step 1.2.3: Data enhancement; Used to increase model training data when there is little data.

3. The method for monitoring and predicting pear borer based on deep learning and BP neural network according to claim 2, characterized in that: In step 1.2.3, the data enhancement used includes hue enhancement, saturation enhancement, brightness enhancement, image translation, image scaling, image flipping, and multi-image stitching.

4. The method for monitoring and predicting pear borer based on deep learning and BP neural network according to claim 2, characterized in that: In step 3, the method for constructing a pear borer image monitoring model based on deep learning is as follows: Step a: Construct model evaluation indicators; The accuracy rate P represents the accuracy of the model, that is, the proportion of true samples in the detected positive class samples. The calculation formula is as follows: Among them, TP is a true positive, indicating that the model successfully identifies the target pest; FP is a false positive, indicating that the model identifies a non-target pest as a target pest; The recall rate R represents the ratio of the number of positive samples detected by the model to the number of all positive samples. The calculation formula is as follows: Among them, FN is true negative, which means that the model did not identify the target pest; in addition, TN is used to represent true negative, that is, the model correctly excludes non-target pests; The average precision AP, that is, with the recall rate R as the X-axis and the accuracy rate P as the Y-axis, calculate the area under the PR curve: AP=∫0 1 P(R)d R ×100% F1-Score is the harmonic mean of precision and recall, which is used to balance the relationship between the two. Its calculation formula is as follows: The higher the F1-Score value, the better the overall performance of the model in terms of accuracy and recall; Step b: Test environment and parameter settings; Based on the built test environment, the parameters are set as follows: training rounds are 500, input image resolution is 640×640, all are optimized using stochastic gradient descent, momentum is 0.937, and learning rate is 0.01; Step c: model training; Use the YOLOv8n model for training, use the Hardswish activation function to replace the original SiLU in the baseline model, and use WIoUv3 to replace the original CIoU loss function of the model; At the same time, the CoT attention mechanism is added to the detection head of the baseline model.

5. The method for monitoring and predicting pear borer based on deep learning and BP neural network according to claim 4, characterized in that: In step 4, the short-term BP prediction model of pear borer is: There are 7 input layer factors, namely: the average temperature of the previous week x1, the precipitation of the previous week x2, the average temperature of this week x3, the precipitation of this week x4, the number of insects in the previous two weeks x5, the number of insects in the previous week x6 and the number of insects in this week x7; The weight of the network connection from the input layer to the hidden layer is W1, and the node threshold is b1; the logarithmic S-type logsig is used as the transfer function tansig; The hidden layer contains 11 nodes, the network connection weight from the hidden layer to the output layer is W2, the node threshold is b2, and the linear transfer function purelin is used as the transfer function; Set the maximum number of training times to 100,000, the training target accuracy to 0.01, the learning rate to 0.1, the training function to LM algorithm, and the output layer to the population value of pear borer in the next week.

Images