A method for monitoring the control of insect pests in tea plantations

Abstract

The present application relates to a kind of tea tree planting in the monitoring method of pest control, it includes the following steps: a, first adopt main network to carry out feature extraction to image sample, shallow feature of image sample is simultaneously selected and regression operation is carried out;B, the deep feature in each layer of main network is gradually stacked to shallow feature by the way of up-sampling and convolution layer combination in each layer;C, the deep feature of each layer of main network is fused with the shallow feature of each layer after corresponding stacking again, as the output layer of neural network, and complete the regression identification of tea green leafhopper;Meanwhile, according to the identification and regression operation result combination demarcated yellow board range, complete the deduplication processing of the same position tea green leafhopper repeated identification on different output layer, finally quantity is calculated.The monitoring method can efficiently complete the identification, counting of tea green leafhopper, and the judgment accuracy is high, time-saving and labor-saving, saves manual cost.

Description

[0001] This application is a divisional application of application number 202010967815.2, entitled "A method for identifying and counting tea green leafhoppers based on convolutional neural networks". Technical Field

[0002] This invention relates to the field of pest and disease control technology in tea gardens, specifically to a monitoring method for pest control in tea tree cultivation. Background Technology

[0003] The tea green leafhopper is one of the most widespread and damaging pests affecting tea yield and quality in my country's tea-growing regions. In the middle and lower reaches of the Yangtze River, losses due to it amount to about 10% to 15% in normal years for summer and autumn tea, but can reach over 50% in severely affected years. Controlling the tea green leafhopper is a primary task in tea garden pest and disease management; however, effective control requires monitoring the number and trends of the leafhopper in the tea garden. Currently, manual monitoring is mainly used, involving surveys conducted on sunny mornings before the dew dries or throughout the day on cloudy days, randomly counting the number of leafhoppers on 100 tender leaves (30 buds). However, this method demands a high level of expertise from technicians, including professional knowledge, judgment (the tea green leafhopper is agile and adept at climbing and jumping), and visual acuity (the leafhopper is relatively small). Furthermore, it requires extensive manual screening, identification, and counting, making it time-consuming and labor-intensive. Meanwhile, its accuracy is greatly affected by human factors, with large fluctuations and errors in each identification, making it impossible to accurately, quickly and effectively monitor the tea green leafhopper. Summary of the Invention

[0004] The purpose of this invention is to provide a monitoring method for pest control in tea plantation. This monitoring method can efficiently identify and count tea green leafhoppers with high accuracy, saving time, effort, and labor costs.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for monitoring pest control in tea plant cultivation, characterized by comprising the following steps:

[0007] a. First, the standard VGG16 network structure is adopted as the backbone network, and features are extracted from image samples containing the entire outline of the yellow board. At the same time, shallow features of the image samples are selected for regression calculation.

[0008] b. Deep features containing rich semantic information in each layer of the backbone network are layered onto shallow features through a combination of upsampling and convolutional layers to enrich the semantic information of the shallow features. In step b, upsampling is performed by directly assigning values ​​at corresponding positions and filling positions with zeros. That is, if data with a width of N and a height of M is sampled, and upsampled by A and B times respectively, data with a width of NA and a height of MB is obtained. The corresponding position of (N, M) in the original image is used as the fill position. All other points after excluding the corresponding positions of all points in the original image are filled positions.

[0009] c. The deep features of each layer of the backbone network are then fused with the corresponding shallow features superimposed on each layer to form the output layer of the neural network. Regression identification of the tea green leafhopper is performed on the output layer of the neural network. During the regression identification process, tea green leafhoppers at different locations are identified, and duplicate tea green leafhoppers at the same location on the same output layer are deduplicated. Simultaneously, based on the identification and regression results of the tea green leafhoppers (the regression results include the location information of the identified tea green leafhoppers), and combined with the defined yellow board area, duplicate identification of tea green leafhoppers at the same location on different output layers is deduplicated. Finally, the number of tea green leafhoppers on the yellow board is calculated.

[0010] The training method of the above network adopts a general iterative training approach for recognition models. The training set is obtained by manually annotating the target by selecting boxes on different data. To reduce the influence of the randomness of camera imaging pixels and shooting distance, template matching is used in step a to estimate the size, thereby selecting the best feature layer in the backbone network as the input layer of the shallow network. The template matching method is as follows:

[0011] First, draw a square frame with a side length of h (in cm) at each of the four corners and the center of the yellow board. Then, perform a template matching operation using rectangular frames in the original image, defining the four corners and the center of the yellow board as the template frames. Next, simultaneously calculate the number of pixels contained in the rectangular frames matched at the four corners and the center, which are n1, n2, n3, n4, and n0 respectively. Finally, estimate the number of pixels in the image of the tea green leafhopper in the original image, using the following formula:

[0012] Where i = 0, 1, 2, 3, 4;

[0013] In the formula, k is the body length of the tea green leafhopper in millimeters; 1 is in millimeters; h represents the side length of the square frame drawn in centimeters.

[0014] Divide the rectangular yellow board into four triangular regions using its four corners and center. Then calculate the average pixel size occupied by the tea green leafhopper in each region. The specific formula is as follows:

[0015] Where i = 1, 2, 3, 4; and x5 = x1;

[0016] Finally, based on the pixel size occupied by the tea green leafhopper in each region, the optimal feature layer for tea green leafhopper identification is selected, using the following formula:

[0017] Where i = 1, 2, 3, 4;

[0018] In the formula, 1 indicates rounding down; k indicates selecting the feature layer output by the kth block as the tea green leafhopper recognition layer for that region, i.e., the input layer of the shallow neural network.

[0019] Because the tea green leafhopper is short and has a very small display area in the entire image including the yellow board, directly using deep neural networks to obtain deeper semantic information, while improving recognition performance and accuracy, may result in deeper network output features that directly ignore the leafhopper's information, leading to detection failure. Conversely, using shallow neural networks for leafhopper recognition results in extremely low detection accuracy due to the lack of high-level semantic information. Furthermore, different shooting distances cause variations in the leafhopper's pixel size, leading to uncertainty in the selection of network feature layers: too deep a network layer results in the loss of leafhopper information; too shallow a network layer lacks sufficient semantic information and perception, further reducing detection accuracy. This invention extracts features of the tea green leafhopper containing yellow boards using a VGG16 network structure as the backbone network. It then selects shallow features for regression calculations and progressively superimposes the rich semantic information of the deep features from the backbone network onto the shallow features through upsampling and convolutional layers. This enriches the semantic information of the shallow features while avoiding the loss of image information of the tea green leafhopper. This allows for efficient and accurate identification and calculation of the tea green leafhopper, thus enabling its monitoring in a time-saving, labor-saving, and cost-effective manner.

[0020] To further optimize, step c determines whether it is a duplicate recognition at the same location by comparing the ratio of the intersection and union regions of the two recognition boxes at the same location with a threshold value.

[0021] For further optimization, the definition of the yellow board range in step c is determined based on the matching results of the rectangles at the four top corners of the yellow board or the image segmentation method (such as the large difference between the yellow board color and the background).

[0022] For further optimization, in step a, the image samples containing the entire outline of the yellow board are image frames obtained by either shooting with a high-definition camera or shooting with a mobile phone. They are preprocessed and trained to be converted into a format that can be directly read by the deep learning framework. When performing regression calculations in step a, the set anchor ratio is 1:k or k:1, where k is the body length of the tea green leafhopper.

[0023] The present invention has the following technical effects:

[0024] This invention provides a monitoring method for pest control in tea plantations. This method uses a VGG16 network structure as the backbone to extract features of the tea green leafhopper (containing yellow sticky traps), selects shallow features for regression calculation, and then layer-by-layer superimposes the rich semantic information of the deep features from the backbone network onto the shallow features through upsampling and convolutional layers. This enriches the semantic information of the shallow features, avoids the loss of image information of the tea green leafhopper, and thus completes the automatic identification, location regression, and automatic calculation of the tea green leafhopper. This monitoring method achieves an accuracy of 98% in identifying the tea green leafhopper even with a 1% false negative rate. Furthermore, it processes high-definition images, such as 1080P data, at millisecond-level speeds, enabling rapid and accurate identification and calculation of the tea green leafhopper. This method provides fast and accurate monitoring of the tea green leafhopper in tea gardens and can be widely applied to the control of tea green leafhoppers in tea trees and fruit trees. In addition, this monitoring method effectively avoids lengthy manual screening, identification, and counting, thus saving time, labor, and labor costs. Attached Figure Description

[0025] Appendix Figure 1 This is a block diagram of the backbone network structure in an embodiment of the present invention.

[0026] Appendix Figure 2 This is a general block diagram of the convolutional neural network in an embodiment of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments.

[0028] Example

[0029] As attached Figure 1 and 2 As shown, a monitoring method for pest control in tea plantation is characterized by the following steps:

[0030] a. First, a standard VGG16 network structure is adopted as the backbone network. Feature extraction is performed on image samples containing the entire outline of the yellow board. Simultaneously, shallow features of the image samples are selected for regression calculation. Specifically, image samples containing the entire outline of the yellow board are image frames obtained by either high-definition camera shooting or mobile phone shooting. They are preprocessed and trained to be converted into a format that can be directly read by the deep learning framework, and then feature extraction is performed. To reduce the influence of randomness in camera imaging pixels and shooting distance, template matching is used to estimate the size, thereby selecting the best feature layer in the backbone network as the input layer of the shallow network. The specific steps are as follows:

[0031] First, draw a square frame with a side length of h (e.g., 1cm) at each of the four corners and the center of the yellow board. Then, perform template matching operations using rectangular frames in the original image, defining the four corners and the center of the yellow board as the template frames. Next, simultaneously calculate the number of pixels contained in the rectangular frames matched at the four corners and the center, which are n1, n2, n3, n4, and n0 respectively. Finally, estimate the number of pixels in the image of the tea green leafhopper in the original image, using the following formula:

[0032] Where i = 0, 1, 2, 3, 4;

[0033] In the formula, k represents the body length of the tea green leafhopper in millimeters. For example, if a tea green leafhopper with a body length of 3.5 mm is used, then the above reaction formula is:

[0034] Where i = 0, 1, 2, 3, 4;

[0035] Divide the rectangular yellow board into four triangular regions using its four corners and center. Then calculate the average pixel size occupied by the tea green leafhopper in each region. The specific formula is as follows:

[0036] Where i = 1, 2, 3, 4; and x5 = x1;

[0037] Finally, based on the pixel size occupied by the tea green leafhopper in each region, the optimal feature layer for tea green leafhopper identification is selected, using the following formula:

[0038] Where i = 1, 2, 3, 4;

[0039] In the formula, 1 indicates rounding down; k indicates selecting the feature layer output by the kth block as the tea green leafhopper recognition layer for that region, i.e., the input layer of the shallow neural network.

[0040] When performing regression calculations, the ratio of anchors is set to 1:3.5 or 3.5:1.

[0041] b. Deep features containing rich semantic information in each layer of the backbone network are layered onto shallow features through upsampling and convolutional layers to enrich the semantic information of the shallow features (as shown in the appendix). Figure 2 (As shown); Upsampling uses the method of directly assigning values ​​at the corresponding positions and filling the positions with zeros.

[0042] c. The deep features of each layer of the backbone network are then fused with the corresponding shallow features obtained from the superposition of each layer, and used as the output layer of the neural network (e.g., in...). Figure 2 In this process, the output layers consist of features from layers 0, 1, 2, 3, 4, and 5, outputting the results of each layer and completing the regression identification of the tea green leafhopper. During regression identification, tea green leafhoppers at different locations are identified, and duplicate tea green leafhoppers at the same location on the same output layer are deduplicated (the method for determining whether a duplicate is at the same location is to compare the ratio of the intersection and union regions of two identification boxes at the same location with a threshold value). Simultaneously, based on the identification and regression results of the tea green leafhoppers (the regression results include the location information of the identified tea green leafhoppers), combined with the defined yellow board area (which can be defined by image segmentation, such as when the yellow board color differs greatly from the background; or by matching the rectangular boxes at the four corners of the yellow board), duplicate identification of tea green leafhoppers at the same location on different output layers is deduplicated. Finally, the number of tea green leafhoppers on the yellow board is calculated.

[0043] The training method for the above network can use a general iterative training approach for recognition models, and the training set is obtained by manually labeling the target by selecting boxes on different data.

[0044] This monitoring method can achieve an accuracy of 98% in identifying the tea green leafhopper even with a 1% false negative rate. Furthermore, it can process high-resolution images, such as 1080P data, at millisecond-level speeds, enabling rapid and accurate identification and calculation of the tea green leafhopper. It is widely applicable to the control of tea green leafhoppers in tea trees, fruit trees, and other fields.

Claims

1. A monitoring method for pest control in tea tree cultivation, characterized in that, Includes the following steps: a. First, the standard VGG16 network structure is adopted as the backbone network, and features are extracted from image samples containing the entire outline of the yellow board. At the same time, shallow features of the image samples are selected for regression calculation. b. Deep features containing rich semantic information in each layer of the backbone network are layered onto shallow features through upsampling and convolutional layers to enrich the semantic information of the shallow features. In step b, upsampling is done by directly assigning values ​​at corresponding positions and filling positions with zeros. That is, if data with a width of N and a height of M is sampled, and upsampled by A and B times respectively, data with a width of NA and a height of MB is obtained; where (NA+ MB+ () represents the corresponding position of (N, M) in the original image. All other points after excluding the corresponding positions of all points in the original image are fill positions. c. The deep features of each layer of the backbone network are then fused with the corresponding shallow features superimposed on each layer to form the output layer of the neural network. Regression and identification of tea green leafhoppers are performed on the output layer of the neural network. During the regression and identification process, tea green leafhoppers at different positions are identified, and duplicate tea green leafhoppers at the same position on the same output layer are deduplicated. At the same time, based on the identification and regression results of tea green leafhoppers, combined with the defined yellow board range, duplicate identification of tea green leafhoppers at the same position on different output layers is deduplicated. Finally, the number of tea green leafhoppers on the yellow board is calculated. The network is trained using a general iterative training method for recognition models, and the training set is obtained by manually labeling targets by selecting boxes on different datasets. In step a, template matching is used to estimate the size, thereby selecting the best feature layer in the backbone network as the input layer of the shallow network. The template matching method is as follows: First, draw a square frame with a side length of h (in cm) at each of the four corners and the center of the yellow board. Then, perform a template matching operation using rectangular frames in the original image, defining the four corners and the center of the yellow board as the template frames. Next, simultaneously calculate the number of pixels contained in the rectangular frames matched at the four corners and the center, which are n1, n2, n3, n4, and n0 respectively. Finally, estimate the number of pixels in the image of the tea green leafhopper in the original image, using the following formula: = ;in i=0,1,2,3,4； In the formula, k is the body length of the tea green leafhopper in millimeters; 1 is in millimeters; h represents the side length of the square frame drawn in centimeters. Divide the rectangular yellow board into four triangular regions using its four corners and center. Then calculate the average pixel size occupied by the tea green leafhopper in each region. The specific formula is as follows: ;in I=1,2,3,4； and ； Finally, based on the pixel size occupied by the tea green leafhopper in each region, the optimal feature layer for tea green leafhopper identification is selected, using the following formula: = ;in I=1,2,3,4； In the formula, 1 indicates rounding down; k indicates selecting the feature layer output by the kth block as the tea green leafhopper recognition layer for that region, i.e., the input layer of the shallow neural network.

2. The monitoring method for pest control in tea tree cultivation as described in claim 1, characterized in that: In step c, the determination of whether it is a duplicate recognition at the same location is made by comparing the ratio of the intersection and union regions of the two recognition boxes at the same location with a threshold value.

3. The monitoring method for pest control in tea tree cultivation as described in claim 1 or 2, characterized in that: In step c, the area defined by the yellow board is determined based on the matching results of the rectangles at the four vertices of the yellow board or the image segmentation method.

4. The monitoring method for pest control in tea tree cultivation as described in claim 3, characterized in that: In step a, the image samples containing the entire outline of the yellow board are image frames obtained by either shooting with a high-definition camera or shooting with a mobile phone. They are preprocessed and trained to be converted into a format that can be directly read by a deep learning framework. When performing regression calculations in step a, the anchor ratio is set to 1:k or k:1, where k is the body length of the tea green leafhopper.

Images