A method for building a tea green leafhopper damage symptom detection model and its application

Abstract

The present invention discloses a method for building a tea green leafhopper damage symptom detection model and its application. The building method comprises the following steps: Step 1: Acquire a tea bud image dataset with different degrees of damage caused by the tea green leafhopper; Step 2: Preprocess the acquired tea bud image datasets respectively; Step 3: Build a tea green leafhopper damage symptom detection model and perform model evaluation; Step 4: Train and test the model built in step 3 to obtain a deep learning model for tea green leafhopper damage symptom detection. The present invention also provides a method for tea green leafhopper damage symptom detection and a device for tea green leafhopper damage symptom detection. The present invention reveals the tea green leafhopper damage symptom classification potential based on RGB and hyperspectral imaging, and provides a necessary technical approach for the construction of smart tea gardens. At the same time, the present invention also provides a specific reference for accurate, efficient and non-destructive monitoring of other crop diseases and insect pests.

Description

Technical Field

[0001] The present invention relates to the fields of smart agricultural technology and tea green leafhopper detection technology, and in particular to a method for building a tea green leafhopper damage symptom detection model and its application. Background Art

[0002] The tea green leafhopper (Empoasca onukii Matsuda) is one of the most destructive pests in tea-producing areas of Asia. It can produce about 10-17 generations each year, usually living on the back of the second and third leaves below the buds. Due to its small size and high mobility, it is usually not easy to attract attention before it reproduces in large numbers. The tea green leafhopper uses its mouthparts to suck phloem sap from young buds and leaves, causing plant nutrients and water loss, resulting in symptoms such as tea trees withering and yellowing, wrinkled leaves, and premature leaf fall, which ultimately seriously affects tea yield and quality. Therefore, accurate, non-destructive, and efficient tea green leafhopper detection methods are crucial for pest control.

[0003] Traditional methods for diagnosing tea green leafhopper damage symptoms mostly use manual inspection and sampling methods, including field surveys of symptoms, incidence and severity. Although these traditional methods can obtain relatively accurate and reliable results, they are prone to subjective errors and are time-consuming and labor-intensive, limiting large-scale rapid surveys. Therefore, a true, efficient and non-destructive method is needed to detect and control the disease in a timely manner.

[0004] In recent years, the rapid development of image processing and computer vision has provided an effective means for monitoring plant leaf diseases and pests. In particular, RGB imaging has great advantages in cost, operation, and portability, and has been widely used in crop classification, growth monitoring, and yield estimation. Hyperspectral imaging is not limited to the information of the three bands of red, green, and blue, and has a large amount of information and high precision. Due to the integration of spectrum and image, hyperspectral imaging technology shows significant advantages in providing objective, accurate, non-destructive, and intuitive plant pest diagnosis results. However, previous studies have only used unit sensors and have not fully utilized the complementarity and fusion of multiple sensor data. Therefore, the value of multiple remote sensing data deserves further exploration. Summary of the invention

[0005] The present invention collects tea buds with different degrees of damage caused by tea leafhoppers, and obtains RGB and hyperspectral images. By rotating the RGB image data by 90°, 180°, 270°, vertically flipping and horizontally flipping, the sample data becomes 6 times the original sample size, and then the wavelet transform enhancement technology is used for the sample. For the acquired hyperspectral images, the spectral data are preprocessed by using multivariate scatter correction (MSC), SG smoothing (Savitzky-Golay), first-order differential (1D) and second-order differential (2D) algorithms, and the characteristic bands of the spectrum are filtered using UVE, CARS and SPA algorithms. A classification model of the damage degree of tea leafhoppers is established for RGB image data using deep learning algorithms such as ResNet18, VGG16 and AlexNet. A classification model of the damage degree of tea leafhoppers is established for hyperspectral image data using SVM and LSTM algorithms, and the model is evaluated using four indicators. This work demonstrates the potential of tea leafhopper damage degree classification based on RGB and hyperspectral imaging, which can provide an accurate, non-destructive and efficient method for monitoring the occurrence and damage of the insect. The overall framework of the present invention is as follows Figure 1 shown.

[0006] The specific technical solutions are as follows:

[0007] In a first aspect, the present invention provides a method for building a tea green leafhopper damage symptom detection model, comprising the following steps:

[0008] Step 1: Obtain a dataset of tea bud images with different degrees of damage caused by tea leafhoppers;

[0009] Step 2: Preprocess the acquired tea bud image sets respectively;

[0010] Step 3: Build a tea green leafhopper damage symptom detection model and conduct model evaluation;

[0011] Step 4: Train and test the model built in step 3 to obtain a deep learning model for detecting damage symptoms of tea green leafhoppers.

[0012] Furthermore, the tea bud image set in step 1 includes an RGB image set and / or a hyperspectral image set;

[0013] The RGB image set is obtained by taking a number of images of tea buds damaged to different degrees by green leafhoppers in a tea garden field environment, and the hyperspectral image set is obtained by picking tea buds damaged to different degrees by green leafhoppers in a tea garden field and using a built-in push-scan in a hyperspectral imaging system.

[0014] Furthermore, in step 2, the obtained tea bud image set is preprocessed including data amplification and hyperspectral data preprocessing;

[0015] The RGB data preprocessing includes data amplification and data enhancement processing;

[0016] The hyperspectral data preprocessing includes extracting spectral reflectance, spectral preprocessing and characteristic band screening; the data amplification includes rotating the RGB image samples obtained in step 1 by 90°, 180°, 270°, vertically flipping and horizontally flipping, so that the sample data becomes 6 times the original sample volume; dividing the sample volume of the RGB image into a training set, a test set and a validation set in a ratio of 3:1:1;

[0017] The data enhancement processing: the RGB image set is enhanced by 2D-DWT; in the process of 2D-DWT decomposing the image, the LL component can be circulated multiple times before meeting the requirements. For the discrete wavelet transform of the image f(x, y) with a size of M×N, the LL component is only circulated once, and the formula is as follows:

[0018]

[0019] In the formula, j 0 is an arbitrary starting scale; is at scale j 0 The approximate coefficient at ; represents the scaling function;

[0020] Extracting spectral reflectance: The hyperspectral image data set collected in step 1 is subjected to lens correction and reflection correction using the analysis tool of SpecVIEW; the hyperspectral image in RAW format is opened using ENVI 5.3 software, the entire area of ​​the sample is selected as the region of interest (ROI), and the average spectral reflectance of the region of interest is calculated as the spectral data of the sample;

[0021] The spectral preprocessing: the hyperspectral image is preprocessed by using MSC, SG smoothing, first-order derivative, and second-order derivative;

[0022] The spectral characteristic band screening: the hyperspectral image adopts UVE, CARS and SPA algorithms to select representative bands as "characteristic bands" in the full-band spectral data.

[0023] Furthermore, a tea green leafhopper damage symptom detection model is built as described in step 3, and RGB data modeling is carried out using ResNet18, VGG16, AlexNet, WT-ResNet18, WT-VGG16 and WT-AlexNet deep learning algorithms; hyperspectral data modeling is carried out using SVM algorithm and LSTM algorithm.

[0024] Furthermore, in the modeling algorithm described in step 3, the RGB image data is modeled using the ResNet18, VGG16, and AlexNet algorithms, and the hyperspectral image data is modeled using the SVM and LSTM algorithms;

[0025] Furthermore, the model evaluation described in step 3 is evaluated using Accuracy, Precision, Recall, and F1-score respectively.

[0026] Specifically, the model evaluation described in step 3 can also use the confusion matrix method.

[0027] In the second aspect, based on the same invention, the present invention proposes the application of the tea green leafhopper damage symptom detection model obtained by any of the above-mentioned construction methods in tea trees.

[0028] In a third aspect, the present invention provides a method for detecting damage symptoms of tea green leafhoppers, comprising: obtaining an image of tea buds in a natural environment; inputting the image of tea buds into a tea green leafhopper damage symptom detection model obtained by building a tea green leafhopper damage symptom detection model according to any one of the methods described in the first aspect, and obtaining the degree of tea green leafhopper damage symptoms output by the model.

[0029] In a fourth aspect, the present invention further provides a device for detecting the symptoms of damage caused by the tea green leafhopper, the device comprising at least one processor and at least one memory; the memory stores program instructions of a tea green leafhopper damage symptom detection model obtained by any one of the construction methods described in the first aspect; when the processor executes the program instructions, it can detect the degree of damage symptoms caused by the tea green leafhopper on tea buds.

[0030] The present invention has the following beneficial effects:

[0031] The present invention provides a detection model for grading the degree of damage of tea green leafhoppers based on RGB and hyperspectral imaging technology. RGB and hyperspectral images of tea buds with different degrees of damage caused by tea green leafhoppers are collected; ResNet18, VGG16, AlexNet, WT-ResNet18, WT-VGG16 and WT-AlexNet deep learning algorithms are used to establish a grading model of the degree of damage of tea green leafhoppers for RGB image data; MSC, SG, 1D and 2D methods are used to perform spectral preprocessing on hyperspectral image data; UVE, CARS and SPA methods are used to screen spectral feature bands; SVM and LSTM algorithms are used to establish a grading model of the severity of damage symptoms of tea green leafhoppers for hyperspectral image data.

[0032] The tea green leafhopper damage symptom detection model of the present invention can accurately, efficiently and non-destructively monitor the occurrence degree of tea green leafhopper. The accuracy rates of 14 classification models ResNet18, VGG16, AlexNet, WT-ResNet18, WT-VGG16, WT-AlexNet, UVE-SVM, CARS-SVM, SPA-SVM, NONE-SVM, UVE-LSTM, CARS-LSTM, SPA-LSTM and NONE-LSTM are 65%, 70%, 62%, 78%, 80%, 69%, 69%, 86%, 89%, 74%, 82%, 94%, 96%, and 90% respectively. Among them, the SPA-LSTM model is better than other classification models and is more suitable for monitoring the damage symptoms of tea green leafhopper.

[0033] In the RGB imaging technology classification model, the neural network enhanced by wavelet transform outperformed other networks, and WT-VGG16 performed best. In the hyperspectral imaging technology classification model, the SPA method performed better than the CARS and UVE band screening methods. The deep learning algorithm performed better than the machine learning algorithm. Hyperspectral imaging technology performed better than RGB imaging technology.

[0034] The SPA-LSTM model achieved excellent results in monitoring the damage symptoms of tea green leafhoppers, with an accuracy rate of 96%. The present invention reveals the potential of grading the damage symptoms of tea green leafhoppers based on RGB and hyperspectral imaging, providing the necessary technical approach for the construction of smart tea gardens. At the same time, the present invention also provides a specific reference for the accurate, efficient and non-destructive monitoring of other crop diseases and insect pests. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0036] Figure 1 : Overall framework diagram of the model building method of the present invention.

[0037] Figure 2 : Location map for carrying out the experiment of the present invention.

[0038] Figure 3:A: original image, where (a) is the first-level symptom-free tea bud, (b) is the second-level mild symptom tea bud, and (c) is the third-level severe symptom tea bud; B: 2D-DWT principle diagram, C: 2D-DWT treatment of tea buds with different insect pest degrees, where (a) is the first-level symptom-free tea bud, (b) is the second-level mild symptom tea bud, and (c) is the third-level severe symptom tea bud.

[0039] Figure 4 : Diagram of the hyperspectral imaging device used in the present invention, wherein, A: real-life diagram of the hyperspectral device, B: model diagram of the hyperspectral device, (a) cube dark box; (b) hyperspectral camera; (c) 200W halogen light source; (d) tea bud sample; (e) computer.

[0040] Figure 5 : The network structure of ResNet18, VGG16 and AlexNet of the present invention, wherein A: ResNet18, B: VGG16, C: AlexNet.

[0041] Figure 6 : Original data and spectrum after preprocessing by the present invention, (a) original spectrum of tea sample; (b) spectrum after preprocessing by SNV+2D+SG algorithm scheme.

[0042] Figure 7 : Distribution of characteristic bands of the present invention; (a) UVE, (b) CARS, (c) SPA.

[0043] Figure 8 : Evaluation results of different disease severity classifications using different network models of the present invention: (a) standard; (b) mild; (c) severe.

[0044] Fig. 9 : Confusion matrix of the network model of the present invention; wherein, (a) ResNet; (b) VGGNet; (c) AlexNet; (d) WT-ResNet; (e) WT-VGGNet; (f) WT-AlexNet; (g) UVE-SVM; (h) CARS-SVM; (i) SPA-SVM; (j) NONE-SVM; (k) UVE-LSTM; (l) CARS-LSTM; (m) SPA-LSTM; (n) NONE-LSTM. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention. In the present invention, unless otherwise specified, the equipment and raw materials used can be purchased from the market or are commonly used in the art. The methods in the following embodiments are conventional methods in the art unless otherwise specified.

[0046] Test area:

[0047] The experiment was conducted in Chunxi Tea Garden, Fei County, Linyi City, Shandong Province, China (117°77′E, 35°22′N). The region has a warm temperate continental monsoon climate, with dry spring, hot and rainy summer, clear and crisp autumn, and cold and dry winter. Located in a high-latitude tea area, the temperature difference between day and night is large. The annual average temperature is 13°C, the annual average sunshine time is about 2500 hours, and the annual average precipitation is about 800 mm. The experimental tea garden covers an area of ​​more than 300 acres. The soil bulk density is 1.50g / cm3, the organic matter content is 1.65%, and the pH is 5.8. The tea varieties planted in this tea garden include Jinxuan, Zhongcha 108, Qimen, Huangshan, etc. The location map of the experimental area is as follows Figure 2 shown.

[0048] Sample preparation:

[0049] This study was conducted in the season (autumn) when tea trees in multiple plots of Chunxi Tea Garden showed obvious symptoms of damage by tea green leafhoppers. Since the tea green leafhoppers damage tea buds, about 400 test samples (one bud and one leaf, one bud and two leaves) were randomly selected. The collected tea buds were divided into three levels according to the severity of the damage (level one: no symptoms, level two: mild symptoms, and level three: severe symptoms). Figure 3 (A) shows tea buds with (a) no symptoms, (b) mild symptoms, and (c) severe symptoms. RGB and hyperspectral data of tea buds were collected immediately after picking.

[0050] Test environment:

[0051] The conditions for processing data in the present invention are as follows:

[0052] Hardware processor: Inter Xeon CPU E5-2640 V4@2.4GHZ 2.40GHZ (two processors); RAM: 128GB; Software environment: CUDA Toolkit 10.1; CUDN V7.6.0; MATLAB 2020; Python 3.8; PytorchGPU 1.6.0; Operating system: Windows 10.

[0053] Example 1: RGB data collection and preprocessing

[0054] 1.1 RGB data collection

[0055] Under natural light conditions, RGB image data were collected using a digital camera (EOS·6D, Canon Co. Ltd, Beijing, China) with an image resolution of (5184 pixels × 3456 pixels).

[0056] Among them, 357, 430 and 435 photos were taken for level 1 asymptomatic, level 2 mild symptomatic and level 3 severe symptomatic samples, respectively, for a total of 1,222 photos. The images are stored in JPEG format, and the shooting angle is perpendicular to the ground.

[0057] 1.2 Two-dimensional discrete wavelet transform of images with different hazard levels

[0058] Two-dimensional Discrete Wavelet Transform (2D-DWT) can meet the requirements of time-frequency signal analysis, can focus on any details of the signal, and has the characteristics of multi-resolution analysis. In order to better restore and express the image of tea green leafhoppers harming tea buds, the present invention uses 2D-DWT to enhance the RGB image, effectively improve the model accuracy, and the formed model has a strong generalization ability.

[0059] In order to reduce or eliminate the correlation between different features of the extracted image, 2D-DWT is used to separate the main information and detail information of the signal.

[0060] RGB images of tea buds with different degrees of damage are shown in Figure 2. Figure 3 As shown in (A), Figure 3 In (A), (a) is the first-level symptom-free tea bud, (b) is the second-level mild symptom tea bud, and (c) is the third-level severe symptom tea bud.

[0061] 2D-DWT passes the image data through row filters and column filters to obtain four components of high-frequency and low-frequency signals, such as Figure 3 (B) As shown in FIG. LL component represents the low-frequency information of the image, HL component represents the high-frequency information of the image in the horizontal direction, LH component represents the high-frequency information of the image in the vertical direction, and HH component represents the high-frequency information on the diagonal line of the image.

[0062] The images of tea buds with different damage degrees after 2D-DWT processing are as follows: Figure 3 (C) shown. Figure 3 (C) (a) shows the image of the first-level symptom-free tea bud after 2D-DWT processing; Figure 3(C) (b) shows the tea bud image with secondary mild symptoms after 2D-DWT processing; Figure 3 (C) (c) shows the image of tea buds with three levels of severe symptoms after 2D-DWT processing.

[0063] In the process of 2D-DWT decomposing an image, the LL component can be circulated multiple times before reaching the requirement. In the present invention, for the discrete wavelet transform of an image f(x, y) of size M×N, the LL component is circulated only once, and the formula is as follows:

[0064]

[0065] In the formula, j 0 is an arbitrary starting scale; is at scale j 0 The approximate coefficient at ; represents the scaling function.

[0066] Example 2: Hyperspectral data acquisition and preprocessing

[0067] 2.1 Hyperspectral equipment

[0068] The hyperspectral imaging system used in the present invention is mainly composed of a computer, a dark box, a light source system and an imaging system; the light source system is composed of four 200W halogen light sources (hsia-ls-t-200W, China); the imaging system is composed of a hyperspectral camera (GaiaField-Pro-V10, Jiangsu Dualix Spectral Image Technology Co. Ltd, China) and a lens. The wavelength of the image captured by the hyperspectral camera is 397.899994nm-1001.400024nm, and the reflectivity of 176 bands can be measured. The spectral resolution is 3.5nm, and the pixel of the hyperspectral camera is 1936×1456 (space×Spectral). The imaging mode is built-in push-scan, the frame rate is 7s / cube, the data interface is USB2.0, and the weight is 3kg. Hyperspectral imaging equipment such as Figure 4 shown.

[0069] 2.2 Hyperspectral data acquisition

[0070] The specific steps to obtain hyperspectral images are as follows:

[0071] First, turn on the hyperspectral camera and halogen light source system in a dark box for preheating until the light source is stable; set the camera exposure time to less than 19.6ms; perform black and white calibration to compensate for the uneven spatial light intensity distribution and dark current in the camera; scan a white board to obtain a full-white calibration image, and cover the camera lens with a lens cap to obtain a full-black calibration image.

[0072] Secondly, the prepared tea bud samples were placed in a cubic dark box. In order to ensure that they were not affected by other reflected light sources, black velvet cloth was placed under the tea bud samples to collect hyperspectral images. The same operation was repeated to obtain hyperspectral images of all tea bud samples.

[0073] Finally, the data was corrected for lens and reflection using the analysis tool of the data preprocessing software SpecVIEW (Jiangsu Dualix Spectral Image Technology Co. Ltd, China); the hyperspectral image in RAW format was opened using ENVI 5.3 (Research System Inc, Boulder, CO, USA) software, the entire area of ​​the sample was selected as the region of interest (ROI), and the average spectral reflectance of the region of interest was calculated as the spectral data of the sample. Finally, 983 spectral data were collected as the initial data of the tea bud sample.

[0074] 2.3 Hyperspectral data preprocessing

[0075] Due to the influence of hyperspectral acquisition instruments or environmental factors, spectral data are easily interfered by useless signals. Therefore, multivariate scatter correction (MSC), SG smoothing (Savitzky-Golay), first-order derivative (1D) and second-order derivative (2D) are used to preprocess the extracted spectral data.

[0076] The MSC algorithm can eliminate artifacts or defect spectra in the data matrix. The processed spectral data can effectively eliminate scattering effects and improve the quality of spectral information. Savitzky-Golay (SG) obtains the best estimate of the smooth point by fitting or averaging the data within the spectral data points, effectively reducing the random noise of the average reflectance spectrum. The first-order derivative (1D) and second-order derivative (2D) can eliminate baseline drift and spectral line overlap to obtain clearer spectral profile changes.

[0077] The relevant formula is as follows:

[0078] Multiple scattering correction:

[0079]

[0080]

[0081] Among them, X is the original spectral matrix of the sample, X i , k i , b i, X i(msc) are the spectral value of the ith sample, the average of all spectral data, the baseline shift, the baseline shift, and the spectrum of the ith sample after MSC correction.

[0082] Savitzky-Golay (SG)

[0083] Where X i and are the spectral data before and after SG smoothing, W j is the weight factor obtained by smoothing a window with a width of 2R+1.

[0084] First derivative:

[0085] Second derivative:

[0086] Where y is the spectral absorbance, λ is the wavelength, and y i is the spectrum of the ith sample, and Δλ is the wavelength interval.

[0087] 2.4 Hyperspectral data characteristic band screening

[0088] The present invention obtains spectral data of 176 bands within the spectral range of 397-1001 nm. Too many band variables will increase the amount of data calculation and affect the prediction accuracy and stability of the model.

[0089] In order to improve the efficiency of later modeling, UVE, CARS and SPA algorithms are used to select representative bands as "feature bands" in the full-band spectral data and compare them with the full-band (NONE) data. The Uninformative variable elimination (UVE) algorithm can remove wavelength variables with low coefficient modeling efficiency, reduce the complexity of spectral data, optimize model variables, and improve the prediction ability of the model. Competitive adaptive reweighted sampling (CARS) is a variable screening method based on the "survival of the fittest" principle in Darwin's theory of evolution. The wavelength points with larger absolute value weights of the regression coefficients are retained, and the wavelength points with smaller weights are removed to effectively find the best spectral combination. Successive projections algorithm (SPA) can extract the variables with the smallest collinearity in the full band, thereby eliminating redundant information in the original spectral matrix.

[0090] The basic parameters of the UVE, CARS, and SPA algorithms used in this study are shown in Table 1.

[0091] Table 1 Parameters of the characteristic band screening algorithm.

[0092]

[0093] Example 3: RGB data modeling

[0094] In the present invention, deep learning algorithms such as ResNet18, VGG16, and AlexNet are used to model RGB image data.

[0095] 3.1ResNet18

[0096] The basic architecture of the ResNet 18 network is ResNet, which contains 18 convolutional layers. However, the network depth in this invention refers to the weight layer of the network, including convolutional layers and fully connected layers, excluding pooling layers and BN layers. ResNet18 is a classic and effective deep convolutional neural network model with good feature extraction and classification capabilities. It can be applied to computer vision tasks such as image classification and target detection. The specific structure is as follows Figure 5 (a) shown.

[0097] 3.2 VGG16

[0098] VGG16 is a convolutional neural network (CNN) pre-trained network that completes the target task through the model weights obtained by pre-training on a large data set. The VGG16 network has a total of 16 layers, of which 13 are convolutional layers, 5 are maximum pooling layers, and 3 are fully connected layers. The VGG model has 7 different hierarchical structures A-E, which are basically composed of 3×3 convolutional layers and 2×2 maximum pooling layers in order. In the feature extraction part, the VGG network uses multiple 3×3 small convolution kernels to replace the previous single larger convolution kernel. This not only reduces the number of parameters while ensuring the receptive field, but also increases the depth of the network and enhances the ability of feature learning. In addition, all hidden layers use the ReLU function, which greatly improves the nonlinear fitting ability of the model. The VGG16 model is widely used in image classification and positioning tasks due to its tiny filters. The specific structure is as follows Figure 5 (b) as shown.

[0099] 3.3AlexNet

[0100] The AlexNet network has 5 convolutional layers, 3 pooling layers, and 3 fully connected layers (including the output layer). The AlexNet model used ReLU as the activation function of CNN for the first time, which greatly reduced the amount of calculation and accelerated the convergence speed, much faster than the equivalent tanh. Through the overlapping pooling operation, the use of Dropout and data augmentation during training, the overfitting problem in the model training process was avoided, and the LRN layer was proposed to enhance the generalization ability of the model. The specific structure is as follows Figure 5 (c) as shown.

[0101] Example 4: Hyperspectral Data Modeling

[0102] In the present invention, the effects of SVM algorithm and LSTM algorithm on modeling hyperspectral image data are explored and compared.

[0103] 4.1Support Vector Machine(SVM)

[0104] The core idea of ​​the SVM algorithm is linearization, that is, mapping the sample space from low-dimensional nonlinearity to high-dimensional space, and combining the linear learning machine to solve the classification in the feature space. The specific parameters are shown in Table 2.

[0105] 4.2Long Short-Term Memory(LSTM)

[0106] LSTM is a type of RNN that makes up for the defect that ordinary neural networks cannot rely on long-term information. A single LSTM neuron mainly consists of four layers: Forget Gate, Input Gate, Output Gate, and cell state. LSTM uses gates to control the forgetting or adding of cell states, and uses a special memory cell (cell state) to store long-term states to achieve long-term memory learning. The validation set is a data set used to evaluate model performance and adjust hyperparameters. Select the model parameters with the highest single-round accuracy and load the model parameters into the training set. The parameters for the best model effect are shown in Table 2:

[0107] Table 2 Main parameters of SVM and LSTM models.

[0108]

[0109] Example 5: Model performance indicators

[0110] In order to accurately calculate the classification effect of the above model and observe the performance of the model in each category, four indicators were used, namely Accuracy, Precision, Recall, and F1-score. The Accuracy index refers to the proportion of correctly identified samples, which can evaluate the overall ability of the tea green leafhopper pest classification model; Precision refers to the ratio of the number of correctly identified tea green leafhopper damage samples to the total number of identified tea green leafhopper damage samples; Recall refers to the ratio of the number of correctly identified tea green leafhopper damage samples to the total number of tea green leafhopper damage samples; and F1-score attempts to be an evaluation indicator for the coordination of accuracy and recall.

[0111] The relevant calculation formulas for the evaluation indicators are shown below.

[0112]

[0113]

[0114]

[0115]

[0116] "TP" (True Positive) indicates the number of samples correctly identified as tea green leafhoppers. "FN" (False Negative) indicates the number of samples not identified as tea green leafhoppers. "FP" (False Positive) indicates the number of samples incorrectly identified as tea green leafhoppers. "TN" (True Negative) indicates the number of samples correctly identified as healthy samples.

[0117] Experimental results and data analysis

[0118] 1 Data Preprocessing

[0119] 1.1 Preprocessing of RGB data

[0120] The RGB image samples taken in Example 1 are rotated 90°, 180°, 270°, vertically flipped, and horizontally flipped to make the sample data 6 times the original sample size. The sample size of the RGB image is divided into a training set, a test set, and a validation set in a ratio of 3:1:1.

[0121] 1.2 Hyperspectral data preprocessing

[0122] From the perspective of the original average spectral reflectance, the spectral reflectance curves of different hazard levels are different, but the change trends are relatively similar. The spectral reflectance fluctuates and rises in the visible light range of 397nm-673nm, and the spectral reflectance value is relatively small. The spectral reflectance rises rapidly in the range of 673-811nm, and tends to stabilize and maintain a high spectral reflectance value in the range of 811-1018nm. In addition, there are significant differences between the spectra of the three hazard levels, which can be clearly distinguished.

[0123] In addition to sample-related information, the raw spectral data also contains information such as baseline drift and noise, which will reduce the robustness and accuracy of the prediction or classification model. In order to reduce the interference of noise and redundant signals, the MSC, SG, 1D and 2D combined method is used to preprocess the hyperspectral data. The spectral curve after preprocessing is shown in Figure 2. Figure 6 As shown in (b, d).

[0124] The results show that compared with the original spectrum, the spectral curves after MSC, SG, 1D and 2D combined preprocessing are more stable, the absorption peaks and reflection valleys of the spectrum can be clearly observed, and the resolution and sensitivity of the spectrum are greatly improved. Similarly, the spectral characteristics after preprocessing show that the overall trend of the spectral reflectance of the three hazard symptom severity is similar. It increases exponentially in the wavelength range of 673-742nm, decreases rapidly in the wavelength range of 742-811nm, and there are obvious absorption peaks at 535nm and 742nm. In addition, in the range of 397-1018nm, the spectral reflectance curves of different hazard symptom severity are significantly different. The more severe the hazard, the greater the spectral reflectance. The existence of this difference is the basis for the subsequent use of hyperspectral to establish a hazard symptom classification model.

[0125] 2. Feature band selection of spectral data

[0126] The screened spectral data include 176 bands. Considering the problem of band collinearity data redundancy, UVE, CARS and SPA algorithms are used to select characteristic bands, which is conducive to simplifying the complexity of the model and improving the efficiency and reliability of the model.

[0127] The number and distribution of characteristic bands are as follows Figure 7 , as shown in Table 3. The results show that among the characteristic band screening methods, UVE screened the largest number of characteristic bands, which is 85, and CARS screened the least number of characteristic bands, which is 12.

[0128] In general, the variable selection capabilities of CARS and SPA are better than those of UVE, and the feature bands filtered by the UVE algorithm are richer than those of CARS and SPA.

[0129] Table 3 Band screening results

[0130]

[0131] 3 Overall accuracy of different network models for different pest classifications

[0132] Accuracy is widely used in the evaluation of model accuracy. By statistically analyzing the accuracy of different models under RGB imaging technology and hyperspectral imaging technology, it can be used to test the performance of the two imaging technologies under different models.

[0133] Both imaging technologies were tested with various models under the same test samples and test environment. As shown in Table 4, overall, the accuracy of the eight models of hyperspectral imaging technology (69%-96%) is better than the accuracy of the six models of RGB imaging technology (62%-80%), and the accuracy of the wavelet transform enhanced RGB imaging technology model (69%-80%) is better than the accuracy of the ordinary RGB imaging technology model (62%-70%). And the SPA-LSTM model has the highest accuracy (96%).

[0134] Under RGB imaging technology, the model accuracy of ResNet18, VGG16, AlexNet, WT-ResNet18, WT-VGG16 and WT-AlexNet are 65%, 70%, 62%, 78%, 80% and 69% respectively, and the WT-VGGNet16 model has the highest accuracy (80%). Moreover, the accuracy of the model after wavelet transform enhancement technology is significantly improved by 7-13%, but it is lower than the accuracy of hyperspectral imaging technology as a whole, which shows that hyperspectral imaging technology is significantly better than RGB imaging technology.

[0135] The accuracy of each model under hyperspectral imaging technology was further compared. The model accuracy of UVE-SVM, CARS-SVM, SPA-SVM, NONE-SVM, UVE-LSTM, CARS-LSTM, SPA-LSTM and NONE-LSTM were 69%, 86%, 89%, 74%, 82%, 94%, 96% and 90% respectively. Among them, the model accuracy of SPA-LSTM was the highest (96%). The results show that under the same feature band screening algorithm (UVE, CARS and SPA), the model accuracy of LSTM is significantly higher than that of SVM (7-16%), indicating that the deep learning method is significantly better than the traditional machine learning method. The classification accuracy of different feature band screening methods under the same modeling algorithm (SVM and LSTM) is SPA>CARS>NONE>UVE method, indicating that the SPA method is better than the CARS method, UVE method and full band in screening feature bands.

[0136] Table 4 Accuracy of different classification models under two imaging technologies.

[0137]

[0138]

[0139] 4. Comparison of performance of different network models for different disease classifications

[0140] In order to intuitively compare the classification performance of RGB imaging and hyperspectral imaging technology under different models, we used three evaluation indicators, Recall, Precision and F1-score, to evaluate the 14 models ( Figure 8 ).

[0141] The results show that, overall, the indicators of the eight models using hyperspectral image technology are significantly better than those of the six models using RGB image technology, and the SPA-LSTM model has the best comprehensive performance in grading the severity of damage caused by tea green leafhoppers.

[0142] Among the 6 models under RGB image technology, compared with the models without wavelet transform enhancement, the processed models have obvious improvements in various indicators. Among them, the WT-VGG16 model with an accuracy of 80% performs well under RGB image technology. Among the 8 models under hyperspectral image technology, compared with the SVM model, the LSTM model has obvious advantages in various indicators. Among them, the SPA-LSTM model with an accuracy of 96% performs outstandingly under RGB image technology.

[0143] In general, for Class 1 and Class 2, the SPA-LSTM model has the highest three evaluation indicators, which may be because the damage symptoms of Class 1 and Class 2 are relatively similar, and the networks and deep learning networks with relatively few feature band screening can distinguish them well. For the grading of 3, the SPA-LSTM model has the highest Recall index, and the Precision and F1-score evaluation indicators are 95.74% and 97.83%, respectively, which are not significantly different from the 98.88% and 98.32% of the NONE-LSTM model (only 3.14% and 0.49%). This is because 3 is a severe damage level, and its spectral characteristics are significantly different from the other two categories. All models can distinguish the three types of damage symptoms well. In short, the SPA-LSTM model has the best comprehensive performance for grading the severity of damage to tea green leafhoppers.

[0144] 5. Confusion Matrix

[0145] Confusion matrix is ​​one of the methods to evaluate model performance. In order to compare the misclassification of the severity of three types of hazard symptoms by different network models, we plotted the confusion matrix diagrams of 15 network models ( Fig. 9 ). In general, the eight models of hyperspectral imaging technology ( Fig. 9The misclassification rate of the six models of RGB imaging technology (g,h,I,j,k,l,m,n) is significantly lower than that of the six models of RGB imaging technology ( Fig. 9 a,b,c,d,e,f), wavelet transform enhanced RGB imaging technology model ( Fig. 9 d, e, f) The misclassification rate is significantly lower than that of the common RGB imaging technology model ( Fig. 9 a,b,c). And the SPA-LSTM model ( Fig. 9 m) had the least misclassification, indicating that the model can accurately distinguish the three severity levels of hazard symptoms.

[0146] Among the six models of RGB imaging technology, none of them can distinguish between Class 1 and Class 2 very well. About 45% of the samples belonging to Class 1 are wrongly predicted to the other two classes, about 35% of the samples belonging to Class 2 are wrongly predicted to the other two classes, and only about 26% of the samples belonging to Class 3 are wrongly predicted to the other two classes. This shows that the system has some problems in distinguishing Class 1 from Class 2, but the effect of distinguishing Class 3 is still good. There are two reasons: First, because the hazard characteristics of Class 1 and Class 2 are similar, it is difficult for the model to distinguish. Secondly, the information provided by RGB images is limited, and the accuracy of identifying the degree of hazard symptoms is much lower than that of hyperspectral images of 176 bands, which leads to confusion between the characteristics of the two hazard symptoms. Therefore, in subsequent research, we can try to increase the data set and use RGB cameras with higher pixels to improve the accuracy of the model.

[0147] Among the eight models of hyperspectral imaging technology, the misclassification rate is much lower than that of the RGB imaging technology model. Fig. 9 k,l,m,n) has a lower misclassification rate than the SVM model ( Fig. 9 g,h,I,j), indicating that deep learning methods are significantly better than traditional machine learning methods. Finally, we found that the SPA-LSTM model can accurately classify and has the least misclassification. Samples belonging to category 1 are completely correctly predicted, only about 6% of samples belonging to category 2 are incorrectly predicted to the other two categories, and only 4% of samples belonging to category 3 are incorrectly predicted to the other two categories. The accuracy of the model is 96%. The results show that the SPA-LSTM model has good robustness, which also shows the effectiveness and stability of the method, and can accurately classify the degree of damage of tea green leafhoppers.

[0148] Based on the above embodiments and data analysis, the best model of the present invention is the SPA-LSTM model, which has an accuracy rate of up to 96%.

[0149] The present invention is described above by way of examples, but the present invention is not limited to the above specific embodiments, and any changes or modifications made based on the present invention belong to the scope of protection claimed by the present invention.

Claims

1. A method for constructing a tea green leafhopper damage symptom detection model, characterized in that: The following steps are involved: Step 1: Obtain a dataset of tea bud images with different degrees of damage caused by tea leafhoppers; Step 2: Preprocess the acquired tea bud image sets respectively; Step 3: Build a tea green leafhopper damage symptom detection model and conduct model evaluation; Step 4: Train and test the model built in step 3 to obtain a deep learning model for detecting damage symptoms of tea green leafhoppers; The tea bud image dataset in step 1 includes an RGB image dataset and / or a hyperspectral image dataset; The RGB image dataset is obtained by picking tea buds damaged by tea leafhoppers to different degrees in the tea garden field using a digital camera; the hyperspectral image dataset is obtained by picking tea buds damaged by tea leafhoppers to different degrees in the tea garden field using a built-in push scan in a hyperspectral imaging system; In step 2, the obtained tea bud image set is preprocessed including RGB data preprocessing and hyperspectral data preprocessing; The RGB data preprocessing includes data amplification and data enhancement processing; The data augmentation: rotating the RGB image samples obtained in step 1 by 90°, 180°, 270°, vertically flipping, and horizontally flipping, so that the sample data becomes 6 times the original sample size; dividing the sample size of the RGB image into a training set, a test set, and a validation set in a ratio of 3:1:1; The data enhancement processing: the RGB image set is enhanced by 2D-DWT; in the process of 2D-DWT decomposing the image, the LL component can be circulated multiple times before meeting the requirements. For the discrete wavelet transform of the image f (x, y) of size M×N, the LL component is only circulated once, and the formula is as follows: In the formula, is an arbitrary starting scale; is in scale The approximate coefficient at ; represents the scaling function; The hyperspectral data preprocessing includes extracting spectral reflectance, spectral preprocessing and characteristic band screening; Extracting spectral reflectance: The hyperspectral image data set collected in step 1 is subjected to lens correction and reflection correction using the analysis tool of SpecVIEW; the hyperspectral image in RAW format is opened using ENVI 5.3 software, the entire area of ​​the sample is selected as the region of interest (ROI), and the average spectral reflectance of the region of interest is calculated as the spectral data of the sample; The spectrum preprocessing: the hyperspectral image is preprocessed by MSC, SG smoothing, first-order derivative, and second-order derivative; it increases exponentially in the wavelength range of 673-742nm, decreases rapidly in the wavelength range of 742-811nm, and there are obvious absorption peaks at 535nm and 742nm; The spectral characteristic band screening: the hyperspectral image adopts UVE, CARS and SPA algorithms to select representative bands as "characteristic bands" in the full-band spectral data; The tea green leafhopper damage symptom detection model described in step 3 is built, and the RGB data modeling adopts ResNet18, VGG16, AlexNet, WT-ResNet18, WT-VGG16 and WT-AlexNet deep learning algorithms for modeling; the hyperspectral data modeling adopts UVE-SVM, CARS-SVM, SPA-SVM, NONE-SVM, UVE-LSTM, CARS-LSTM, SPA-LSTM and NONE-LSTM algorithms for modeling; The model evaluation described in step 3 is evaluated using Accuracy, Precision, Recall, and F1-score respectively; and then the confusion matrix method is used for evaluation; The deep learning model for detecting damage symptoms of tea green leafhopper obtained in step 4 is the SPA-LSTM model, and its accuracy rate reaches 96%.

2. Application of the tea green leafhopper damage symptom detection model constructed by the construction method as described in claim 1 in detecting tea green leafhopper damage symptoms in tea trees.

3. A method for detecting the damage symptoms of tea green leafhopper, characterized in that: include: Acquire a tea bud image in a natural environment; input the tea bud image into the tea green leafhopper damage symptom detection model obtained by the method for building a tea green leafhopper damage symptom detection model as described in claim 1, and obtain the degree of tea green leafhopper damage symptoms output by the model.

4. A device for detecting the damage symptoms of tea green leafhopper, characterized in that: The device includes at least one processor and at least one memory; the memory stores program instructions of the tea green leafhopper damage symptom detection model obtained by the construction method as described in claim 1; when the processor executes the program instructions, it can realize the detection of the degree of damage symptoms of tea green leafhopper on tea buds.

Images