Tomato yellow leaf curl disease condition grading method and system based on hyperspectral imaging technology and DB-ResNet

Abstract

The invention discloses a tomato yellow leaf curl disease condition grading method and system based on a hyperspectral imaging technology and DB-ResNet, and the method comprises the steps: firstly obtaining the hyperspectral imaging data of tomato leaves at different infection stages, and converting the hyperspectral imaging data into normalized reflectivity through black and white correction; then marking an area-of-interest of an upper leaf of the plant, cutting the image according to coordinates and spatially standardizing the image to a fixed pixel size, and then removing a noise wave band to construct a training data set; then establishing a DB-ResNet model and training, finally inputting the preprocessed to-be-detected sample data into the model, and outputting a tomato yellow leaf curl disease grade classification result. According to the method, hyperspectral data of a tomato plant sample to be detected is obtained in a lossless manner by applying a hyperspectral imaging technology, and the data is evaluated and graded by applying a modeling method including deep learning, so that a disease degree evaluation result of the tomato yellow leaf curl disease is efficiently and accurately obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of plant disease assessment and prevention and control, and specifically relates to a tomato yellow leaf curl disease grading method and system based on hyperspectral imaging technology and DB-ResNet. Background Art

[0002] Tomato is an herbaceous plant of the genus Tomato in the Solanaceae family. Its fruit is rich in nutrients and has become an important economic crop widely cultivated worldwide. Tomatoes are susceptible to a variety of diseases during their growth process. Among them, Tomato yellow leaf curl (TYLCD), a devastating disease caused by the genus Begomovirus of the family Geminiviridae, has a long incubation period and a rapid spread. This disease can cause plants to exhibit typical symptoms such as curled and yellowed leaves and malformed fruit. Infected plants can suffer a yield reduction of 30%-50% at best, or even complete crop failure at worst, resulting in significant economic losses. Therefore, establishing an effective early disease detection and assessment system is of great significance to the formulation of subsequent prevention and control measures.

[0003] Traditional methods for detecting tomato yellow leaf curl rely primarily on visual inspection and biological testing. The former cannot distinguish the disease in its asymptomatic stage, and due to the long incubation period and complex symptoms of the disease, it is difficult to accurately identify the disease even weeks after the plant has been infected. The latter includes molecular diagnostic technologies such as DNA-based polymerase chain reaction (PCR) and protein-based enzyme-linked immunosorbent assay (ELISA). Although these technologies have high specificity, they suffer from sample destructiveness, complex operations, and insufficient sensitivity for low viral load detection, making it difficult to meet the demand for non-destructive and rapid testing in precision agricultural production management. Given the urgent need for early detection and rapid non-destructive testing of tomato yellow leaf curl, it is necessary to find new technologies and methods to achieve breakthroughs in the accuracy of early detection and efficiency of detection. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a tomato yellow leaf curl disease grading method and system based on hyperspectral imaging technology and DB-ResNet. The method of the present invention uses hyperspectral imaging technology to non-destructively acquire hyperspectral data of tomato plant samples to be tested, and uses modeling methods including deep learning to evaluate and grade the data, so as to efficiently and accurately obtain tomato yellow leaf curl disease severity assessment results.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] First, a method for grading tomato yellow leaf curl disease based on hyperspectral imaging technology and DB-ResNet includes the following steps:

[0007] Step 1: Acquire hyperspectral imaging data of tomato leaf samples infected with tomato yellow leaf curl disease, wherein the samples are at different infection stages;

[0008] Step 2: performing black and white correction on the hyperspectral imaging data to convert the original reflectivity into a normalized reflectivity value;

[0009] Step 3: Mark the region of interest on the corrected hyperspectral image, where the marked region is limited to the upper leaves of the plant;

[0010] Step 4: Crop the original image spatial resolution according to the marked coordinates, and perform spatial normalization on the cropped sub-images to uniformly adjust them to a fixed pixel size;

[0011] Step 5: Remove the noise bands in the spatially normalized data and construct a training dataset;

[0012] Step 6: Establish a spectral-spatial dual-channel residual network (DB-ResNet) and train the model using the dataset obtained in step 5;

[0013] Step 7: Input the hyperspectral data of the sample to be tested after preprocessing in steps 2-5 into the trained DB-ResNet model and output the disease grade classification result.

[0014] Furthermore, the acquisition of the hyperspectral imaging data is specifically as follows:

[0015] The spectral range of hyperspectral imaging equipment is 400 to 1000 nm;

[0016] After preheating the hyperspectral imaging device, black and white frame data were collected. The black frame was collected by covering the lens cap of the hyperspectral camera, and the white frame was obtained by photographing a standard white board. The standard white board was made of polytetrafluoroethylene and had a reflectivity of 99% in the 200nm to 2500nm band.

[0017] When shooting, the hyperspectral imaging device is placed above the plant to be tested, focusing on the top of the plant, and shooting each sample plant by plant to ensure that the effective area of ​​​​each hyperspectral image includes the upper leaf part of the plant. The upper leaf refers to the newer true leaves located at the top of the plant, close to the stem tip or growth point.

[0018] Furthermore, the hyperspectral imaging data is subjected to black and white correction, specifically:

[0019]

[0020] in:

[0021] DN origin is the digital quantization value of the original hyperspectral image;

[0022] DN dark It is the dark current reference value of the black frame image in the shutter;

[0023] DN white It is the maximum value of a standard white board (99% reflectivity);

[0024] R white is the calibrated reflectivity of the whiteboard.

[0025] After correction, the data are converted into dimensionless relative reflectance values.

[0026] Furthermore, the method for extracting the region of interest and cropping the original image spatial resolution is as follows:

[0027] For each original hyperspectral image, 3 to 7 non-overlapping rectangular regions of interest were manually marked on the latest 3 to 4 true leaves of the plant.

[0028] After the annotation is completed, a Python program is written to extract sub-images from the hyperspectral cube according to the annotated coordinates.

[0029] Furthermore, the hyperspectral data after spatial resolution cropping of the original image is spatially normalized as follows:

[0030] Adaptive spatial normalization is performed based on the aspect ratio of the original image to uniformly adjust the image to the target pixel. The specific principles are as follows:

[0031]

[0032] in:

[0033] scale is the scaling ratio;

[0034] L is the target side length;

[0035] W is the spatial width of the cropped hyperspectral image;

[0036] H is the spatial height of the cropped hyperspectral image.

[0037] The adaptive spatial normalization is specifically divided into two cases:

[0038] (1) If the length or width of the original image is greater than the target side length, bilinear interpolation is used to downsample the image. The principle is as follows:

[0039] P(x,y)=(1-Δx)(1-Δy)P 00 +Δx(1-Δy)P 10+(1-Δx)ΔyP 01 +ΔxΔyP 11

[0040] in:

[0041] P(x,y) is the value at the coordinate point (x,y) to be interpolated;

[0042] Δx and Δy are normalized distances, i.e., the normalized offset of the target point (x, y) relative to the lower left corner of the nearest neighbor grid;

[0043] Reference point P ij are the four corner points of a 2×2 grid around the target point.

[0044] (2) If the length and width of the original image are both smaller than the target side length, bicubic interpolation is used to upsample the image. The principle is as follows:

[0045]

[0046] in:

[0047] W(t) is the weight coefficient during interpolation;

[0048] t represents the relative position of the pixel;

[0049] a is the tension coefficient, which is -0.5 according to the Mitchell-Netravali optimal parameter.

[0050] After completing the adaptive scaling operation on the image, the image is center-cropped. The coordinates of the center point of the image are first calculated, and then the image is cropped to the target side length.

[0051] Furthermore, the noise band removal is specifically as follows:

[0052] The first 30 bands and the last 20 bands of the spectral dimension in the hyperspectral data cube were removed, and the middle 206 bands were retained;

[0053] After the noise band is removed, the spectral range of the data becomes 450~957nm.

[0054] Furthermore, the DB-ResNet model in step (6) includes:

[0055] Spectral feature branch: It consists of cascaded 3D deformable convolution residual blocks, each of which contains a DeformableConv3D module, a batch normalization layer, and a spectral channel attention module;

[0056] Spatial feature branch: consists of ResBlock2D modules, including 3×3 two-dimensional convolutional layers;

[0057] Feature fusion module: maps the dual-branch output to the Hilbert space and realizes spectral-spatial feature fusion through entanglement gate simulation.

[0058] Secondly, the tomato yellow leaf curl disease and DB-ResNet disease grading system based on hyperspectral imaging technology includes a data acquisition module, a correction module, a region annotation module, a spatial processing module, a spectral processing module, a model building module, and a disease assessment module.

[0059] The data acquisition module is used to collect hyperspectral imaging data of tomato leaves;

[0060] The correction module performs black and white correction and reflectivity conversion;

[0061] The region marking module is used to mark the ROI in the upper lobe region;

[0062] The spatial processing module performs image cropping and spatial normalization;

[0063] The spectrum processing module is used to remove noise bands;

[0064] The model building module: building and training the DB-ResNet model;

[0065] The disease assessment module outputs the disease grade of the sample to be tested.

[0066] According to a third aspect, an electronic device includes:

[0067] at least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0068] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet.

[0069] In a fourth aspect, a computer-readable storage medium stores a program, which, when executed by a processor, implements the tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet.

[0070] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0071] The method of the present invention uses deep learning technology and hyperspectral imaging equipment to acquire hyperspectral images of tomato leaf samples (at different stages of infection with tomato yellow leaf curl disease). After processing the hyperspectral images through steps such as black-and-white correction, region-of-interest annotation, spatial resolution cropping, spatial normalization, and noise band removal, the images are input into a trained hierarchical prediction model to obtain corresponding disease severity assessment results. Acquisition of hyperspectral image data can be accomplished using existing hyperspectral imaging equipment, and the entire process from correction, spatial resolution and spectral band processing to data set preparation all utilizes comprehensive hyperspectral data processing technology.

[0072] The establishment of the classification prediction model of the present invention belongs to the application of deep learning technology in the field of artificial intelligence. Therefore, the tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet proposed in the present invention obtains a tomato yellow leaf curl disease grading result with higher efficiency. Moreover, the methods proposed in the present invention are all non-destructive processes and do not involve destructive operations such as crushing and extraction of samples. Therefore, the methods of the present invention are more conducive to application in production practice and other situations where non-destructive and rapid assessment of disease severity is required. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0074] Figure 1 This is a flowchart of a method for grading tomato yellow leaf curl disease based on hyperspectral imaging technology and DB-ResNet in an embodiment of the present invention;

[0075] Figure 2 FIG2 is a diagram showing the cultivation process of a tomato plant sample in an artificial climate chamber according to an embodiment of the present invention;

[0076] Figure 3 A diagram of the hyperspectral imaging device and working scene used in the embodiments of the present invention;

[0077] Figure 4 This is a graph showing the results of PCR testing of experimental tomato samples as a reference standard for the actual disease status of the plants according to an embodiment of the present invention;

[0078] Figure 5 This is a schematic diagram of marking regions of interest in an original hyperspectral image according to an embodiment of the present invention;

[0079] Figure 6Schematic diagram of performing spatial normalization on a sub-image obtained by cropping according to the spatial resolution of the marked area according to an embodiment of the present invention;

[0080] Figure 7 This is a schematic diagram of the model structure of DB-ResNet, a deep learning model originally developed by an embodiment of the present invention;

[0081] Figure 8 is an average spectrum curve of samples at six different disease stages according to an embodiment of the present invention;

[0082] Figure 9a This is a confusion matrix result diagram of the disease stage classification by the 3DCNN model according to an embodiment of the present invention;

[0083] Figure 9b is a confusion matrix result diagram of the HybridSN model for classifying disease stages according to an embodiment of the present invention;

[0084] Figure 9c This is a confusion matrix result diagram of the disease stage classification by the DB-ResNet model according to an embodiment of the present invention;

[0085] Figure 10a 3DCNN model training with training set loss and validation set loss as a function of training rounds;

[0086] Figure 10b Graph showing changes in training set loss and validation set loss over training rounds during HybridSN model training according to an embodiment of the present invention;

[0087] Figure 10c This is a graph showing how the training set loss and validation set loss change with the number of training rounds during DB-ResNet model training according to an embodiment of the present invention.

[0088] Figure 11 This is a block diagram of a nondestructive detection system for tomato yellow leaf curl disease based on hyperspectral imaging technology according to an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0090] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0091] like Figure 1 As shown, this embodiment provides a nondestructive detection method for tomato yellow leaf curl disease based on hyperspectral imaging technology, including the following steps:

[0092] 1. Preparation of diseased tomato plant samples

[0093] In this embodiment, the tomato variety used is Xinxing 101. The seeds are sown in a nutrient medium and cultivated in an artificial climate chamber (temperature 26°C, photoperiod 16h light / 8h dark). After the seedlings germinate and form two cotyledons, several plants with the same growth status are selected and transplanted into separate culture pots. When the plants develop to the stage of 3 to 4 true leaves, an Agrobacterium suspension (carrying tomato yellow leaf curl virus) is prepared and injected into the phloem tissue 1 mm below the cotyledonary node of the plant. The inoculated plants are placed in a biosafety isolation greenhouse for subsequent cultivation, and the environmental parameters are consistent with the germination period.

[0094] 2. PCR testing

[0095] In this example, 28 days after inoculation with Tomato Yellow Leaf Curl Virus, polymerase chain reaction (PCR) was used as a standard to confirm the virus infection of the sample plants. After extracting the total DNA of the tomato sample using the improved NaOH method, the extracted total DNA of the sample plant was used as a template, and specific primers ToLCTwV-F and ToLCTwV-R were designed to perform PCR detection on the sample. The test results are shown in the figure. Figure 4 . Among the 50 inoculated samples, the target fragment was clearly amplified from the DNA of 45 strains. Among the remaining 5 strains, the target fragment was not amplified from the DNA of 2 samples (No. 35 and 46), and from a biological point of view, they were not infected by the virus; the bands of 3 samples (No. 28, 29 and 43) were very weak, and the plants did not show phenotypic symptoms, indicating that the virus load was extremely low, and they may be latent infections, and the disease severity classification was not considered. Finally, based on the PCR test results, the 45 plant samples that tested positive for PCR were used as experimental samples actually infected with tomato yellow leaf curl disease.

[0096] 3. Hyperspectral imaging data acquisition

[0097] In this embodiment, the hyperspectral imaging system used consists of a hyperspectral camera, a halogen light source, a camera stand, a sunshade, a computer, and control software. The core component of the system is the hyperspectral camera, which is a push-broom hyperspectral imager with 256 bands in the 400-1000 nm spectral range and a spatial resolution of 696×697 pixels. The vaccination day is set as the reference day, and a total of 12 collection days are set within 5 to 38 days after vaccination (5th, 7th, 9th, 11th, 13th, 15th, 17th, 19th, 22nd, 25th, 33rd, and 38th days after vaccination). Before collection, turn on the power of the hyperspectral camera and the halogen light source, and preheat the hyperspectral imaging system for 30 minutes to ensure that the equipment is stable. After the system is preheated, the camera exposure time is set to 9 milliseconds. At this value, the image of the standard white plate will not be overexposed or too dark. Then, a dark current reference frame (obtained by covering the hyperspectral camera lens) and a standard white plate (99% reflectivity, made of polytetrafluoroethylene) are collected for correction data. When starting data collection, place the plant and the culture pot on a black background (non-woven fabric with an absorbance of >98%), and use the acquisition control software installed in the computer to obtain a bird's-eye view hyperspectral image of the tomato plant. When scanning, focus the camera lens on the canopy of the tomato plant to ensure that the effective image includes the latest 3 to 4 true leaves.

[0098] 4. Black and white correction

[0099] In this embodiment, the original hyperspectral image is subjected to black and white correction using the dark current reference frame and the standard whiteboard frame obtained in Implementation 3 to eliminate sensor dark current noise and illumination unevenness. The principle of black and white correction is as follows:

[0100]

[0101] in:

[0102] DN origin is the digital quantization value of the original hyperspectral image;

[0103] DN dark It is the dark current reference value of the black frame image in the shutter;

[0104] DN white It is the maximum value of a standard white board (99% reflectivity);

[0105] R white is the calibrated reflectivity of the whiteboard.

[0106] 5. Region of Interest Annotation

[0107] According to the relevant principles of botanical virology, viruses are systemically transported through the phloem sieve tubes of plants and preferentially target active cell division areas (new leaves) for replication. In this embodiment, the obtained hyperspectral image is subjected to region of interest selection to focus on the new leaves targeted by the virus. By selecting the grayscale images corresponding to the specified bands (R: 680.40nm, G: 550.20nm, B: 480.10nm) in the hyperspectral data cube, a true color image is synthesized, and the new true leaf area is manually annotated using Labelme software based on the true color synthesized image. Each region of interest contains several leaflets to ensure data diversity. Each image is annotated with 3 to 7 ROIs, and these ROIs do not repeat each other. After the annotation is completed, the sub-image is extracted from the hyperspectral cube according to the annotated coordinates.

[0108] 6. Spatial resolution cropping

[0109] In this embodiment, based on the coordinates of the region of interest marked in Implementation 5, a Python program is written to automatically extract the sub-image corresponding to the region of interest from the original hyperspectral image.

[0110] 7. Spatial Standardization

[0111] In this embodiment, in order to solve the problem of inconsistent spatial resolution of the hyperspectral image after processing in embodiment 6, a Python program is written to perform adaptive spatial normalization based on the aspect ratio of the original image, adjust the aspect ratio of the image to 1, and adjust the number of pixels to a fixed value. The specific scaling ratio follows the following formula:

[0112]

[0113] in:

[0114] scale is the scaling ratio;

[0115] L is the target side length;

[0116] W is the width of the cropped hyperspectral image in spatial dimension;

[0117] H is the height of the spatial dimension of the cropped hyperspectral image.

[0118] The above formula shows that two sampling methods are applied to the cropped hyperspectral image based on two different conditions of the original image spatial resolution and the target side length:

[0119] (1) If the length or width of the original image is greater than the target side length, bilinear interpolation is used to downsample the image. The principle is as follows:

[0120] P(x,y)=(1-Δx)(1-Δy)P 00 +Δx(1-Δy)P10 +(1-Δx)ΔyP 01 +ΔxΔyP 11

[0121] in:

[0122] P(x,y) is the value at the coordinate point (x,y) to be interpolated;

[0123] Δx and Δy are normalized distances, i.e., the normalized offset of the target point (x, y) relative to the lower left corner of the nearest neighbor grid;

[0124] Reference point P ij are the four corner points of a 2×2 grid around the target point.

[0125] (2) If the length and width of the original image are both smaller than the target side length, bicubic interpolation is used to upsample the image. The principle is as follows:

[0126]

[0127] in:

[0128] W(t) is the weight coefficient during interpolation;

[0129] t represents the relative position of the pixel;

[0130] a is the tension coefficient, which is -0.5 according to the Mitchell-Netravali optimal parameter.

[0131] After completing the adaptive scaling operation on the image, the image is center-cropped. The coordinates of the center point of the image are first calculated, and then the image is cropped to the target side length.

[0132] 8. Noise band removal

[0133] Due to physical limitations, the edge bands of hyperspectral imaging equipment (approximately the first and last 10%) have low quantum efficiency and significant dark current noise, making these bands incapable of contributing to disease classification. In this example, the first 30 and last 20 bands of the spectral dimension of the hyperspectral data cube were removed, retaining the middle 206 bands. After band removal, the spectral range of the dataset was reduced to 450-957 nm.

[0134] 9. Dataset Construction

[0135] In this embodiment, after processing according to methods 4 to 8, the raw hyperspectral imaging data was converted into several standard three-dimensional tensors, which served as the total dataset for training the deep learning model. In terms of dataset partitioning, the samples were divided into three main stages based on the visibility of symptoms: asymptomatic latent period (early stage), partially symptomatic period (middle stage), and typically symptomatic period (late stage). Furthermore, based on the duration of disease progression, the data were further refined into six substages: early stage 1, early stage 2, middle stage 1, middle stage 2, middle stage 3, and late stage. Plants in early stage 1 (5-7 days after inoculation) and early stage 2 (8-11 days after inoculation) had not yet shown visible symptom signs and were therefore classified as asymptomatic latent period (early stage). Some plants in middle stage 1 (12-15 days after inoculation), middle stage 2 (16-19 days after inoculation), and middle stage 3 (20-30 days after inoculation) gradually began to show symptoms, but a significant proportion of individuals remained asymptomatic, indicating that infection status cannot be determined solely based on phenotypic characteristics at this stage. This stage was defined as partially symptomatic period (middle stage). Thirty days after inoculation, most infected plants showed typical symptoms. Although a few samples still had ambiguous symptoms, they could be roughly classified through visual observation and were therefore defined as the typical symptomatic stage (late stage). Based on this disease stage classification method, a multi-stage classification dataset was constructed, containing six disease course categories from early to late stages.

[0136] 10. Deep learning model establishment

[0137] In this embodiment, a spectral-spatial dual-channel residual network (DB-ResNet) is provided and experimentally verified as the optimal model for the system, and two benchmark deep learning models are provided as controls, namely a three-dimensional convolutional neural network (3DCNN) and a hybrid spectral network (HybridSN).

[0138] (1) Spectral-Spatial Dual-Branch Residual Network (DB-Resnet)

[0139] In this embodiment, a spectral-spatial dual-branch residual network (DB-Resnet) is provided. The comparative tests and ablation experiments conducted during the implementation process show that the model can better capture the spatial and spectral features in hyperspectral imaging data and effectively integrate them. The overall network structure diagram and key component structures are shown in Figure 2. Figure 7 DB-Resnet consists of three parts: spectral feature branch, spatial feature branch and feature fusion module:

[0140] a. Spectral feature branch: It consists of a cascade of 3D deformable convolution residual blocks and 3D maximum pooling layers, responsible for extracting high-dimensional spectral features. Data is input into the spectral feature branch in the form of a hyperspectral image cube. The spectral feature branch captures the spectral dimension features of the hyperspectral data cube through a three-dimensional convolution operation, and introduces a 3D maximum pooling layer for downsampling to focus features. It is characterized by the fact that this branch is mainly composed of stacked ResBlock3D modules, each of which contains a DeformableConv3D module, a BatchNorm layer, and a SpectralAttention3D module. The mathematical expression of ResBlock3D is as follows:

[0141] F out =ReLU(F res +H(F in ))

[0142] H(F in )=BN(A(DeformConv3D(F in )))

[0143] The DeformableConv3D module uses deformable convolution to enhance spectral modeling capabilities. Deformable convolution uses an offset generation network to dynamically calculate the offset of each convolution kernel on the input feature map, adjusting the kernel's position as needed to better adapt to spectral characteristics. The deformable convolution kernel has a size of 3×3×3 (depth×height×width), a stride of 1, and a padding of 1. The offset generation network consists of 3×3×3 convolutional layers and outputs 3K channels of offsets (K is the kernel volume). Dynamic sampling is achieved through bilinear interpolation.

[0144] The SpectralAttention3D module strengthens important spectral bands through the channel attention mechanism. Its main calculation process includes two steps: first, the input feature X∈R B×C×H×W×D Perform global average pooling to obtain z∈R B×C ; Secondly, the channel weight α is generated through two layers of fully connected layers and the Sigmoid function. The specific principle is as follows:

[0145] α=σ(W2·ReLU(W1·z))

[0146] in:

[0147] is a learnable parameter;

[0148] Compression ratio r=4.

[0149] b. Spatial feature branch: It uses two-dimensional convolution to extract local spatial patterns. Its core component is the ResBlock2D module, which consists of two 3×3 convolutional layers with a stride of 1 and a padding of 1. The mathematical expression is:

[0150] Y=ReLU(BN(W2*ReLU(BN(W1*X))+X)

[0151] in,

[0152] * indicates convolution operation;

[0153] W1, W2∈R k×k (k=3) is the convolution kernel weight.

[0154] The ResBlock2D module is characterized by using the BN layer to accelerate the convergence process of training, and the introduced residual connection adjusts the channel dimension through 1×1 convolution.

[0155] c. Feature fusion module: The feature fusion module is used to effectively associate and fuse the output features of the two branches. It is characterized by the fact that the feature maps output by the spectral branch and the spatial branch are respectively input into the feature fusion module after passing through the pooling layer, and information interaction is achieved by simulating quantum entanglement fusion. Spatial characteristics The fusion process of the two simulates the superposition principle of quantum entangled states. First, the output features of the two branches are mapped to the Hilbert space through a linear transformation matrix to construct a composite entangled state:

[0156]

[0157] in:

[0158] U s and U P is the linear transformation matrix;

[0159] The symbol represents the tensor product operation.

[0160] Then, a generalized simulation of the entanglement gate is performed, and a strong correlation between spectral and spatial features is established through parameter learning. The key lies in constructing a multi-layer perceptron to simulate the nonlinear transformation of the entangled state by the quantum gate:

[0161] F fused =W2·ReLU(W1·[S;P])

[0162] in:

[0163] [S;P] indicates the splicing operation of spectral features and spatial features;

[0164] W1 and W2 are weight matrices, W2∈R 256×256 ;

[0165] The hidden layer dimension is 256.

[0166] Finally, the fused features are mapped to the category space through a fully connected layer, which can be compared to projecting quantum states onto classical information. The classifier has a dropout layer that randomly blocks a certain number of neurons to suppress overfitting and improve the model's generalization ability. The specific operation is as follows:

[0167]

[0168] (2) Three-dimensional Convolutional Neural Network (3DCNN)

[0169] In this example, the 3D-CNN used has been modified to adapt to the task of hyperspectral image classification. Its main structure consists of two three-dimensional convolutional blocks. The first convolutional block consists of a cascade of a convolutional layer with a kernel size of 5×5×7 and a pooling layer with a kernel size of 3×3×5. The second convolutional block consists of a convolutional layer with a kernel size of 3×3×5 and a pooling layer with a kernel size of 2×2×3. Spectral feature compression is achieved by gradually reducing the spectral dimension. Finally, global average pooling is used to map the three-dimensional feature tensor into a class probability distribution.

[0170] (3) Hybrid Spectral Network (HybridSN)

[0171] In this embodiment, the HybridSN model is adopted, whose spectral-spatial feature learning module consists of three consecutive 3D convolutional layers, which is used to extract joint spectral-spatial features from the input hyperspectral data cube; the spatial feature learning module receives the feature map output by the reshaped 3D convolutional layer and uses three 2D convolutional layers to further refine and abstract the spatial features; finally, the flattened feature map is input to the fully connected layer to map the features to the classification space.

[0172] 11. Evaluation indicators

[0173] In order to quantify the classification performance of the deep learning model, this study uses the confusion matrix and its derived indicators for comprehensive evaluation. The basic structure of the confusion matrix is ​​shown in Table 1.

[0174] Table 1 Basic structure of confusion matrix

[0175]

[0176] in,

[0177] TP is a true positive, which means that the disease sample at a certain stage is correctly predicted to be a sample of that stage;

[0178] FP is a false positive, which means that a disease sample at a certain stage is mistakenly predicted to be a sample at another stage;

[0179] TN is a true negative, where for a sample of a certain stage, a disease sample representing another stage is correctly predicted to be a sample of another stage;

[0180] FNB is a false negative. For samples of a certain stage, disease samples representing other stages are mistakenly predicted as samples of a certain stage.

[0181] Based on the matrix elements, four evaluation indicators are defined, namely overall accuracy (Accuracy), category accuracy (Precision), category recall (Recall) and F1 score (F1 score). The calculation methods are as follows:

[0182]

[0183] 11. Deep Learning Model Training

[0184] In this embodiment, model training is performed on a computing platform equipped with an Intel Core i7-14700KF CPU, 32GB DDR5 memory, and an NVIDIA GeForce RTX 4080 GPU; the software environment includes CUDA 11.8, Python 3.11, and PyTorch 2.5.1 frameworks; in terms of training strategy, a cross-entropy loss function and Adam optimizer are used, with an initial learning rate of 3e-4, and dynamically adjusted with the StepLR learning rate scheduler (gamma = 0.8, attenuation is performed every 20 rounds). A batch size of 8 is used during training, and the model is iterated for 200 rounds.

[0185] 12. Model performance comparison and evaluation

[0186] Based on the trained grading prediction model of this embodiment, when grading tomato yellow leaf curl disease, hyperspectral imaging data of the diseased tomato plants to be tested is collected, the data is black and white corrected, the region of interest is marked, the spatial resolution is cropped and spatially normalized, and then the noise band is removed. The results of the above processing are input into the trained grading prediction model, and the grading prediction model outputs the disease assessment result. The grading performance of the model is evaluated using the given evaluation indicators. The specific performance of the three models in the six-stage grading is shown in Table 2. As can be seen from Table 2, the DB-ResNet model outperforms the 3DCNN and HybridSN models in overall accuracy and the category accuracy, recall rate and F1 score of each stage. In particular, the category accuracy in the early stage reaches 0.825, showing significant superiority.

[0187] Table 2 Effect of the present invention on disease severity classification of tomato yellow leaf curl

[0188]

[0189]

[0190] In this example, ablation experiments were designed to validate the role of key components in the model. The spectral channel attention (SA) module and the deformable convolution (DConv) module were removed from the DB-ResNet model, and the results of the ablation experiments were compared with those of the original model. The results showed that the performance of the models without either the SA or DConv module on the disease classification task decreased to varying degrees compared to the original model. Specific results are shown in Table 3. The ablation experiments demonstrated that the addition of these key components significantly improved the DB-ResNet model's performance in grading the severity of tomato yellow leaf curl disease.

[0191] Table 3 Ablation experiment results of the present invention to explore the contribution of DB-Resnet components

[0192]

[0193]

[0194] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet is characterized by: The method comprises the following steps: Step 1: Acquire hyperspectral imaging data of tomato leaf samples infected with tomato yellow leaf curl disease, wherein the samples are at different infection stages; Step 2: performing black and white correction on the hyperspectral imaging data to convert the original reflectivity into a normalized reflectivity value; Step 3: Mark the region of interest on the corrected hyperspectral image, where the marked region is limited to the upper leaves of the plant; Step 4: Crop the original image spatial resolution according to the marked coordinates, and perform spatial normalization on the cropped sub-images to uniformly adjust them to a fixed pixel size; Step 5: Remove the noise bands in the spatially normalized data and construct a training dataset; Step 6: Establish a spectral-spatial dual-channel residual network (DB-ResNet) and train the model using the dataset obtained in step 5; Step 7: Input the hyperspectral data of the sample to be tested after preprocessing in steps 2-5 into the trained DB-ResNet model and output the disease grade classification result.

2. The tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to claim 1 is characterized in that: In step 1: The spectral range of hyperspectral imaging equipment is 400 to 1000 nm; During acquisition, after the device is preheated, a dark current reference frame and a standard whiteboard frame are captured. The standard whiteboard is made of polytetrafluoroethylene and has a reflectivity of ≥99% in the 200-2500nm band. The shooting position is directly above the plant, focusing on the upper leaf area at the top of the plant.

3. The tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to claim 1 is characterized in that: The black and white correction formula in step (2) is specifically: in: DN origin is the digital quantization value of the original hyperspectral image; DN dark It is the dark current reference value of the black frame image in the shutter; DN white is the maximum value of a standard whiteboard; R white is the calibrated reflectivity of the whiteboard.

4. The tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to claim 1 is characterized in that: In step (3): For each image, 3 to 7 non-overlapping rectangular regions of interest were marked in the upper lobe area; Sub-images are cropped from the hyperspectral cube using a coordinate extraction procedure.

5. The tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to claim 1 is characterized in that: The spatial normalization in step (4) includes: Adaptive spatial normalization is performed based on the aspect ratio of the original image to uniformly adjust the image to the target pixel. The specific principles are as follows: in: scale is the scaling ratio; L is the target side length; W is the spatial width of the cropped hyperspectral image; H is the spatial height of the cropped hyperspectral image; The spatial standardization is specifically divided into two cases: (1) If the length or width of the original image is greater than the target side length, bilinear interpolation is used to downsample the image. The principle is as follows: P(x,y)=(1-Δx)(1-Δy)P 00 +Δx(1-Δy)P 10 +(1-Δx)ΔyP 01 +ΔxΔyP 11 in: P(x,y) is the value at the coordinate point (x,y) to be interpolated; Δx and Δy are normalized distances, i.e., the normalized offset of the target point (x, y) relative to the lower left corner of the nearest neighbor grid; Reference point P ij are the four corner points of the 2×2 grid around the target point; (2) If the length and width of the original image are both smaller than the target side length, bicubic interpolation is used to upsample the image. The principle is as follows: in: W(t) is the weight coefficient during interpolation; t represents the relative position of the pixel; a is the tension coefficient, which is -0.5 according to the Mitchell-Netravali optimal parameter.

6. The tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to claim 1, characterized in that: The noise band removal in step (5) includes: Delete the first 30 bands and the last 20 bands of the spectral dimension; 206 valid bands within the range of 450 to 957 nm are retained.

7. The tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to claim 1, characterized in that: The DB-ResNet model in step (6) includes: Spectral feature branch: It consists of cascaded 3D deformable convolution residual blocks, each of which contains a DeformableConv3D module, a batch normalization layer, and a spectral channel attention module; Spatial feature branch: consists of ResBlock2D modules, including 3×3 two-dimensional convolutional layers; Feature fusion module: maps the dual-branch output to the Hilbert space and realizes spectral-spatial feature fusion through entanglement gate simulation.

8. A tomato yellow leaf curl disease grading system based on hyperspectral imaging technology and DB-ResNet, the system being applicable to the tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet according to any one of claims 1 to 7, characterized in that: The system includes: Data acquisition module: collects hyperspectral imaging data of tomato leaves; Correction module: performs black and white correction and reflectivity conversion; Region annotation module: annotate ROI in the upper lobe area; Spatial processing module: performs image cropping and spatial normalization; Spectral processing module: remove noise bands; Model building module: build and train the DB-ResNet model; Disease assessment module: outputs the disease level of the sample to be tested.

9. An electronic device, characterized in that: The electronic device comprises: A processor and a memory, wherein the memory stores a computer program, and when the program is executed by the processor, the tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet is implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by the processor, the tomato yellow leaf curl disease grading method based on hyperspectral imaging technology and DB-ResNet as described in any one of claims 1 to 7 is implemented.

Images