Early crop disease detection method based on multiple modes

Abstract

The invention is suitable for the technical field of modern agriculture and computers, and provides a multi-modal-based crop early disease detection method, which comprises the following steps: constructing a crop disease detection model based on a convolutional neural network, a gating circulation unit and a multi-head attention mechanism layer; based on the established experimental conditions, the leaf reflectivity spectrum and the leaf physiological variable are measured, and data collection is achieved; taking the collected data as sample data for model training and testing, and screening and preprocessing the sample data to obtain a sample data set; training and testing the crop disease detection model through the sample data set; inputting data for crop detection into the trained crop disease detection model, and outputting a result; according to the method, the GRU and the CNN are integrated, so that space and time features can be effectively captured from data; spectrum, physiological data and data conversion are combined, so that ShB infection can be accurately detected in an early stage.

Description

Technical Field

[0001] The present invention belongs to the field of modern agriculture and computer technology, and in particular to the field of deep learning technology, and specifically provides a multimodal-based crop early disease detection method. Background Art

[0002] Plant diseases pose a major threat to food security, and rice sheath blight (ShB) is a devastating pathogen. Sheath blight is a major disease of rice that can severely affect yield and productivity, especially in intensive cultivation, where it can lead to a 20% to 50% reduction in yield, depending on the severity of the infection; the surge in the prevalence of the disease is mainly due to the widespread cultivation of high-yielding semi-dwarf rice varieties, the extensive use of nitrogen fertilizers, and the favorable microenvironment cultivated by high crop densities. Sheath blight mainly targets rice leaves and sheaths. Depending on environmental factors, infected plants show disease symptoms within 24-144 hours, with the greatest damage during the tillering stage.

[0003] Conventional diagnosis of diseases such as sheath blight relies heavily on phenotypic observations and manual inspection methods, which are often subjective and prone to errors. In addition, these methods often identify the disease at a late stage when substantial damage has already occurred. This delay in detection seriously hinders effective disease management and control measures. Recent advances in detection technologies such as molecular diagnostics have improved the accuracy and timeliness of disease detection, but they still face challenges in cost, complexity, and scalability and are not suitable for high-throughput, on-site disease diagnosis. These challenges highlight the need for accurate, efficient, and non-invasive technologies that can detect early, asymptomatic infections.

[0004] Recent advances in spectroscopy and machine learning have opened up new avenues for plant disease detection, providing the potential for non-destructive, rapid, and precise monitoring. Near-infrared spectroscopy (NIR), in particular, has become a powerful tool in this field due to its ability to capture a wide range of spectral information beyond the capabilities of conventional imaging techniques. NIR exploits the unique spectral signatures of plants to detect subtle physiological and biochemical changes caused by disease. This non-destructive technique captures reflectance data over a continuous spectrum, enabling the identification of specific wavelengths associated with disease symptoms. In addition, handheld instruments like those used in the current study require minimal operator training and are relatively cost-effective. Recent studies have demonstrated the potential of hyperspectroscopy in diagnosing a variety of plant diseases, including rice blast, bacterial leaf blight, and the bacterial disease rice streak, by analyzing changes in spectral reflectance.

[0005] Although there are a few studies on the estimation of ShB disease severity and identification of infected leaf samples at late symptom stages, few studies have been devoted to the spectral detection of ShB at early infection stages and examining the biochemical characteristics of leaves. Several studies have demonstrated that early responses of plants to biotic stress, manifested as alterations in plant physiological parameters or photoprotection mechanisms, can be detected using sensitive spectral indices. Minor spectral and physiological changes can pose a great challenge for the unambiguous identification of infected samples at early stages. Reflectance spectroscopy is beneficial in revealing subtle spectral signals induced by complex plant pathogen infections. As a simple and effective tool for characterizing spectral changes, feature band extraction has also been used to highlight the factors of interest and suppress the influence of other factors, and is a popular tool for plant disease detection.

[0006] Previous studies have mainly focused on the independent use of hyperspectral or physiological measurements. However, the integration of both modalities, exploiting their synergistic potential, remains underexplored. Therefore, the use of advanced mathematical computational methods to combine these data sources could be explored to improve the accuracy and efficiency of ShB detection. In this context, the potential of advanced data analysis techniques such as deep learning remains underexplored.

[0007] Recent studies have shown that machine learning has obvious advantages in plant disease detection using different sensitive indicators as input. Although these studies provide all spectral features or extracted biochemical / biophysical features to build disease models, they do not attempt to optimize the combination of input variables for more effective detection of plant diseases. In addition, there is a lack of in-depth understanding of the contribution of relevant spectral features to disease detection. Furthermore, the study has not fully explored the effective combination of characteristic bands with key physiological parameters to improve the accuracy of disease diagnosis. Previous studies have demonstrated that the combination of carefully selected feature subsets can significantly improve the efficiency of classification models without significantly reducing the classification accuracy. In addition, many studies have found that the number of input variables or spectral features has a significant impact on the performance of machine learning algorithms. However, research on the application of multimodal strategies that fuse spectral information with physiological data in disease diagnosis is still limited. Therefore, the method of integrating spectral and physiological features and optimizing the combination of input variables is worthy of further exploration in future research.

[0008] Recent advances in machine learning and deep learning have further enhanced the potential of these diagnostic methods. Convolutional neural networks (CNNs) and gated recurrent units (GRUs) have been successfully applied to process spectral data and extract meaningful features for disease classification. However, through summary, it is found that the usual research methods mostly focus on the arrangement and analysis of one-dimensional data, and fail to give full play to the advantages of image recognition represented by CNN, which deserves further exploration. Summary of the invention

[0009] The purpose of the present invention is to provide a multimodal early crop disease detection method, which effectively integrates disease-specific spectral features and key synergistic physiological parameters by building a multimodal prediction model, namely a crop disease detection model, to capture subtle pre-symptomatic changes and achieve high accuracy in early ShB infection detection; and to provide an exploratory solution for deep learning in early crop disease detection.

[0010] The present invention is implemented as follows: a method for detecting early crop diseases based on multimodality, the method comprising the following steps:

[0011] A crop disease detection model was constructed based on a dual-branch architecture that integrated convolutional neural networks, gated recurrent units, and multi-head attention mechanism layers.

[0012] Based on the established experimental conditions, the leaf reflectance spectrum and leaf physiological variables are measured to achieve data collection;

[0013] The collected data is used as sample data for model training and testing, and the sample data is screened and preprocessed to obtain a sample data set;

[0014] Training and testing the crop disease detection model using the sample data set;

[0015] The crop detection data is input into the trained crop disease detection model to output the early crop disease detection results.

[0016] Furthermore, the experimental conditions were constructed based on an RXZ-1000 artificial climate chamber, and the experimental crop was Wuyou Rice No. 4;

[0017] The RXZ-1000 artificial climate chamber is equipped with:

[0018] High-precision lighting: 0-8000 lux;

[0019] Constant temperature control: 0-50℃±0.1℃ fluctuation;

[0020] Automatic humidity control: 50-95%, ±3% fluctuation:

[0021] Day and night cycle automatic control system: 0-99 hours;

[0022] The crops were divided into: control group, simulated infection group and actual infection group, and inoculated based on the specified infection technique;

[0023] The specified infection technique is one of the detached leaf method, microchamber method, mist chamber method, mycelial suspension spray method and mycelial embedding method.

[0024] Furthermore, a spectrometer is used to measure the reflectance spectrum of leaves, a portable photosynthetic meter is used to perform non-invasive in-situ synchronous collection of photosynthetic parameters of leaves, and a portable modulated chlorophyll fluorescence meter is used to perform non-destructive evaluation of chlorophyll fluorescence in leaves under natural and dark conditions to obtain fluorescence parameters; the photosynthetic parameters and fluorescence parameters are used together to complete the measurement of leaf physiological variables.

[0025] The spectrometer used is the handheld ASD FieldSpec2 (HH2) spectrometer, which can accurately capture the spectrum between 325 and 1075 nm with a sampling interval of 1.5 nm. The acquired spectral data can be processed by using ViewSpec Pro software V6.20, which simplifies the collection, visualization and analysis of spectral data.

[0026] The portable photosynthetic instrument (model LI-6800) has a 6800-02 red and blue light source leaf chamber (circular, area 2 square centimeters, emission wavelengths of 660nm and 453nm), which can provide robust control of environmental conditions; the device calculates key physiological parameters based on the mass balance of carbon dioxide and water vapor in the leaf chamber; these parameters include net photosynthetic rate (A, μmol·m -2 ·s -1 )), transpiration rate (E, mmol·m -2 ·s -1 )), stomatal conductance (Cond, mol·m -2 ·s -1 ), total water conductivity (CondTotal, molH2O·m -2 ·s -1 ), CO2 total conductivity (CondCO2, molH2O·m -2 ·s -1 ), intercellular carbon dioxide concentration (Ci, μmol·mol -1 ), leaf surface temperature (Tleaf, °C), leaf-air temperature difference (Ti-Ta, °C), leaf surface relative humidity and CO2 concentration (RHsfc, % and C2sfc, μmol·mol -1 )wait.

[0027] To ensure accuracy, the portable photosynthetic meter was calibrated in a climate chamber with the same settings as the growth chamber, which can quickly and accurately control environmental variables such as carbon dioxide concentration, light intensity, humidity and temperature; this setting has minimal impact on the measurement of external factors; the measurement points of the portable photosynthetic meter are consistent with the spectral measurement points mentioned above, and the average data of three points on each leaf is used to represent its photosynthetic parameters; data collection, visualization and output can be performed through FlashAnalysis-v1.0 and Microsoft Excel 2019 software.

[0028] The portable modulated chlorophyll fluorometer (model OS1P) integrates a comprehensive set of light sources, including tunable LED white light, modulated red light, and far-red light. It also has a photosynthetically active radiation (PAR) clamp with sensor and thermistor for accurate environmental readings.

[0029] Under illumination conditions, maximum fluorescence (Fms), steady-state fluorescence (Fs), photosynthetically active radiation (PAR), actual photochemical quantum yield (Y(II)) and electron transfer rate (ETR) were measured to capture light-induced excitation. The measurements were carefully set to ensure accuracy: the modulation intensity was adjusted to 50%, the saturation intensity to 75%, the flash duration was fixed at 0.8 seconds, and automatic gain control was enabled.

[0030] After 30 minutes of dark adaptation, the initial fluorescence (F0), maximum fluorescence (Fm), variable fluorescence (Fv), maximum photochemical quantum yield (Fv / Fm), and PSII potential activity (Fv / Fo) were accurately measured in situ using a dark-adapted leaf clamp. The measurement settings included: modulation intensity of 25%, saturation intensity of 75%, flash duration of 0.8 seconds, and automatic gain control enabled. In addition, the far-red light intensity was set to 50% for 5 seconds, which is a key regulation for activating photosystem I (PSI) and promoting rapid reoxidation of photosystem II (PSII), and other settings were set to default. Microsoft Excel 2019 and Origin 2022 software were used to efficiently complete the collection, analysis, processing, and presentation of fluorescence parameters.

[0031] Furthermore, the step of constructing a crop disease detection model based on a dual-branch architecture integrating a convolutional neural network, a gated recurrent unit, and a multi-head attention mechanism layer specifically includes:

[0032] In the convolutional neural network branch, set the image input layer, connect the image input layer with the convolution layer, set the convolution kernel and stride of the convolution layer; connect the convolution layer, batch normalization layer, pooling layer and fully connected layer in sequence, introduce the activation function ReLU as a rectified linear unit between the batch normalization layer and the pooling layer, and connect a flat layer after the fully connected layer;

[0033] In the gated recurrent unit branch, the image input layer and the gated recurrent unit are connected, and a fully connected layer and a flat layer are also set between the image input layer and the gated recurrent unit;

[0034] In the multi-head attention mechanism layer, the feature fusion layer is used as the input layer, which is connected to the convolutional neural network and the flat layer of the gated recurrent unit. The output of the feature fusion layer is fed into a fully connected layer, and then connected to a normalized index layer and the sheath blight classification layer.

[0035] Furthermore, the step of screening and preprocessing the sample data specifically includes:

[0036] Data screening: Monte Carlo partial least squares and interquartile range methods were used to screen outliers in spectral data and physiological parameters;

[0037] Data preprocessing: Combined processing using two or more data processing techniques; the two or more data processing techniques are: continuous wavelet transform, multivariate scatter correction, Savitzky-Golay convolution smoothing, and minimum-maximum normalization.

[0038] Furthermore, in the step of using the collected data as sample data for model training and testing, the one-dimensional sequence is converted into a two-dimensional image by converting Cartesian coordinates into polar coordinates and calculating the angle between data point vectors based on Gram Angle Sum Field (GASF).

[0039] Furthermore, the method further comprises:

[0040] The performance of the crop disease detection model is analyzed; the performance evaluation indicators include average accuracy, recall, precision and F1 score.

[0041] The present invention provides a method for detecting early crop diseases based on multimodality, wherein the crop disease detection model is a deep learning model (MMCG-MHA) that integrates GRU and CNN dual-branch multimodality and combines a multi-head self-attention mechanism, and can use Gram Angle Sum Field (GASF) to convert one-mode data into a two-dimensional image to achieve data layer or feature layer fusion, and can effectively capture spatial and temporal features from multimodal data; when detecting ShB infection of rice leaves, the leaf spectrum and physiological data as well as data conversion can be combined, so that ShB infection can be accurately detected at an early stage; in addition, the crop disease detection model can be used not only for rice sheath blight detection, but also for a variety of other diseases caused by fungi, and has a wide range of uses. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of a method for detecting early crop diseases based on multimodality in one embodiment;

[0043] Figure 2 are rice leaf images at various disease severity levels defined in one embodiment;

[0044] Figure 3 A schematic diagram of an embodiment of the MMCG-MHA model for early detection of rice sheath blight;

[0045] Figure 4It is a waterfall diagram of the change of spectral reflectance of the control group (CK), simulated disease group (SDG) and infection group (IDG) over time (day 0-6) in one embodiment;

[0046] Figure 5 It is a heat map of the correlation analysis between photosynthesis parameters (a), fluorescence parameters (b) and ShB in one embodiment;

[0047] Figure 6 It is a linear change diagram of each physiological index (A, Ci, E, Ti-Ta, RHsfc, C2sfc, Y(II), ETR) during the duration of the disease in one embodiment;

[0048] Figure 7 and Figure 8 Figure 1 is a comparison chart of key physiological parameters ANOVA analysis and Tukey HSD double Y-axis horizontal bar graph in one embodiment, wherein Figure 7 Shows the mean values ​​of different parameters between single treatment groups. Figure 8 Mean differences between double-treatment groups are presented;

[0049] Fig. 9 A schematic diagram of GASF conversion of 27 variable combinations in one embodiment;

[0050] Fig.10 A confusion matrix diagram of the classification accuracy of the optimal processing CARS-IVISSA in distinguishing the health status of rice and the course of ShB disease in the MMCG-MHA model of an embodiment;

[0051] Fig.11 A schematic diagram of sensitive bands selected for different feature band extraction methods in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0054] In one embodiment, Figures 1 to 11 As shown, a method for detecting early crop diseases based on multimodality may include the following steps:

[0055] A crop disease detection model was constructed based on a dual-branch architecture that integrated convolutional neural networks, gated recurrent units, and multi-head attention mechanism layers.

[0056] Based on the established experimental conditions, the leaf reflectance spectrum and leaf physiological variables are measured to achieve data collection;

[0057] The collected data is used as sample data for model training and testing, and the sample data is screened and preprocessed to obtain a sample data set;

[0058] Training and testing the crop disease detection model using the sample data set;

[0059] The crop detection data is input into the trained crop disease detection model to output the early crop disease detection results.

[0060] In one example of this embodiment, the experimental conditions have been established:

[0061] It was built based on the RXZ-1000 artificial climate chamber. The rice variety used in the experiment was Wuyoudao No. 4, also known as Daohuaxiang No. 2;

[0062] The RXZ-1000 artificial climate chamber is equipped with:

[0063] High-precision lighting: 0-8000 lux;

[0064] Constant temperature control: 0-50℃±0.1℃ fluctuation;

[0065] Automatic humidity control: 50-95%, ±3% fluctuation:

[0066] Day and night cycle automatic control system: 0-99 hours;

[0067] The crops were divided into: control group, simulated infection group and actual infection group, and inoculated based on the specified infection technique;

[0068] The specified infection technique is one of the detached leaf method, microchamber method, mist chamber method, mycelial suspension spray method and mycelial embedding method.

[0069] Exemplarily, the experiment involves rice planting and artificial inoculation.

[0070] The experiment was conducted in a glass greenhouse at the Nanling Campus of Jilin University, China (Changchun, China, 44°50′N, 125°18′E) from June to August 2022, in a dedicated artificial climate chamber of model RXZ-1000 produced by Jiangnan Instrument Factory. It included two aspects. Two different stages of infection experiments started on June 25 and July 15, respectively. Initially, 100 tillering rice plants were used to determine the most effective infection technique from five different methods. Subsequently, the second stage involved setting up a total of 80 barrels in three different groups: control group, simulated infection group and actual infection group, with 20, 20 and 40 barrels allocated to each group, respectively. To reduce the risk of cross-infection, each group was isolated separately in an RXZ-1000 artificial climate chamber, with 20 barrels placed in each chamber.

[0071] The RXZ-1000 artificial climate chamber, the entire rice growth and Rhizoctonia solani infection are carefully controlled in this artificial climate chamber, for example:

[0072] Throughout the nursery phase, the experiment followed the following steps: selecting quality seeds, drying, soaking, promoting germination, sowing, managing seedlings, transplanting, regular watering, fertilizing, and applying pesticides, but not fungicides. A key adjustment was made during the pre-sowing phase to enhance seed health: the seeds were first soaked in clean water for 12 hours to ensure adequate hydration. Subsequently, the seeds were soaked in a 25% cypermethrin solution diluted 2,000 to 4,000 times for 10-12 hours. This disinfection process is designed to effectively eradicate surface pathogens before germination and sowing, ensuring a healthier start for the seedlings.

[0073] Five artificial inoculation techniques for rice sheath blight were evaluated. These methods included the detached leaf method, microchamber method, mist chamber method, mycelial suspension spraying method, and mycelial embedding method. For example, after preliminary experiments, the mycelial embedding method was selected because of its moderate disease development rate and simple operation. The experiment was set up with three groups: a blank control group (CK), a mock inoculation group (SDG) using PDA (potato dextrose agar) blocks without the pathogen, and an actual inoculation group (IDG) using PDA blocks cultured with R. solani. The inoculation method and location can also be flexibly adjusted according to actual conditions. Approximately four-week-old seedlings were inoculated or mock-inoculated at the base of the stem using two 0.5-cm diameter round blocks of R. solani mycelium cultured on PDA blocks for 72 hours, with the mycelial side facing the stem. As a control, the plants remained untreated. After inoculation, specific environmental conditions were maintained in an artificial climate chamber as described above.

[0074] In this experiment, Daohuaxiang No. 2 was selected because its specific characteristics were suitable for the research objectives. After four weeks of rice cultivation, only healthy seedlings with similar growth conditions were carefully selected. These selected seedlings were then transplanted into plastic cultivation buckets (25.5 cm long, 16.7 cm wide, and 26.5 cm high), each of which could accommodate 4 seedlings, for infection experiments. The Rhizoctonia solani strain selected for this purpose was a second-generation standard strain provided by the Department of Microbiology of Nankai University and produced by Beijing Baozang Biotechnology Co., Ltd., ensuring high standards and consistency in the infection process.

[0075] Before starting the infection process, the conditions in the climatic chamber were carefully adjusted to reflect the conditions in the natural field environment. When infecting, in order to closely mimic the environmental conditions where the sheath blight is highly prevalent in culture, especially in humid and rainy weather, the environment of the climatic chamber was adjusted to maintain a temperature of 28 ± 0.2 °C, a relative humidity of 95 ± 3%, a light intensity of 4000-4500 LUX, and a diurnal cycle of 12 h light followed by 12 h dark;

[0076] Furthermore, in one example, disease severity in rice was defined, e.g. Figure 2 As shown, leaves without any lesions are considered healthy ( Figure 2 a). The early asymptomatic stage is the period before visible lesions appear on the leaves. During this stage, although the lesions are not obvious, they are characterized by a small number of water-soaked lesions, usually one or two, which are gray-green in color, and the rest of the leaf surface is unaffected ( Figure 2 b). In contrast, the severe infection stage can be identified by the appearance of multiple, irregularly shaped, marbled lesions on the leaf surface, characterized by a whitish center and brown edges ( Figure 2 c). To avoid misclassification, continuous visual inspection was performed during the study period. Before inoculation, representative rice samples were selected for in-depth spectroscopic and biochemical analysis from before inoculation (Heal), early stage without obvious symptoms (days after inoculation (DAI), DAI 1-5) to severe infection (ShB, DAI 6-7).

[0077] In an example of this embodiment, the collected data is used as sample data for model training and testing, and the sample data is screened and preprocessed to obtain a sample data set;

[0078] In this step, data screening was included: using Monte Carlo partial least squares and interquartile range methods to screen outliers in spectral data and physiological parameters;

[0079] Data preprocessing: Combined processing using two or more data processing techniques; the two or more data processing techniques are: continuous wavelet transform, multivariate scatter correction, Savitzky-Golay convolution smoothing, and minimum-maximum normalization.

[0080] Among them, eliminating outliers can prevent distortion of results and influence on subsequent analysis due to measurement errors, instrument failures or sample characteristics. By adopting appropriate outlier detection methods, such as statistics or machine learning techniques, spectral and physiological parameters that do not conform to the data distribution can be effectively identified and removed, thus providing an accurate and robust basis for subsequent model building and data interpretation.

[0081] Exemplarily, Monte Carlo partial least squares (MC-PLS) and interquartile range (IQR) were used to screen outliers in spectral data and physiological parameters.

[0082] Preprocessing of spectral data: continuous wavelet transform (CWT), multivariate scatter correction (MSC), Savitzky-Golay convolution smoothing (SG) and minimum-maximum normalization (MMN), which can be any one or any combination of these schemes;

[0083] The selection was based on their respective unique advantages. CWT is effective for baseline correction, while MSC precisely targets scattering issues while minimizing the effects of baseline drift. SG plays a vital role in reducing noise in spectral data, thereby significantly improving the signal-to-noise ratio. Finally, MMN standardizes the scale of the dataset, making it easier to identify overlapping spectral features and ensuring that all data points are adjusted to the same numerical range.

[0084] In one example of this embodiment, the preprocessed data is randomly divided into a training set (66.7%) and a test set (33.3%) for subsequent processing, model training and testing, etc.

[0085] In one example of this embodiment, feature selection techniques such as Monte Carlo partial least squares (MC-PLS) and interquartile range (IQR) separate disease-specific spectral features, which may have different effects on model results.

[0086] In this example, this impact can be evaluated; namely, the effect of two feature selection techniques and the impact of interval variables on model effectiveness, and the best combination of interval and individual variable selection as well as the best feature and model combination can be determined to improve the robustness of the model.

[0087] Furthermore, five interval variable feature extraction methods can be used: interval combinatorial optimization (ICO), interval partial least squares (IPLS), back interval partial least squares (BIPLS), collaborative interval partial least squares (SIPLS) and interval variable iterative space shrinkage method (IVISSA), and the importance of interval variables in each method can be compared; as well as two individual variable selection techniques: competitive adaptive reweighted sampling (CARS) and successive projection algorithm (SPA).

[0088] In an example of this embodiment, Figure 3 As shown, the steps of constructing a crop disease detection model based on a dual-branch architecture integrating a convolutional neural network, a gated recurrent unit, and a multi-head attention mechanism layer specifically include:

[0089] In the convolutional neural network branch, set the image input layer, connect the image input layer with the convolution layer, set the convolution kernel and stride of the convolution layer; connect the convolution layer, batch normalization layer, pooling layer and fully connected layer in sequence, introduce the activation function ReLU as a rectified linear unit between the batch normalization layer and the pooling layer, and connect a flat layer after the fully connected layer;

[0090] Among them, the image input layer can be defined using the imageInputLayer function, which receives a color image of size [227,227,3]. In the convolution layer, a 3x3 convolution kernel is used and the padding is set to 3 to maintain the spatial dimension of the feature map. The stride is set to 2, which effectively reduces the spatial dimension of the feature map while retaining important edge information through the padding strategy. This convolution operation captures local spatial features and provides basic features for subsequent deeper network layers. For the activation function ReLU, the negative value is set to 0 to alleviate the vanishing gradient and enhance the complex function representation; a fully connected layer fullyconnectedlayer (FC) and a flat layer (flat1) map the high-dimensional features to a 128-dimensional fully connected layer FC1000; then flattening is performed to provide input for classification or other tasks.

[0091] In the gated recurrent unit (GRU) branch, the image input layer and the gated recurrent unit are connected, and a fully connected layer and a flat layer are also set between the image input layer and the gated recurrent unit;

[0092] Among them, the gated recurrent unit enables the model to remember relevant information in a long sequence while ignoring irrelevant parts; initially, the data is input through the image input layer, which allows the model to receive one-dimensional sequence data with variable features. Next, the data is normalized to ensure standardization and optimize the convergence and generalization capabilities of the model. Then, the flat layer (flat2) maps the multi-dimensional features to a one-dimensional vector, providing the necessary data structure for deep processing in the model. GRU is configured with 128 output dimensions to further capture the dynamic features in the sequence data, and effectively processes the information flow of long sequences through its internal update and reset gate mechanism.

[0093] In the multi-head attention mechanism layer, the feature fusion layer is used as the input layer, which is connected to the convolutional neural network and the flat layer of the gated recurrent unit. The output of the feature fusion layer is fed into a fully connected layer, and then connected to a normalized index layer and the sheath blight classification layer; the feature fusion layer uses the addLayers function.

[0094] Among them, the GRU and CNN outputs are combined with the fusion layer through the connectLayers function to fuse the spatiotemporal features; the feature map is flattened into a vector and fed into a fully connected layer (FC1000) for feature extraction to adapt to the number of classes. Finally, the feature vector is converted into a probability distribution through the normalized exponential layer softmax (Prob), and the classification prediction is generated through the sheath blight classification layer (ShBclassification). The normalized exponential layer ensures that the sum of the probabilities is 1, which is suitable for multi-class tasks.

[0095] Such a crop disease detection model can process raw one-dimensional numerical data as well as two-dimensional Gram Angle Sum Field (GASF) images, which are mapped through specific transformations to accommodate the complexity of multimodal data; and the multi-head attention mechanism layer allows the model to process attention in multiple subspaces in parallel, enhancing its ability to build rich feature relationships and identify feature contributions between different locations;

[0096] The multi-head attention mechanism layer is configured with multiple attention heads and 50 value channels, and the output size is 100.

[0097] In an example of this embodiment, a ten-fold cross validation method is used to evaluate the generalization ability of the model on different data subsets. The evaluation indicators include average accuracy, recall, precision, and F1 score, which provide a comprehensive understanding of the performance of the model in various scenarios and evaluate its potential and effectiveness in real-world applications.

[0098] In one example of this embodiment, Figure 4As shown, by analyzing the raw spectral data, it was observed that the average spectral reflectance of the infected group samples gradually decreased with the extension of the disease course, reaching the lowest point when lesions appeared. In contrast, the reflectance of the control group and the simulated disease group increased slightly over time. Therefore, the spectral data of the control group and the simulated disease group were classified as the healthy (Heal) group. Although there was no significant difference in the overall shape of the near-infrared (NIR) spectra between the different treatments, there were differences in the average NIR reflectance intensity in certain spectral regions. These differences can be effectively used to diagnose the duration of the disease.

[0099] For example, in order to analyze the relationship between photosynthetic and fluorescence parameters, Pearson correlation analysis is used to select key indicators to avoid redundancy and improve analysis efficiency. Figure 5 As shown in the figure, each cell in the correlation matrix represents a parameter, and the color intensity and cell size indicate the strength of the correlation. The darker the color, the stronger the correlation; red indicates positive correlation, and blue indicates negative correlation. ShB is highly positively correlated with RHsfc, Ti-Ta, Ci, and C2sfc, and highly negatively correlated with A, E, Y(Ⅱ), and ETR. Therefore, these eight highly significant indicators were selected as co-physiological parameters. These indicators were input into the model together with the spectral features to verify the improvement in accuracy.

[0100] In the study of rice sheath blight, the changes of various physiological parameters at different time points and among different groups (control group (CK), simulated inoculation group (SDG) and actual inoculation group (IDG)) provided a basis for in-depth understanding of the impact of the disease. Figure 6 The changes of various physiological indices (A, Ci, E, Ti-Ta, RHsfc, C2sfc, Y(II), ETR) during the disease were shown, and the results among different groups were compared.

[0101] Specifically, the CK group A remained relatively stable at 5.74-6.12 μmol·m -2 ·s -1 The A value of the SDG and IDG groups was similar to that of the CK group in the early stage of the disease, but the A value of the IDG group decreased significantly in the later stage, reaching the lowest point of 11.65 and 11.34 μmol·m at DAI 2 and 3, respectively. -2 ·s -1 , and further decreased to 4.88 μmol·m -2 ·s -1This indicates that the disease has a significant inhibitory effect on photosynthesis. This trend shows that the effect of the disease on photosynthesis rate becomes more obvious with the increase of infection time. The Ci of the CK and SDG groups showed relatively stable small fluctuations overall, but the Ci value of the IDG group continued to rise and significantly increased to 201.98 μmol·mol on ShB (sixth day after inoculation). -1 . This indicates that the early stage of the disease does not have a great effect on the intercellular CO2 concentration, but as the disease worsens, gas exchange is significantly interrupted. The E values ​​of the CK and SDG groups showed consistent performance and fluctuated within a relatively stable range, while the E value of the IDG group was not much different from that of the CK in the early stage, but dropped significantly at DAI 2 and 3, and then gradually recovered to a higher level. This is related to water use and stomatal regulation, and the disease may affect leaf water balance and photosynthetic efficiency. Ti-Ta showed small differences among the groups; the difference between the CK and SDG groups was not significant, while the temperature difference in the IDG group increased slightly in the late stage of the disease, indicating that the disease reduced the transpiration rate and reduced the cooling effect of the leaves, resulting in an increase in the temperature of the diseased leaves. The RHsfc value of the CK group remained stable, while the relative humidity of the SDG and IDG groups increased in the late stage of the disease, especially DAI5. This may be related to the disease-induced stomatal closure and transpiration inhibition, which affected the dynamics of leaf surface water and changed the leaf surface microenvironment. C2sfc was relatively stable, with a slight increase in the CK and SDG groups. In the IDG group, C2sfc increased significantly during the onset of disease, reaching 295.87 μmol·mol on ShB (sixth day after inoculation). -1 . This suggests that the disease affected gas exchange and CO2 accumulation in the leaves, possibly due to CO2 accumulation caused by stomatal closure or increased pathogen-induced CO2 release. The Y(II) performance among the groups did not change much in the CK and SDG groups and remained basically stable. However, the actual light quantum yield of Y(II) in the IDG group decreased slightly in the later stage of the disease, indicating that the disease had a negative impact on the electron transport chain of photosystem II, further confirming the decrease in net photosynthetic rate. The ETR of the CK and SDG groups remained stable, but the IDG group showed obvious fluctuations during the disease period, reaching 38.83 μmol·m -2 ·s -1 This indicates that the disease has increasingly interfered with the electron transfer involved in photosynthesis, further affecting photosynthetic efficiency.

[0102] In summary, there were significant differences in the changes of various physiological indicators during the disease process among the different treatment groups. The CK and SDG groups remained stable in most physiological indicators, indicating that the simulated disease treatment had no significant effect on rice physiology. In contrast, the IDG group showed significant changes in all indicators, indicating the combined effects of the disease on rice photosynthesis, water regulation, and overall physiological status.

[0103] In one example of this embodiment, ANOVA (analysis of variance) was used to compare the differences in 8 physiological indicators among the three treatment groups. If the variance ANOVA showed significant differences, a post hoc multiple comparison (Tukey HSD) test was performed using a statistical model to further determine the significant differences between the different treatment groups. Figure 7 As shown in Figure 2, except for Y(II), RHsfc, and ETR, 5 of the 8 physiological indices were significantly different with p < 0.05, so the Tukey HSD test was performed ( Figure 8 The results showed that there were no significant differences in all physiological parameters between CK and SDG at different infection stages. Figure 6 Therefore, subsequent studies will consider these infection stages as healthy.

[0104] In one example of this embodiment, Fig.11 As shown, a schematic diagram of sensitive bands selected by different characteristic band extraction methods is shown. These selected bands have common areas, mainly concentrated in the range of 500-585nm and 737-855nm, indicating that these key sensitive bands are crucial for identifying ShB.

[0105] In this case, the selection is mainly concentrated in the trough (500-530nm), peak (550-585nm) and near infrared region (830-855nm) of the original spectrum. These wavelengths reflect the impact of diseases on the spectral characteristics of plant leaves to varying degrees. Analyzing these wavelengths can further understand the mechanism by which diseases affect plant physiological and biochemical processes, and provide a theoretical basis for subsequent disease diagnosis and prediction.

[0106] In one example of this embodiment, reference Fig. 9 , showing the sensitive bands selected by different feature band extraction methods and the GASF (Gram Angle and Field) transformation including synergistic physiological parameters; whether it is a single sensitive band or combined with synergistic physiological parameters, the data after GASF transformation show different structural characteristics. These differences not only reveal the diversity and complexity of feature band extraction methods in the selection process, but also highlight the importance of synergistic physiological parameters in improving and optimizing the GASF transformation process.

[0107] In an example of this embodiment, the effects of different feature extraction methods and multimodal data fusion strategies on model performance were compared; the experimental results show that the multimodal fusion model combining physiological parameters and specific sensitive bands shows superior performance in all test sets. Among them, the IVISSA-CARS (interval variable iterative spatial shrinkage method-competitive adaptive reweighted sampling) sensitive feature band combined with physiological parameters has the highest accuracy in MMCG-MHA, reaching 94.1667%. The specific classification results are as follows: Fig.10 As shown, this effectively verifies the effectiveness of this strategy in improving the accuracy of disease diagnosis.

[0108] In addition, by comparing the performance of a single model on the same dataset (Table 1), it can be found that the MMCG-MHA model has significant advantages in qualitative analysis tasks. It not only improves the efficiency of the model, but also significantly improves the accuracy of the model. Compared with the unimodal one-dimensional data feature classification model GRU, the improvement of MMCG-MHA is as high as 18.958%, and compared with the unimodal CNN classic image classification model, the improvement is as high as 11.6525%. This shows that MMCG-MHA can not only process and integrate key information from different modalities, but also effectively merge this information and capture its intrinsic connections, thereby obtaining more comprehensive and accurate qualitative analysis results. In contrast, classic models often only analyze a single modality and fail to fully utilize the complementarity and synergy between multimodal data.

[0109] Table 1 shows the accuracy of different characteristic band extraction methods in determining rice health status and sheath blight course in various models

[0110]

[0111] It is obvious from Table 1 that for the original full spectrum, both the single-modal and multimodal models performed poorly, with an accuracy rate of less than 50%. After applying CWT (Continuous Wavelet Transform) + MSC (Multivariate Scattering Correction) + SG (SG-Smoothing) + MMN (Maximum Minimum Normalization) spectral preprocessing, the performance of the model improved slightly, but not significantly, due to data redundancy and low efficiency. In addition, using only spectral sensitive feature bands or physiological parameters resulted in generally low recognition accuracy, indicating that there is room for improvement. However, combining the two will lead to significant performance improvements. The recognition effects of different processing methods vary greatly. Compared with the full spectrum, selecting the ShB sensitive feature band not only improves the accuracy of the model, but also significantly improves the efficiency by reducing the input parameters. In addition, combining physiological parameters as auxiliary information with specific sensitive bands effectively captures biological changes under disease conditions, thereby enhancing the discriminative ability and accuracy of the model. This finding further verifies the key role of physiological parameters in improving the performance of disease recognition models.

[0112] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A multimodal early crop disease detection method, characterized in that: The following steps are involved: A crop disease detection model was constructed based on a dual-branch architecture that integrated convolutional neural networks, gated recurrent units, and multi-head attention mechanism layers. Based on the established experimental conditions, the leaf reflectance spectrum and leaf physiological variables are measured to achieve data collection; The collected data is used as sample data for model training and testing, and the sample data is screened and preprocessed to obtain a sample data set; Training and testing the crop disease detection model using the sample data set; The crop detection data is input into the trained crop disease detection model to output the early crop disease detection results.

2. The multimodal early crop disease detection method according to claim 1, characterized in that: The experimental conditions described above were built based on the RXZ-1000 type artificial climate chamber, and the experimental crop was Wuyou Rice No. 4; The RXZ-1000 artificial climate chamber is equipped with: High-precision lighting: 0-8000 lux; Constant temperature control: 0-50℃±0.1℃ fluctuation; Automatic humidity control: 50-95%, ±3% fluctuation: Day and night cycle automatic control system: 0-99 hours; The crops were divided into: control group, simulated infection group and actual infection group, and inoculated based on the specified infection technique; The specified infection technique is one of the detached leaf method, microchamber method, mist chamber method, mycelial suspension spray method and mycelial embedding method.

3. The multimodal early crop disease detection method according to claim 1, characterized in that: in, A spectrometer was used to measure the reflectance spectrum of leaves, a portable photosynthetic meter was used to collect photosynthetic parameters of leaves in a non-invasive in-situ synchronous manner, and a portable modulated chlorophyll fluorescence meter was used to non-destructively evaluate the chlorophyll fluorescence in leaves under natural and dark conditions to obtain fluorescence parameters. The photosynthetic parameters and fluorescence parameters were used together to complete the measurement of leaf physiological variables.

4. The multimodal early crop disease detection method according to claim 1, characterized in that: The steps of constructing a crop disease detection model based on a dual-branch architecture by integrating a convolutional neural network, a gated recurrent unit, and a multi-head attention mechanism layer specifically include: In the convolutional neural network branch, set the image input layer, connect the image input layer with the convolution layer, set the convolution kernel and stride of the convolution layer; connect the convolution layer, batch normalization layer, pooling layer and fully connected layer in sequence, introduce the activation function ReLU as a rectified linear unit between the batch normalization layer and the pooling layer, and connect a flat layer after the fully connected layer; In the gated recurrent unit branch, the image input layer and the gated recurrent unit are connected, and a fully connected layer and a flat layer are also set between the image input layer and the gated recurrent unit; In the multi-head attention mechanism layer, the feature fusion layer is used as the input layer, which is connected to the convolutional neural network and the flat layer of the gated recurrent unit. The output of the feature fusion layer is fed into a fully connected layer, and then connected to a normalized index layer and the sheath blight classification layer.

5. The multimodal early crop disease detection method according to claim 1, characterized in that: The step of screening and preprocessing the sample data specifically includes: Data screening: Monte Carlo partial least squares and interquartile range methods were used to screen outliers in spectral data and physiological parameters; Data preprocessing: Combined processing using two or more data processing techniques; the two or more data processing techniques are: continuous wavelet transform, multivariate scatter correction, Savitzky-Golay convolution smoothing, and minimum-maximum normalization.

6. The multimodal early crop disease detection method according to claim 1, characterized in that: In the step of using the collected data as sample data for model training and testing, the one-dimensional sequence is converted into a two-dimensional image by converting Cartesian coordinates into polar coordinates based on the Gram angle and field and calculating the angle between data point vectors.

7. The multimodal early crop disease detection method according to claim 1, characterized in that: The method further comprises: The performance of the crop disease detection model is analyzed; the performance evaluation indicators include average accuracy, recall, precision and F1 score.

Images