Method and device for identifying rice plant diseases and pests, electronic device and storage medium

Abstract

The application discloses a rice disease and pest identification method and device, electronic equipment and storage medium, relates to the field of artificial intelligence or other related technical fields, wherein the method comprises: preprocessing multispectral images and visible light images to obtain preprocessed image data; combining real-time weather data and rice growth stage information to extract spectral features, disease spot spatiotemporal evolution features, virtual microscopic features and ecological correlation features from the preprocessed image data, and fusing the spectral features, disease spot spatiotemporal evolution features, virtual microscopic features and ecological correlation features into a fusion feature vector through an attention mechanism; inputting the fusion feature vector into a rice disease and pest identification model to output disease and pest types, severity levels and confidence levels, and generating a rice disease and pest identification result based on the same. The application solves the technical problems of low identification efficiency and low identification accuracy in the related art of manually patrolling and identifying rice diseases and pests.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence or other related technical fields. Specifically, it relates to a method, apparatus, electronic device and storage medium for identifying rice diseases and pests. Background Technology

[0002] As a crucial food crop, rice's yield and quality directly impact food security. Pests and diseases are key factors affecting stable and high rice yields. Achieving early, accurate, and efficient identification and control of pests and diseases is one of the core requirements of intelligent management in modern agriculture.

[0003] In related technologies, the monitoring of rice diseases and pests still mainly relies on manual field inspections, depending on experienced agricultural technicians who make judgments by visually observing leaf symptoms and insect morphology. This method is not only labor-intensive, with limited coverage and slow response, resulting in low identification efficiency, but it is also significantly affected by subjective experience, making it prone to missed diagnoses and misdiagnoses, especially in the early stages of disease or in cases of multiple infections, where the accuracy rate drops significantly.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for identifying rice diseases and pests, to at least solve the technical problems of low efficiency and low accuracy in the manual inspection and identification of rice diseases and pests in related technologies.

[0006] According to one aspect of the present invention, a method for identifying rice diseases and pests is provided, comprising: acquiring multispectral images, visible light images, real-time meteorological data, and rice growth stage information of a target rice field; preprocessing the multispectral images and the visible light images to obtain preprocessed image data; combining the real-time meteorological data and the rice growth stage information, extracting spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features from the preprocessed image data, and using an attention mechanism to integrate the spectral features, the spatiotemporal evolution features of lesions, the virtual microscopic features, and the ecological association features. The components are fused into a fused feature vector; the fused feature vector is input into a rice pest and disease identification model that has been pre-trained through transfer learning, and the model outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-constructed deep convolutional neural network model for identifying rice pests and diseases. The rice pest and disease identification model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. The rice pest and disease identification result is generated based on the pest and disease type, the severity level, and the confidence score.

[0007] Furthermore, after generating rice pest and disease identification results based on the pest and disease type, the severity level, and the confidence level, the method further includes: calling a drug use history database, a pest and disease resistance database, and an agricultural knowledge base to obtain knowledge base data; combining the knowledge base data with the real-time meteorological data, the rice growth stage information, and soil moisture content data to generate a pest and disease control strategy, wherein the pest and disease control strategy includes at least one of the following: drug combination, application dosage, and application timing.

[0008] Furthermore, after generating a pest and disease control strategy by combining the knowledge base data with the real-time meteorological data and the rice growth stage information, the method further includes: collecting image data and pest and disease regression data of the target rice field within the target time period after the implementation of the pest and disease control strategy; inputting the image data and pest and disease regression data into a residual comparison network, and calculating the treatment effect deviation of the pest and disease control strategy through the residual comparison network; and adjusting the feature extraction weights, classification parameters, and adaptation rules of the agricultural knowledge base of the rice pest and disease identification model based on the treatment effect deviation.

[0009] Further, the step of inputting the fused feature vector into a rice pest and disease identification model pre-trained by transfer learning and outputting the pest and disease type, severity level, and confidence score includes: inputting the fused feature vector as input data into the rice pest and disease identification model pre-trained by transfer learning; performing a linear transformation on the fused feature vector through the feature projection layer of the rice pest and disease identification model to output a feature space; inputting the feature space into the global average pooling layer of the rice pest and disease identification model to output pest and disease semantic features; inputting the pest and disease semantic features in parallel into the fully connected layer adapted to multiple types of rice pests and diseases, the severity classification network, and the confidence output branch network; calculating the probability distribution of pest and disease types through the fully connected layer and outputting the pest and disease type; outputting the severity level through the severity classification network; and outputting the confidence score through the confidence output branch network; and encapsulating the pest and disease type, the severity level, and the confidence score into triples for output.

[0010] Further, the step of extracting spectral features from the preprocessed image data includes: extracting spectral images of the pest and disease sensitive bands based on the image data to obtain sensitive spectral images; and calculating the band ratio features and spectral reflectance based on the sensitive spectral images to obtain the spectral features.

[0011] Furthermore, the step of extracting the spatiotemporal evolution features of lesions from the preprocessed image data includes: for the visible light image in the image data, using the background subtraction method to extract the lesion change region between consecutive frames; for the lesion change region, calculating the lesion expansion rate and the lesion diffusion direction vector to obtain the spatiotemporal evolution features of the lesions.

[0012] Further, the step of extracting virtual microscopic features and ecological association features from the preprocessed image data includes: inputting the multispectral image from the image data into a high-resolution conditional generative adversarial network to reconstruct the multispectral image into a high-resolution virtual microscopic image; identifying the lesion boundary and hyphal network structure in the high-resolution virtual microscopic image; calculating morphological indices based on the lesion boundary and the hyphal network structure, and generating the virtual microscopic features based on the morphological indices; and extracting ecological association features from the image data by combining the growth stage information and the real-time meteorological data, wherein the ecological association features include at least one of the following: infection features and environmental features.

[0013] According to another aspect of the present invention, a device for identifying rice diseases and pests is also provided, comprising: a data acquisition unit for acquiring multispectral images, visible light images, real-time meteorological data, and rice growth stage information of a target rice field; a processing unit for preprocessing the multispectral images and the visible light images to obtain preprocessed image data; and an extraction unit for combining the real-time meteorological data and the rice growth stage information to extract spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features from the preprocessed image data, and using an attention mechanism to integrate the spectral features, the spatiotemporal evolution features of lesions, the virtual microscopic features, and the ecological association features. The associated features are fused into a fused feature vector; the identification unit is used to input the fused feature vector into a rice disease and pest identification model that has been pre-trained by transfer learning, and output the disease and pest type, severity level and confidence level. The rice disease and pest identification model is a pre-constructed deep convolutional neural network model for identifying rice diseases and pests. The rice disease and pest identification model includes at least a fully connected layer adapted to multiple types of rice diseases and pests, a severity grading network and a confidence output branch network. The severity grading network is used to evaluate the severity of rice diseases and pests; the generation unit is used to generate rice disease and pest identification results based on the disease and pest type, the severity level and the confidence level.

[0014] Furthermore, the rice pest and disease identification device further includes: a first calling module, used to call a drug use history database, a pest and disease resistance database, and an agricultural knowledge base to obtain knowledge base data; and a first generation module, used to combine the knowledge base data with the real-time meteorological data, the rice growth stage information, and the soil moisture content data to generate a pest and disease treatment strategy, wherein the pest and disease treatment strategy includes at least one of the following: drug combination, application dosage, and application timing.

[0015] Furthermore, the rice pest and disease identification device further includes: a first acquisition module, used to acquire image data and pest and disease regression data of the target paddy field within a target time period after the implementation of the pest and disease treatment strategy; a first calculation module, used to input the image data and pest and disease regression data into a residual comparison network, and calculate the treatment effect deviation of the pest and disease treatment strategy through the residual comparison network; and a first adjustment module, used to adjust the feature extraction weights, classification parameters, and adaptation rules of the agricultural knowledge base of the rice pest and disease identification model according to the treatment effect deviation.

[0016] Further, the identification unit includes: a first transformation module, used to input the fused feature vector as input data into the rice disease and pest identification model pre-trained by transfer learning, and to perform a linear transformation on the fused feature vector through the feature projection layer of the rice disease and pest identification model to output a feature space; a first output module, used to input the feature space into the global average pooling layer of the rice disease and pest identification model to output disease and pest semantic features; a second output module, used to input the disease and pest semantic features in parallel into the fully connected layer, the severity grading network, and the confidence output branch network adapted to multiple types of rice diseases and pests, to calculate the probability distribution of disease and pest types through the fully connected layer and output the disease and pest types, to output the severity level through the severity grading network, and to output the confidence level through the confidence output branch network; and a first encapsulation module, used to encapsulate the disease and pest types, the severity level, and the confidence level into triples for output.

[0017] Furthermore, the extraction unit includes: a first extraction module, used to extract the spectral image of the sensitive band of pests and diseases based on the image data to obtain a sensitive spectral image; and a second calculation module, used to calculate the band ratio feature and spectral reflectance based on the sensitive spectral image to obtain the spectral features.

[0018] Furthermore, the extraction unit includes: a second extraction module, used to extract the lesion change region between consecutive frames of images using the background subtraction method for the visible light image in the image data; and a third calculation module, used to calculate the lesion expansion rate and lesion diffusion direction vector for the lesion change region to obtain the spatiotemporal evolution characteristics of the lesion.

[0019] Further, the extraction unit includes: a first reconstruction module, used to input the multispectral image in the image data into a high-resolution conditional generative adversarial network to reconstruct the multispectral image into a high-resolution virtual microscopic image; a first recognition module, used to recognize the lesion boundary and hyphal network structure in the high-resolution virtual microscopic image; a second generation module, used to calculate morphological indicators based on the lesion boundary and the hyphal network structure, and generate the virtual microscopic features based on the morphological indicators; and a third extraction module, used to combine the growth stage information and the real-time meteorological data to extract ecological association features from the image data, wherein the ecological association features include at least one of the following: infection features and environmental features.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described methods for identifying rice diseases and pests.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described methods for identifying rice diseases and pests.

[0022] In this application, the following steps are taken: Multispectral images, visible light images, real-time meteorological data, and rice growth stage information of the target paddy field are collected; the multispectral and visible light images are preprocessed to obtain preprocessed image data; combined with real-time meteorological data and rice growth stage information, spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features are extracted from the preprocessed image data; and an attention mechanism is used to fuse the spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features into a fused feature vector. The feature vectors are fused and input into a pre-trained rice pest and disease identification model, which outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-built deep convolutional neural network model for identifying rice pests and diseases. The rice pest and disease identification model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases, and the rice pest and disease identification results are generated based on the pest and disease type, severity level, and confidence score.

[0023] In this application, multi-source rice data is collected, and features are extracted from spectral, dynamic, microscopic, and ecological data to obtain multi-dimensional pest and disease characteristics. A rice pest and disease identification model is then used for intelligent identification to determine the type and severity of pests and diseases, thereby achieving intelligent identification of rice pests and diseases, improving the efficiency and accuracy of identification, and solving the technical problem of low efficiency and low accuracy in the manual inspection and identification of rice pests and diseases in related technologies. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for identifying rice diseases and pests is shown.

[0026] Figure 2 This is a flowchart of an optional method for identifying rice diseases and pests according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional rice pest and disease identification process according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an optional rice pest and disease identification device according to an embodiment of the present invention;

[0029] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) that performs an optional method for identifying rice diseases and pests according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should be noted that the rice disease and pest identification method and apparatus in this application can be used in the field of artificial intelligence for disease and pest identification based on multi-source data and artificial intelligence, and can also be used in any field other than the field of blockchain technology for disease and pest identification based on multi-source data and artificial intelligence. This application does not limit the application field of the rice disease and pest identification method and apparatus.

[0033] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0034] The following embodiments of the present invention can be applied to various rice disease and pest identification systems / applications / equipment. The present invention collects multi-source data, integrates multi-source features such as spectral characteristics, spatiotemporal evolution characteristics of lesions, and ecological association characteristics, eliminates light interference through adaptive histogram equalization, synthesizes microscopic features of lesions using artificial intelligence, extracts lesion expansion rate and direction vectors from continuous frame images, and then associates rice genotype data with meteorological factors to construct a comprehensive feature system. Through deep learning networks, it accurately identifies the type and severity of diseases and pests, improves early identification and anti-interference capabilities, and enhances the efficiency and accuracy of rice disease and pest identification.

[0035] The present invention will now be described in detail with reference to various embodiments.

[0036] Example 1

[0037] According to an embodiment of the present invention, an embodiment of a method for identifying rice diseases and pests is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for identifying rice diseases and pests is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0039] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the rice pest and disease identification method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned rice pest and disease identification method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0042] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0043] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for identifying rice diseases and pests shown is implemented by a rice disease and pest identification system.

[0044] Figure 2 This is a flowchart of an optional method for identifying rice diseases and pests according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0045] Step S201: Collect multispectral images, visible light images, real-time meteorological data, and rice growth stage information of the target paddy field.

[0046] In step S201 above, an unmanned aerial vehicle (UAV) equipped with a multispectral sensor and a visible light camera flies at a preset altitude (e.g., 5–10 meters) along a planned route over the target paddy field at a constant speed, simultaneously acquiring multispectral and visible light images of the paddy field area.

[0047] Multispectral images refer to image data collected in the 450nm to 950nm wavelength range, divided into 16 discrete bands (including visible light bands of 450nm, 550nm, and 670nm, and near-infrared bands of 750nm, 800nm, 850nm, 900nm, and 950nm). Each image corresponds to the reflectance information of ground objects at a specific wavelength, which is used to reflect changes in the physiological state of rice leaves, such as chlorophyll content, water stress, and early spectral response characteristics of lesions.

[0048] The visible light image is a standard RGB three-channel image with a resolution of 2048×2048 pixels. It is used to provide intuitive visual information such as color, shape, and texture to assist in subsequent image segmentation and lesion boundary recognition.

[0049] Meanwhile, real-time meteorological data, including ambient temperature, relative humidity, precipitation, wind speed, and sunshine duration, is collected using meteorological data acquisition tools deployed around the paddy fields. The sampling frequency can be set to once every 5 minutes and is synchronized with the image acquisition timestamp. This meteorological data is used to characterize the field microclimate environment and provides a basis for subsequent environmental causal analysis.

[0050] In addition, farmers can input or retrieve rice growth stage information for the target paddy field through mobile applications or field management platforms. The growth stages can include tillering stage, jointing stage, booting stage, heading stage, and grain filling stage. This information is used to determine the physiological sensitivity of the current plant. For example, the sensitivity to rice blast is significantly higher during the booting stage than during the tillering stage.

[0051] All collected data is transmitted to edge computing nodes or cloud servers via wireless communication modules and precisely time-aligned according to the image acquisition time to ensure that multispectral images, visible light images, meteorological data and growth stage information form complete data pairs in spatial coordinates and time dimensions.

[0052] Through the above steps, a multimodal and spatiotemporally synchronized input foundation is provided for pest and disease identification, avoiding the information loss caused by relying on a single image source. By synchronously collecting environmental and physiological parameters, an interpretable identification context is constructed, improving the system's adaptability to complex field scenarios.

[0053] Step S202: Preprocess the multispectral image and the visible light image to obtain preprocessed image data.

[0054] In step S202 above, the multispectral image and visible light image are preprocessed, which may include: ensuring spatial comparability of multi-source images by unifying resolution; improving the visual contrast of visible light images under complex lighting conditions by adaptive histogram equalization; eliminating numerical drift caused by sensor and environment through spectral normalization; and focusing on the rice plant area by semantic segmentation and background masking to eliminate non-target interference such as soil, water, and shadows, significantly improving the accuracy and efficiency of subsequent feature extraction. This yields the preprocessed image data.

[0055] By performing standardization, illumination correction, and semantic segmentation on multispectral and visible light images, preprocessed image data with consistent structure, clean background, and stable brightness is generated. This provides high-quality, low-noise, and high signal-to-noise ratio data input for subsequent multidimensional feature fusion, effectively solving the recognition interference problem caused by changes in illumination and complex backgrounds in field-collected images.

[0056] Step S203: Combining real-time meteorological data and rice growth stage information, spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features are extracted from the preprocessed image data. Then, the spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features are fused into a fused feature vector through an attention mechanism.

[0057] In step S203 above, feature extraction is performed on multi-source data to achieve a leap from single image information to multimodal, multi-dimensional, and environment-coupled feature expression. Feature extraction mainly includes four types of features: spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features. Virtual microscopic features are used to make up for the shortcomings of insufficient resolution of macroscopic images. Ecological association features are used to introduce agronomic knowledge and environmental variables to improve the interpretability and context adaptability of recognition. Subsequently, an attention mechanism is used to achieve intelligent weighted fusion of heterogeneous features to avoid information redundancy and noise interference, and generate a fusion feature vector with strong discriminative ability.

[0058] By combining real-time meteorological data with rice growth stage information, spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features are extracted from preprocessed image data. These features are then dynamically fused into a high-dimensional fusion feature vector using an attention mechanism. This constructs a four-dimensional collaborative perception system that surpasses traditional image recognition methods, significantly enhancing the ability to identify early-stage diseases, complex diseases, and diseases under complex environments. This provides a feature foundation with high information density and strong semantic association for subsequent accurate identification.

[0059] Furthermore, the steps for extracting spectral features from the preprocessed image data include: extracting spectral images of the sensitive bands of pests and diseases from the image data to obtain sensitive spectral images; and calculating the band ratio features and spectral reflectance based on the sensitive spectral images to obtain spectral features.

[0060] Specifically, the first step is to select disease and pest sensitive bands with high responsiveness to rice diseases and pests from the preprocessed image data. These sensitive bands refer to specific wavelength ranges within the 450–950 nm multispectral range that are significantly correlated with changes in the physiological state of rice leaves. These include: the 550 nm green light band, which is sensitive to changes in chlorophyll content, with increased reflectivity when chlorophyll is degraded by disease; the 670 nm red light band, which is related to photosynthetic efficiency, with reduced absorption in lesion areas; the 750 nm, 800 nm, and 850 nm near-infrared bands, which reflect the integrity of leaf cell structure, with a significant increase in reflectivity when cells rupture due to disease; and the 900 nm and 950 nm water absorption bands, which are sensitive to changes in leaf water content, allowing detection of abnormal transpiration caused by drought stress or disease.

[0061] From the preprocessed multispectral image, single-band images corresponding to the above seven bands are cropped to form a set of sensitive spectral images. Each image is a 512×512 pixel grayscale image, retaining the pixel values ​​of the rice plant area. Subsequently, based on the sensitive spectral images, the band ratio features and spectral reflectance are calculated to generate the final spectral features.

[0062] In another optional embodiment, the band ratio feature is calculated as follows: pixel-level division is performed on any two sensitive band images to form a ratio image, and then the mean value of the rice plant area in the image is calculated as the feature value. Specific ratios include: NDVI = (800nm–670nm) / (800nm+670nm), used to assess vegetation health; SR = 800nm / 670nm, enhancing the distinction between greenness and disease; NDRE = (800nm–705nm) / (800nm+705nm), more sensitive to early chlorophyll loss; and WI = 900nm / 950nm, reflecting the degree of leaf water stress.

[0063] In another optional embodiment, spectral reflectance is calculated as follows: for each sensitive band, the reflectance values ​​of all pixels within the masked area of ​​the rice plant are extracted from a single image, and their average value is calculated to form a scalar value of reflectance for that band. This directly characterizes the radiation response capability of the target area at specific wavelengths, reflects changes in the content of biochemical components such as chlorophyll, water, and cellulose, and is a primary physical quantity for identifying physiological abnormalities.

[0064] Finally, the band ratio feature is concatenated with the spectral reflectance to form a spectral feature vector, which serves as the input for the subsequent feature fusion module.

[0065] Furthermore, the steps for extracting the spatiotemporal evolution features of lesions from the preprocessed image data include: for visible light images in the image data, using background subtraction to extract the lesion change regions between consecutive frames; for the lesion change regions, calculating the lesion expansion rate and lesion diffusion direction vector to obtain the spatiotemporal evolution features of the lesions.

[0066] Specifically, when extracting the spatiotemporal fireworks features of lesions, the first step is to process the visible light image sequence in the preprocessed image data using the background subtraction method to extract the lesion change regions. The background subtraction method refers to using the first frame of five consecutive visible light images as the reference background frame, and performing pixel-level difference operations on each subsequent frame with the background frame to obtain a difference image. In the difference image, areas where the absolute value of pixel grayscale change exceeds a preset threshold are marked as change regions. These regions may correspond to lesion expansion, pest activity, or environmental disturbance. To suppress pseudo-changes caused by light fluctuations and sensor noise, morphological closing operations (structural element: 3×3 circular kernel) are performed on the difference image to connect broken regions, and small noise blocks with an area of ​​less than 100 pixels are removed through connected component analysis. The finally retained region is the lesion change region, which is a binary mask image where a pixel value of 1 indicates a dynamic lesion expansion region and 0 indicates a region with no change.

[0067] Secondly, for the extracted lesion change areas, the lesion expansion rate and lesion diffusion direction vector are calculated to constitute the spatiotemporal evolution characteristics of the lesions. The lesion expansion rate is defined as the total area increase of the lesion change area divided by the time interval within a time window of multiple consecutive frames (e.g., a time interval of 10 minutes), with units of pixels² / minute. By calculating the lesion expansion rate, the activity level of the disease is quantified, distinguishing between rapidly spreading diseases and slowly developing damage, thus improving the ability to distinguish the development stage of the disease. The lesion diffusion direction vector is defined as the displacement vector of the centroid coordinates of the lesion change area over time. By calculating the coordinate sequence of the lesion centroid from frame 1 to frame N, a linear trend is fitted to obtain a two-dimensional direction vector, representing the main expansion direction of the lesion in the image plane. This captures the spatial propagation pattern of the disease, helps to distinguish the directional characteristics of fungal spores spreading by wind from the non-directional distribution of insect pest aggregation damage, and improves the distinguishability of disease type identification.

[0068] Furthermore, the steps for extracting virtual microscopic features and ecological association features from the preprocessed image data include: inputting the multispectral image from the image data into a high-resolution conditional generative adversarial network to reconstruct the multispectral image into a high-resolution virtual microscopic image; identifying lesion boundaries and hyphal network structures in the high-resolution virtual microscopic image; calculating morphological indicators based on lesion boundaries and hyphal network structures, and generating virtual microscopic features based on morphological indicators; and extracting ecological association features from the image data by combining growth stage information and real-time meteorological data, wherein the ecological association features include at least one of the following: infection features and environmental features.

[0069] Specifically, when extracting virtual microscopic features and ecological association features, the multispectral image from the preprocessed image data is first fed into a high-resolution conditional generative adversarial network (GRAN). The generator of the GRAN consists of multi-scale residual blocks, and the discriminator uses a region-based discrimination method. Based on the input multispectral image, it outputs a high-resolution virtual microscopic image with a resolution of 2048×2048 pixels. During the training phase, the network uses paired data of real microscopic images and corresponding multispectral images for supervised learning, so that the generated image approximates the real microscopic image in terms of texture, edge, and structure.

[0070] Secondly, the generated high-resolution virtual microscopic images are used to identify disease structures. A pre-trained model is used to segment the images at the pixel level, outputting a binary mask of the lesion boundary and the mycelial network structure. The lesion boundary refers to the spatial boundary between the diseased area and healthy tissue, reflecting the extent of disease spread. The mycelial network structure refers to the filamentous network morphology formed by pathogenic fungi on the leaf surface. Its branch density, trunk length, and connection complexity are key criteria for disease type identification. For example, rice blast mycelia are radial, while sheath blight mycelia are interwoven in a network. Then, based on the above identification results, morphological indicators are calculated to generate virtual microscopic features. Morphological indicators may include: the ratio of lesion boundary perimeter to area, which reflects the complexity of lesion shape; the number of mycelial network branches, average branch length, and connection point density; the mycelial fractal dimension, which quantifies structural complexity; and the mycelial coverage density within the lesion, i.e., the percentage of mycelial pixels per unit area. This transforms visual structural information into quantifiable and comparable numerical features, enabling the model to distinguish the microscopic morphological differences of different pathogens and improve its ability to differentiate between diseases that are morphologically similar but have different pathogens.

[0071] Meanwhile, combining rice growth stage information with real-time meteorological data, ecological association features are extracted from the preprocessed image data. The ecological association feature is a composite vector containing at least one of the following: infection features, which calculate the theoretical susceptibility index of the plant to this type of disease based on the rice growth stage and variety, with higher values ​​indicating greater susceptibility; environmental features, which calculate the disease occurrence probability index based on real-time meteorological data (temperature, humidity, and rainfall), for example, when the temperature is 18–28℃, the relative humidity is >85%, and there is no rain for 24 consecutive hours, the probability of rice blast occurrence is 0.85.

[0072] Step S204: Input the fused feature vector into the rice pest and disease identification model that has been pre-trained by transfer learning, and output the pest and disease type, severity level and confidence level.

[0073] It should be noted that the above-mentioned rice disease and pest identification model is a pre-built deep convolutional neural network model for identifying rice diseases and pests. The rice disease and pest identification model includes at least a fully connected layer adapted to multiple types of rice diseases and pests, a severity rating network, and a confidence output branch network. The severity rating network is used to assess the severity of rice diseases and pests.

[0074] In step S204 above, the fused feature vector is fed into a rice pest and disease identification model that has been pre-trained through transfer learning. This model uses a deep convolutional neural network as its base network, and its terminal structure contains three parallel output branches, each undertaking a different task: a fully connected layer adapting to multiple types of rice pests and diseases. This layer is a fully connected neural network layer, with a 64-dimensional fused feature vector as input and 15 neurons as output, corresponding to 15 common rice pests and diseases (such as rice blast, sheath blight, rice planthopper, and rice leaf roller); after normalization using a mathematical function, the probability distribution of each category is output, and the category with the highest probability corresponds to the pest or disease type. This achieves accurate multi-category classification of major rice pests and diseases, breaking through the limitation of traditional models that only support 3–5 categories, and covering the main threats in the field.

[0075] The severity grading network is an independent fully connected subnetwork that shares the same front-layer features as the pest and disease classification branch, but has independent weights. The input is the same 64-dimensional fused feature vector, and the output consists of 5 neurons, corresponding to severity levels 1–5 (1 = no symptoms, 5 = large-scale death and plant lodging). The output is constrained to the [0,1] interval by an activation function and then mapped to discrete severity levels through threshold division (e.g., >0.8 for level 5). Based on the identification of disease types, the network quantifies the degree of damage, providing key parameters for subsequent treatment plans.

[0076] The confidence output branch network is a single-neuron fully connected layer that outputs a real value (0-1) representing the model's confidence level in the current recognition result. This branch is trained independently using a binary cross-entropy loss function, with its input still being the fused feature vector, and its output being a probability estimate of whether the prediction is reliable. This provides the model with the ability to self-evaluate its own predictions; when the confidence level falls below a preset threshold (e.g., 0.7), it can trigger a manual review or data re-collection mechanism, improving system reliability.

[0077] By inputting the fused feature vector into a rice pest and disease identification model that has been pre-trained through transfer learning, and by utilizing its built-in fully connected layer adapted to multiple types of rice pests and diseases, severity classification network, and confidence output branch network, the synchronous and collaborative output of pest and disease type, severity, and identification confidence is achieved, which significantly improves the completeness, practicality, and decision support capabilities of the identification results.

[0078] Further, the steps of inputting the fused feature vector into a pre-trained rice disease and pest identification model using transfer learning, and outputting the disease / pest type, severity level, and confidence score, include: inputting the fused feature vector as input data into the pre-trained rice disease and pest identification model using transfer learning; performing a linear transformation on the fused feature vector through the feature projection layer of the rice disease and pest identification model to output a feature space; inputting the feature space into the global average pooling layer of the rice disease and pest identification model to output disease / pest semantic features; inputting the disease / pest semantic features in parallel into a fully connected layer adapted for multiple types of rice diseases and pests, a severity grading network, and a confidence output branch network; calculating the probability distribution of disease / pest types through the fully connected layer and outputting the disease / pest type; outputting the severity level through the severity grading network; and outputting the confidence score through the confidence output branch network; and encapsulating the disease / pest type, severity level, and confidence score into triples for output.

[0079] First, a 64-dimensional fused feature vector is fed into a rice disease and pest identification model pre-trained through transfer learning. The feature projection layer is a fully connected layer whose weights are fine-tuned during the transfer learning stage. It has an input dimension of 64 and an output dimension of 512. The feature projection layer performs a linear transformation on the input fused feature vector, outputting a feature space. Its function is to map the high-dimensional semantic information after multi-source fusion to a feature subspace more suitable for classification, thereby enhancing class separability.

[0080] Secondly, the feature space is input into the global average pooling layer. The global average pooling layer calculates the average value of each channel (512 in total) in the feature space in the spatial dimension (e.g., 7×7) and outputs a 512-dimensional semantic feature vector of pests and diseases. This operation eliminates the redundancy of the spatial dimension, preserves the semantic response strength of the channel dimension, makes the output insensitive to changes in the location of lesions, and enhances the robustness of the model.

[0081] Next, the semantic features of pests and diseases are input in parallel to a fully connected layer adapted for multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The fully connected layer adapted for multiple types of rice pests and diseases is designed for 15 types of rice pests and diseases (such as rice blast, sheath blight, and rice planthopper), and includes a fully connected layer with an output dimension of 15, followed by an activation function; the output is a 15-dimensional probability distribution vector, with each dimension corresponding to the prediction confidence of a type of pest or disease; the category label corresponding to the maximum probability value is taken as the output pest or disease type. The severity grading network is an independent fully connected network, with 512-dimensional semantic features of pests and diseases as input and a 5-dimensional vector as output, corresponding to severity levels 1–5 (1=weak, 5=severe). The activation function is used to output the membership probability of each level, and the final severity level is determined by weighted summation or the maximum probability method. The confidence output branch network is a single-neuron fully connected layer. It takes 512-dimensional pest and disease semantic features as input and outputs a scalar value in the range [0,1], representing the model's confidence in the identification result. Finally, the three outputs—pest and disease type, severity level, and confidence—are combined and encapsulated to obtain a triple as the final output.

[0082] By inputting the fused feature vector into the rice pest and disease identification model optimized by transfer learning, and then sequentially passing it through the feature projection layer, the global average pooling layer, and the parallel output branch, the model achieves synchronous high-precision output of pest and disease type, severity level, and confidence level, forming a unified triplet identification result and improving the accuracy of pest and disease identification.

[0083] Step S205: Generate rice pest and disease identification results based on pest and disease type, severity level, and confidence level.

[0084] In step S205 above, based on the pest and disease type, severity level and confidence level output by the model, and combined with the original rice image data, pest and disease labeling is performed to generate the final identification results of rice pests and diseases.

[0085] Furthermore, after generating rice pest and disease identification results based on pest and disease type, severity level, and confidence level, the process also includes: calling the historical database of drug use, the pest and disease resistance database, and the agricultural knowledge base to obtain knowledge base data; combining the knowledge base data with real-time meteorological data, rice growth stage information, and soil moisture content data to generate pest and disease control strategies, wherein the pest and disease control strategies include at least one of the following: drug combination, application dosage, and application timing.

[0086] Specifically, external databases are accessed, including a pesticide use history database, a pest and disease resistance database, and an agricultural knowledge base. The pesticide use history database stores the types of pesticides used in various paddy fields in the target area over a period of time, including application time, frequency, dosage, and pest and disease recurrence after application, identifying whether the same pesticide has been repeatedly used in the same plot and assessing resistance risk. The pest and disease resistance database contains records of known rice pests and diseases' resistance to mainstream pesticides, categorized and labeled by region, strain type, and resistance level. The agricultural knowledge base stores agronomic standards in a structured manner, including recommended control programs for various pests and diseases, pesticide incompatibilities, safety intervals, differences in pesticide sensitivity at different growth stages of rice, and biological control alternatives, used to match compliant pesticide application strategies to the current rice pest and disease situation.

[0087] Based on the pest and disease type and resistance database, pesticides with resistance records exceeding a preset level in the area are prioritized for exclusion (e.g., if pesticide A has been used continuously in the field within the past year, the system automatically excludes it as the main pesticide); usable pesticide combinations are screened from the agricultural knowledge base. The dosage is dynamically adjusted according to the severity level and rice growth stage: if it is early stage (level 1–2), the standard recommended dosage is applied; if it is moderate to severe stage (level ≥3), the dosage is increased proportionally within the safety limit (e.g., +15%); the spray concentration is automatically reduced based on wind speed in real-time meteorological data (e.g., wind speed >3m / s) to prevent drift damage; the field moisture level is judged based on soil moisture content (e.g., >25%), and if the humidity is too high, the water consumption per unit area is reduced to improve pesticide adhesion efficiency.

[0088] Based on real-time meteorological data, select a time window with no rain, humidity <80%, and temperature 18–28℃ within a certain period of time; combined with the rice growth stage, avoid applying pesticides during the flowering period (which is prone to pesticide damage), and prioritize applying pesticides in the early morning or evening; if the pesticide use history database shows that the last application of the plot was less than 7 days ago, then trigger "interval period verification" and delay the application until the safe interval period is met.

[0089] Finally, the system encapsulates the generated pesticide combinations, application dosages, and application timings into a structured pest and disease control strategy, and outputs it to mobile devices or smart agricultural machinery control terminals.

[0090] Furthermore, after generating pest and disease control strategies by combining knowledge base data with real-time meteorological data and rice growth stage information, the process also includes: collecting image data and pest and disease regression data of the target rice field within the target time period after the implementation of the pest and disease control strategies; inputting the image data and pest and disease regression data into a residual comparison network, and calculating the treatment effect deviation of the pest and disease control strategies through the residual comparison network; and adjusting the feature extraction weights, classification parameters, and agricultural knowledge base adaptation rules of the rice pest and disease identification model based on the treatment effect deviation.

[0091] Specifically, firstly, after the pest and disease control strategy is implemented, the system collects image data and pest and disease regression data of the target paddy field within the target time period (i.e., 72 hours after pesticide application). The image data consists of visible light or multispectral images uploaded by drones or farmers' mobile devices, covering the original identification area, with a resolution of no less than 1080p, and the timestamps and location information are strictly aligned. The pest and disease regression data includes quantitative indicators such as the rate of change in lesion area, the percentage decrease in leaf lesion coverage, and the percentage reduction in insect population density, which are manually or automatically identified and entered into the system by field monitoring equipment or agricultural technicians. This set of data serves as a post-treatment status characterization and is used for comparative analysis with the pre-treatment status.

[0092] Secondly, the aforementioned image data and pest and disease regression data are used as inputs and fed into the residual contrast network. The residual contrast network is a two-input convolutional neural network structure containing two parallel encoding branches: one branch processes the pre-treatment image (i.e., the original image used to generate the treatment strategy); the other branch processes the post-treatment image; the two branches share a weight structure and extract semantic features of the lesion areas in the images respectively; through feature difference calculation, a residual map is output, which is the pixel-level change response of the lesion area in the two frames of images; at the same time, pest and disease regression data (such as the lesion area reduction rate) is used as a supervision signal and aligned with the residual map to train the network to learn the mapping relationship between image changes and the actual degree of regression; finally, a scalar value is output, which is the treatment effect deviation.

[0093] Furthermore, three adaptive adjustments are made based on the deviation in treatment effect: the feature extraction weights of the rice pest and disease identification model are adjusted. For example, if the deviation in treatment effect is >15% (such as the actual regression rate being much lower than expected), it indicates that the model has a systematic misjudgment of the severity or type of disease; the weights of the last three convolutional layers in the backbone network of the model are fine-tuned through transfer learning to enhance the sensitivity to the features of residual lesions after treatment.

[0094] Adjust the classification parameters of the rice disease and pest identification model: If a certain type of disease or pest shows high bias after multiple treatments, it indicates that the original classification boundary is blurred; retrain the weight matrix and bias term of the fully connected layer, optimize the decision boundary between categories, and use the weighted cross-entropy loss function to assign a higher penalty coefficient to high-biased samples.

[0095] Adjust the adaptation rules of the agricultural knowledge base: If a certain type of pesticide combination generally shows a treatment effect deviation of >20% under specific growth stages and weather conditions, the system will automatically mark the combination-scenario combination as inefficient; update the recommendation rules in the agricultural knowledge base, and at the same time, include the case in the resistance evolution and environmental sensitivity label library.

[0096] By collecting images and fading data after treatment, the deviation of the treatment effect is calculated through a residual comparison network. Based on this, the feature extraction weights, classification parameters, and adaptation rules of the agricultural knowledge base of the rice pest and disease identification model are dynamically adjusted, realizing the continuous self-learning and performance iteration of the intelligent identification and intelligent treatment system.

[0097] Through the above steps, multispectral images, visible light images, real-time meteorological data, and rice growth stage information of the target paddy field are collected. The multispectral and visible light images are preprocessed to obtain preprocessed image data. Combined with real-time meteorological data and rice growth stage information, spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features are extracted from the preprocessed image data. The spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features are fused into a fused feature vector through an attention mechanism. The fused feature vector is input into a rice pest and disease identification model that has been pre-trained by transfer learning. The model outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-constructed deep convolutional neural network model for identifying rice pests and diseases. The rice pest and disease identification model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. The rice pest and disease identification results are generated based on the pest and disease type, severity level, and confidence score.

[0098] In this embodiment, multi-source rice data is collected, and features are extracted from spectral, dynamic, microscopic, and ecological data to obtain multi-dimensional pest and disease characteristics. A rice pest and disease identification model is then used for intelligent identification to determine the type and severity of pests and diseases. This achieves intelligent identification of rice pests and diseases, improving the efficiency and accuracy of identification. It also solves the technical problem of low efficiency and low accuracy in the manual inspection and identification of rice pests and diseases in related technologies.

[0099] The following describes in detail another optional implementation method.

[0100] Figure 3 This is a schematic diagram of an optional rice pest and disease identification process according to an embodiment of the present invention, as shown below. Figure 3 As shown, the specific process for identifying rice diseases and pests includes:

[0101] Step 1: Data Collection;

[0102] Acquire multispectral images and continuous frame visible light images of paddy fields containing the 450nm-950nm wavelength band, and simultaneously collect real-time meteorological data and information on rice growth stages.

[0103] Step 2, Image preprocessing;

[0104] Images were standardized to a resolution of 512×512, and adaptive histogram equalization was used to eliminate lighting differences and shadow interference. A semantic segmentation model was used to separate rice plants from the soil background to reduce irrelevant interference.

[0105] Step 3: Multi-dimensional feature extraction and fusion;

[0106] Spectral feature extraction: Extract features such as spectral reflectance and band ratio of rice leaves from multispectral images, and focus on screening differential information in sensitive bands for pests and diseases;

[0107] Spatiotemporal feature extraction: extracting spatiotemporal evolution features such as lesion expansion rate and direction vector from preprocessed images of multiple consecutive frames;

[0108] Virtual microscopic images of lesions in the 750nm-900nm band were synthesized to enhance the mycelial characteristics of pathogens, and ecological association characteristics were generated by combining real-time meteorological data.

[0109] Step 4: Pest and disease identification using the model (corresponding to the pest and disease identification model described above);

[0110] The model is a deep convolutional neural network model pre-trained based on sample construction and sample augmentation.

[0111] For model building, a deep convolutional neural network was selected as the base network. Since the feature distribution of rice disease and pest images differs greatly from that of general images, ordinary classification layers cannot effectively map the features extracted by the base network to the disease and pest categories. Therefore, a new fully connected layer was added, specifically designed for multiple types of rice diseases and pests, with output dimensions that fully match the classification requirements. At the same time, branches for severity grading and confidence calculation were added, covering all common rice diseases and pests that meet the full output requirements.

[0112] Sample augmentation, first according to the formula For routine pest and disease samples, perform a ±30° random rotation, and then use the formula... After adding Gaussian noise, then according to the RGB color dithering formula Color dithering is performed, where α is the contrast adjustment coefficient ((α∈[0.8,1.2]), randomly selected), and β is the brightness adjustment offset ((β∈[-20,20]), randomly selected), simulating color changes under different lighting conditions to enhance the environmental adaptability of the samples. Finally, rare pest and disease samples are generated using cGAN to expand the diversity of the dataset.

[0113] The fused feature vectors are input into the model in batches. The learner rate is dynamically adjusted by configuring the optimizer and cosine annealing scheduling strategy. The weighted cross-entropy loss function is used to optimize the training. The batch size is set to 64, and the training is carried out for 50 rounds. In each round, the F1 score, accuracy, recall, and precision are calculated using an independent validation set. If the validation loss does not decrease for 10 consecutive rounds, the training is terminated and the best weights are saved.

[0114] Step 5: Generation of dynamic response plan;

[0115] Based on the pest and disease identification results output by the model, the system calls upon the regional pesticide use history database, pathogen resistance database, and agricultural knowledge base, and simultaneously acquires real-time meteorological data, rice growth stage, and soil moisture information. Through a multi-agent reinforcement learning model, with the goal of minimizing the rate of resistance development, it generates resistance avoidance agent combinations and application dosages, and adjusts the application concentration based on wind speed data. According to the severity level of pests and diseases, it provides agricultural guidance such as adjusting irrigation cycles and optimizing fertilizer ratios. For pest types, it provides collaborative solutions for biological and chemical control. It integrates pesticide application plans, agricultural adjustment suggestions, and application timing to generate a visualized treatment report.

[0116] Step Six: Implementation of the Disposal Plan;

[0117] Step 7: Performance monitoring and feedback optimization.

[0118] Images of paddy fields and pest and disease regression data were collected within a preset time period after pesticide application. The deviation of the treatment effect was quantified using a residual comparison network. The formula for calculating the deviation of the treatment effect is as follows: Based on the bias data, transfer learning was used to fine-tune the feature extraction weights and classification parameters of the model, and to optimize the feature fusion weights and loss function coefficients.

[0119] The treatment plan database is updated based on feedback results, and the appropriate matching rules for pesticide combinations and agricultural guidance parameters are adjusted to form a continuous optimization closed loop.

[0120] This invention collects multi-source data, integrates multi-source features such as spectral features, spatiotemporal evolution features of lesions, and ecological association features, eliminates light interference through adaptive histogram equalization, synthesizes microscopic features of lesions using artificial intelligence, extracts the lesion expansion rate and direction vector by combining continuous frame images, and then associates rice genotype data with meteorological factors to construct a comprehensive feature system. Through deep learning networks, it accurately identifies the type and severity of pests and diseases, improves the early identification and anti-interference ability of pests and diseases, and enhances the efficiency and accuracy of rice pest and disease identification.

[0121] The following is a detailed description with reference to another embodiment.

[0122] Example 2

[0123] The rice pest and disease identification device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the aforementioned method embodiment, and will not be repeated here.

[0124] Figure 4 This is a schematic diagram of an optional rice pest and disease identification device according to an embodiment of the present invention, such as... Figure 4 As shown, the rice pest and disease identification device may include: a collection unit 41, a processing unit 42, an extraction unit 43, an identification unit 44, and a generation unit 45, wherein,

[0125] Acquisition unit 41 is used to acquire multispectral images, visible light images, real-time meteorological data and rice growth stage information of the target paddy field;

[0126] Processing unit 42 is used to preprocess multispectral images and visible light images to obtain preprocessed image data;

[0127] Extraction unit 43 is used to combine real-time meteorological data and rice growth stage information to extract spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features from the preprocessed image data, and to fuse the spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features into a fused feature vector through an attention mechanism;

[0128] The identification unit 44 is used to input the fused feature vector into the rice disease and pest identification model that has been pre-trained by transfer learning, and output the disease and pest type, severity level and confidence level. The rice disease and pest identification model is a pre-built deep convolutional neural network model for identifying rice diseases and pests. The rice disease and pest identification model includes at least a fully connected layer adapted to multiple types of rice diseases and pests, a severity classification network and a confidence output branch network. The severity classification network is used to evaluate the severity of rice diseases and pests.

[0129] The generation unit 45 is used to generate rice pest and disease identification results based on pest and disease type, severity level and confidence level.

[0130] The aforementioned rice disease and pest identification device acquires multispectral images, visible light images, real-time meteorological data, and rice growth stage information of the target rice field through the acquisition unit 41; the processing unit 42 preprocesses the multispectral and visible light images to obtain preprocessed image data; and the extraction unit 43 combines the real-time meteorological data and rice growth stage information to extract spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features from the preprocessed image data, and then uses an attention mechanism to fuse the spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features into a fused feature. The fused feature vector is input into the rice pest and disease identification model, which has been trained by transfer learning, through the identification unit 44. The model outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-built deep convolutional neural network model for identifying rice pests and diseases. The rice pest and disease identification model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. The generation unit 45 generates the rice pest and disease identification result based on the pest and disease type, severity level, and confidence score.

[0131] In this embodiment, multi-source rice data is collected, and features are extracted from spectral, dynamic, microscopic, and ecological data to obtain multi-dimensional pest and disease characteristics. A rice pest and disease identification model is then used for intelligent identification to determine the type and severity of pests and diseases. This achieves intelligent identification of rice pests and diseases, improving the efficiency and accuracy of identification. It also solves the technical problem of low efficiency and low accuracy in the manual inspection and identification of rice pests and diseases in related technologies.

[0132] Furthermore, the rice pest and disease identification device also includes: a first calling module, used to call the drug use history database, the pest and disease resistance database, and the agricultural knowledge base to obtain knowledge base data; and a first generation module, used to combine the knowledge base data with real-time meteorological data, rice growth stage information, and soil moisture content data to generate pest and disease control strategies, wherein the pest and disease control strategies include at least one of the following: drug combination, application dosage, and application timing.

[0133] Furthermore, the rice pest and disease identification device also includes: a first acquisition module, used to acquire image data and pest and disease regression data of the target paddy field within the target time period after the implementation of the pest and disease treatment strategy; a first calculation module, used to input the image data and pest and disease regression data into the residual comparison network, and calculate the treatment effect deviation of the pest and disease treatment strategy through the residual comparison network; and a first adjustment module, used to adjust the feature extraction weights, classification parameters, and adaptation rules of the agricultural knowledge base of the rice pest and disease identification model according to the treatment effect deviation.

[0134] Furthermore, the identification unit includes: a first transformation module, used to input the fused feature vector as input data into a rice disease and pest identification model pre-trained by transfer learning, and to perform a linear transformation on the fused feature vector through the feature projection layer of the rice disease and pest identification model to output a feature space; a first output module, used to input the feature space into the global average pooling layer of the rice disease and pest identification model to output disease and pest semantic features; a second output module, used to input the disease and pest semantic features in parallel into a fully connected layer, a severity grading network, and a confidence output branch network adapted to multiple types of rice diseases and pests, to calculate the probability distribution of disease and pest types through the fully connected layer and output the disease and pest types, to output the severity level through the severity grading network, and to output the confidence level through the confidence output branch network; and a first encapsulation module, used to encapsulate the disease and pest type, severity level, and confidence level into triples for output.

[0135] Furthermore, the extraction unit includes: a first extraction module, used to extract the spectral image of the sensitive band of pests and diseases based on the image data, to obtain the sensitive spectral image; and a second calculation module, used to calculate the band ratio feature and spectral reflectance based on the sensitive spectral image, to obtain the spectral features.

[0136] Furthermore, the extraction unit includes: a second extraction module, used to extract the lesion change region between consecutive frames of images using the background subtraction method for the visible light image in the image data; and a third calculation module, used to calculate the lesion expansion rate and lesion diffusion direction vector for the lesion change region to obtain the spatiotemporal evolution characteristics of the lesion.

[0137] Furthermore, the extraction unit includes: a first reconstruction module, used to input the multispectral image from the image data into a high-resolution conditional generative adversarial network to reconstruct the multispectral image into a high-resolution virtual microscopic image; a first recognition module, used to recognize the lesion boundary and hyphal network structure in the high-resolution virtual microscopic image; a second generation module, used to calculate morphological indicators based on the lesion boundary and hyphal network structure, and generate virtual microscopic features based on the morphological indicators; and a third extraction module, used to extract ecological association features from the image data by combining growth stage information and real-time meteorological data, wherein the ecological association features include at least one of the following: infection features and environmental features.

[0138] It should be noted that the acquisition unit 41, processing unit 42, extraction unit 43, identification unit 44, and generation unit 45 mentioned above correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0139] The invention will now be described in conjunction with another alternative embodiment.

[0140] Example 3

[0141] The present invention can also provide an electronic device. Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for performing an optional method for identifying rice diseases and pests according to an embodiment of the present invention, such as... Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (Only one is shown) processor 502, memory 504, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0142] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0143] The processor can access information and applications stored in memory via a transmission device to execute the following steps: acquiring multispectral images, visible light images, real-time meteorological data, and rice growth stage information of the target paddy field; preprocessing the multispectral and visible light images to obtain preprocessed image data; combining real-time meteorological data and rice growth stage information, extracting spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features from the preprocessed image data, and using an attention mechanism to integrate these features. The data is fused into a fused feature vector. This fused feature vector is then input into a pre-trained rice pest and disease identification model, which outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-built deep convolutional neural network model for identifying rice pests and diseases. The model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. The rice pest and disease identification results are generated based on the pest and disease type, severity level, and confidence score.

[0144] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: call the drug use history database, the pest and disease resistance database, and the agricultural knowledge base to obtain knowledge base data; combine the knowledge base data with real-time meteorological data, rice growth stage information, and soil moisture content data to generate a pest and disease control strategy, wherein the pest and disease control strategy includes at least one of the following: drug combination, application dosage, and application timing.

[0145] The processor can access the information and application programs stored in the memory via the transmission device to perform the following steps: collecting image data and pest and disease regression data of the target paddy field within the target time period after the implementation of the pest and disease control strategy; inputting the image data and pest and disease regression data into the residual comparison network, and calculating the treatment effect deviation of the pest and disease control strategy through the residual comparison network; and adjusting the feature extraction weights, classification parameters, and adaptation rules of the agricultural knowledge base of the rice pest and disease identification model based on the treatment effect deviation.

[0146] The processor can access information and applications stored in memory via a transmission device to execute the following steps: Inputting the fused feature vector as input data into a rice pest and disease identification model pre-trained through transfer learning; performing a linear transformation on the fused feature vector through the feature projection layer of the rice pest and disease identification model to output a feature space; inputting the feature space into the global average pooling layer of the rice pest and disease identification model to output pest and disease semantic features; inputting the pest and disease semantic features in parallel into a fully connected layer adapted for multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network; calculating the probability distribution of pest and disease types through the fully connected layer and outputting the pest and disease types; outputting the severity level through the severity grading network; and outputting the confidence level through the confidence output branch network; and encapsulating the pest and disease type, severity level, and confidence level into triples for output.

[0147] The processor can access the information and application programs stored in the memory via the transmission device to perform the following steps: extracting spectral images of the sensitive bands of pests and diseases from the image data to obtain sensitive spectral images; calculating the band ratio characteristics and spectral reflectance based on the sensitive spectral images to obtain spectral characteristics.

[0148] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: for visible light images in the image data, use the background subtraction method to extract the lesion change region between consecutive frame images; for the lesion change region, calculate the lesion expansion rate and lesion diffusion direction vector to obtain the spatiotemporal evolution characteristics of the lesion.

[0149] The processor can access information and applications stored in memory via a transmission device to perform the following steps: inputting multispectral images from image data into a high-resolution conditional generative adversarial network to reconstruct the multispectral images into high-resolution virtual microscopic images; identifying lesion boundaries and hyphal network structures in the high-resolution virtual microscopic images; calculating morphological indicators based on lesion boundaries and hyphal network structures, and generating virtual microscopic features based on the morphological indicators; and extracting ecological association features from the image data by combining growth stage information and real-time meteorological data, wherein the ecological association features include at least one of the following: infection features and environmental features.

[0150] This invention provides a scheme for identifying rice diseases and pests. It collects multi-source rice data and extracts features from spectral, dynamic, microscopic, and ecological data to obtain multi-dimensional disease and pest characteristics. An intelligent identification model is then used to determine the type and severity of diseases and pests, thereby achieving intelligent identification of rice diseases and pests. This improves the efficiency and accuracy of disease and pest identification, and solves the technical problems of low efficiency and low accuracy associated with manual inspection and identification of rice diseases and pests in related technologies.

[0151] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0152] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0153] The invention will now be described in conjunction with another alternative embodiment.

[0154] Example 4

[0155] This invention also provides a computer-readable storage medium. Optionally, in this invention, the computer-readable storage medium can be used to store the program code executed by the rice pest and disease identification method provided in Embodiment 1.

[0156] Optionally, in this embodiment of the invention, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0157] This invention also provides a computer program product, which, when executed on a data processing device, is suitable for performing the steps of a method for identifying rice diseases and pests: acquiring multispectral images, visible light images, real-time meteorological data, and rice growth stage information of a target rice field; preprocessing the multispectral and visible light images to obtain preprocessed image data; combining real-time meteorological data and rice growth stage information, extracting spectral features, spatiotemporal evolution features of lesions, virtual microscopic features, and ecological association features from the preprocessed image data, and using an attention mechanism to integrate the spectral features, spatiotemporal evolution features of lesions, and virtual microscopic features... Features and ecological association features are fused into a fused feature vector. The fused feature vector is then input into a rice pest and disease identification model that has been pre-trained through transfer learning. The model outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-built deep convolutional neural network model for identifying rice pests and diseases. The model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. The rice pest and disease identification results are generated based on the pest and disease type, severity level, and confidence score.

[0158] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0160] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0164] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying rice diseases and pests, characterized in that, include: Collect multispectral images, visible light images, real-time meteorological data, and rice growth stage information of the target paddy field; The multispectral image and the visible light image are preprocessed to obtain preprocessed image data; Combining the real-time meteorological data and rice growth stage information, spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features are extracted from the preprocessed image data, and the spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features are fused into a fused feature vector through an attention mechanism; The fused feature vector is input into a rice pest and disease identification model that has been pre-trained by transfer learning, and the model outputs the pest and disease type, severity level, and confidence score. The rice pest and disease identification model is a pre-built deep convolutional neural network model for identifying rice pests and diseases. The rice pest and disease identification model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network, and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. Rice pest and disease identification results are generated based on the pest and disease type, the severity level, and the confidence level.

2. The method according to claim 1, characterized in that, After generating rice pest and disease identification results based on the pest and disease type, the severity level, and the confidence level, the method further includes: Access the drug use history database, pest and disease resistance database, and agricultural knowledge base to obtain knowledge base data; By combining the knowledge base data with the real-time meteorological data, the rice growth stage information, and the soil moisture content data, a pest and disease control strategy is generated. The pest and disease control strategy includes at least one of the following: pesticide combination, application dosage, and application timing.

3. The method according to claim 2, characterized in that, After generating a pest and disease control strategy by combining the knowledge base data, the real-time meteorological data, and the rice growth stage information, the strategy further includes: Collect image data and pest and disease decline data of the target paddy field within the target time period after the implementation of the pest and disease control strategy; The image data and pest and disease decline data are input into the residual comparison network, and the residual comparison network is used to calculate the treatment effect deviation of the pest and disease treatment strategy. The feature extraction weights, classification parameters, and adaptation rules of the agricultural knowledge base of the rice pest and disease identification model are adjusted based on the deviation of the treatment effect.

4. The method according to claim 1, characterized in that, The steps of inputting the fused feature vector into a rice disease and pest identification model pre-trained by transfer learning, and outputting the disease / pest type, severity level, and confidence level include: The fused feature vector is input as input data to the rice disease and pest identification model that has been pre-trained by transfer learning. The fused feature vector is linearly transformed through the feature projection layer of the rice disease and pest identification model to output the feature space. The feature space is input into the global average pooling layer of the rice disease and pest identification model to output semantic features of diseases and pests. The semantic features of the pests and diseases are input in parallel to the fully connected layer, the severity classification network, and the confidence output branch network adapted to multiple types of rice pests and diseases. The probability distribution of pest and disease types is calculated through the fully connected layer and the pest and disease types are output. The severity level is output through the severity classification network and the confidence level is output through the confidence output branch network. The pest / disease type, severity level, and confidence level are encapsulated into a triple and output.

5. The method according to claim 1, characterized in that, The steps for extracting spectral features from the preprocessed image data include: Based on the image data, spectral images of the sensitive bands for pests and diseases are extracted to obtain sensitive spectral images; The spectral features are obtained by calculating the band ratio characteristics and spectral reflectance based on the sensitive spectral image.

6. The method according to claim 1, characterized in that, The steps for extracting the spatiotemporal evolution features of lesions from the preprocessed image data include: For the visible light images in the image data, the background subtraction method is used to extract the lesion variation regions between consecutive frame images; For the lesion change area, the lesion expansion rate and lesion diffusion direction vector are calculated to obtain the spatiotemporal evolution characteristics of the lesion.

7. The method according to claim 1, characterized in that, The steps for extracting virtual microscopic features and ecological association features from the preprocessed image data include: The multispectral image from the image data is input into a high-resolution conditional generative adversarial network to reconstruct the multispectral image into a high-resolution virtual microscopic image; Identify the lesion boundaries and hyphal network structure in the high-resolution virtual micrograph; Morphological indices are calculated based on the lesion boundaries and the hyphal network structure, and the virtual microscopic features are generated based on the morphological indices. Combining the growth stage information and the real-time meteorological data, ecological association features are extracted from the image data, wherein the ecological association features include at least one of the following: infection features and environmental features.

8. A device for identifying rice diseases and pests, characterized in that, include: The acquisition unit is used to acquire multispectral images, visible light images, real-time meteorological data, and information on the rice growth stages of the target paddy field. The processing unit is used to preprocess the multispectral image and the visible light image to obtain preprocessed image data. The extraction unit is used to combine the real-time meteorological data and rice growth stage information to extract spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features from the preprocessed image data, and to fuse the spectral features, spatiotemporal evolution features of lesions, virtual microscopic features and ecological association features into a fused feature vector through an attention mechanism; The identification unit is used to input the fused feature vector into a rice pest and disease identification model that has been pre-trained by transfer learning, and output the pest and disease type, severity level and confidence level. The rice pest and disease identification model is a pre-constructed deep convolutional neural network model for identifying rice pests and diseases. The rice pest and disease identification model includes at least a fully connected layer adapted to multiple types of rice pests and diseases, a severity grading network and a confidence output branch network. The severity grading network is used to assess the severity of rice pests and diseases. The generation unit is used to generate rice pest and disease identification results based on the pest and disease type, the severity level, and the confidence level.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the rice pest and disease identification method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, The method includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for identifying rice diseases and pests as described in any one of claims 1 to 7.

Images