Method for detecting early plant diseases and insect pests of fruit tree leaves in combination with RGB camera and portable hyperspectral radiometer
By combining a portable hyperspectral radiometer with an RGB camera for multimodal data fusion, the labor intensity and equipment complexity of traditional pest detection methods are solved, and early, accurate and large-scale detection of pests and diseases in fruit tree leaves is achieved, providing efficient pest warning and prevention and control support.
Patent Information
- Application Number
- CN202510500191.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-08-12
Smart Images
Figure CN120472309A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of agricultural technology, and in particular to a method for detecting early-stage diseases and insect pests on fruit tree leaves by combining an RGB camera with a portable hyperspectral radiometer. Background Art
[0002] With the development of precision agriculture and smart farming, agricultural pest and disease detection and monitoring technologies have gradually become a research hotspot. Traditional pest and disease detection methods rely primarily on manual visual inspection and chemical analysis. These methods are not only labor-intensive but also subject to significant human influence, making it difficult to achieve early, accurate, and large-scale pest and disease monitoring. In recent years, spectral imaging technology, especially hyperspectral imaging technology, has been gradually applied to agriculture due to its high-resolution spectral characteristics, achieving significant progress in the early detection of crop pests and diseases.
[0003] Hyperspectral imaging technology can capture the reflected light information of plant leaves in different bands by collecting a wide range of spectral data. Studies have shown that the spectral reflectance of leaves is closely related to the occurrence and development of pests and diseases. In particular, when important components such as chlorophyll and water in plants change, the spectral characteristics of plants will show significant differences. Common hyperspectral image acquisition equipment includes portable hyperspectral radiometers, which can obtain spectral information in multiple bands, facilitating feature extraction and analysis. RGB cameras use visible light to capture crop surface images. Unlike hyperspectral imaging technology, RGB cameras typically have lower spectral resolution. However, they can extract color features through simple image processing techniques to assist in identifying surface symptoms of pests and diseases, especially in their early stages. Although hyperspectral imaging technology offers high resolution and accuracy, its equipment is typically bulky, expensive, and complex to operate. This makes its application in the field challenging, particularly for large-scale agricultural monitoring, where real-time and widespread implementation are difficult. Furthermore, existing technologies mostly rely on a single technique and lack efficient multimodal data fusion solutions. Therefore, a method for early detection of fruit tree pests and diseases using foliage is proposed, combining an RGB camera with a portable hyperspectral radiometer. Summary of the Invention
[0004] The purpose of the present invention is to provide a method for early detection of diseases and insect pests on fruit tree leaves by combining an RGB camera and a portable hyperspectral radiometer, so as to solve the problems raised in the above background technology.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting early-stage pests and diseases on fruit tree leaves by combining an RGB camera with a portable hyperspectral radiometer, comprising the following steps: S1, spectral data preprocessing; S2, RGB image data preprocessing; S3, using an information fusion method to fuse the spectral data and the RGB image data; S4. Early warning of pests and diseases, generating pest and disease warning information to help fruit farmers take corresponding prevention and control measures.
[0006] Preferably, the above step S1 includes the following sub-steps: S101, using a spectroradiometer to collect spectral data of fruit tree leaves, the collected spectral data is ; S102, performing denoising processing on the spectral data; S103, performing spectral standardization processing on the spectral data to eliminate the influence of ambient light changes; S104, performing PCA feature extraction from spectral data; S105. Using the extracted spectral features, classification is performed through a support vector machine (SVM) model to determine the type and occurrence degree of the pests and diseases.
[0007] As a preference, in the above: in S102, in the noise removal process, the The denoising calculation formula is:
[0008] in, Indicates wavelength.
[0009] As an advantage, in the above: in S103, the spectrum standardization process The calculation formula is:
[0010] in, wavelength The mean difference of the sample data is wavelength The standard deviation of the sample data.
[0011] Preferably, in the above step S104, the PCA feature extraction calculation formula is:
[0012] in, is the principal component vector.
[0013] Preferably, in the above step S105, the calculation formula of the classification result of the SVM model is:
[0014] in, is the classification result, is the classifier function, is the spectral feature.
[0015] Preferably, the above step S2 includes the following sub-steps: S201, using an RGB camera to obtain visible light phenotypic images of fruit tree leaves, wherein the phenotypic images mainly reflect the color, morphology and texture characteristics of the leaves. The collected RGB image data is ; S202, preprocessing the RGB image, including denoising, contrast enhancement, and color normalization; The calculation formula for the contrast enhancement is:
[0016] in, The enhanced image is The pixel value of the position, Indicates that the original image is The pixel value of the position, Indicates the new pixel value after enhancing the original image; S203, extracting phenotypic features of color, morphology and texture from RGB images; The image color normalization calculation formula is as follows:
[0017] in, Represents the color characteristics of the image, Extracting color information from enhanced images; The image texture feature extraction calculation formula is as follows:
[0018] in, Represents the texture features of the image, Extracting texture information from enhanced images; S204, using a convolutional neural network (CNN) to analyze the extracted phenotypic features and identify the type and occurrence of pests and diseases; The recognition result calculation formula is:
[0019] in, The enhanced image is The pixel value at the location.
[0020] Preferably, the above step S3 includes the following sub-steps: S301, performing feature-level fusion of spectral features and phenotypic features to form a comprehensive feature vector; S302: The classification results of the spectral data and the classification results of the RGB image are fused at the decision level using a weighted average method.
[0021] Preferably, in the above step S301, the calculation formula of the comprehensive feature vector is as follows:
[0022] in, for dimensional spectral feature vector, is the phenotypic texture feature vector, is the phenotypic color feature vector.
[0023] Preferably, in the above step S302, the calculation formula of the weighted average method is as follows:
[0024] in, and is the weight coefficient, is the classification result of spectral data, is the classification result of RGB image.
[0025] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: The present invention overcomes the limitations of traditional single technologies by combining a portable hyperspectral radiometer with an RGB camera, providing a more efficient and accurate solution for detecting pests and diseases. Although hyperspectral imaging technology alone has a high spectral resolution, it is difficult to be widely used in field monitoring due to the large size, high price and complex operation of the equipment; and although the RGB camera is easy to operate, its spectral resolution is low and it is difficult to fully reveal the potential characteristics of pests and diseases. By fusing these two technologies into multimodal data, the physiological characteristics and phenotypic characteristics of fruit tree leaves can be obtained simultaneously, which not only improves the accuracy of pest and disease detection, but also enhances the universality and real-time performance of the system. Therefore, by combining these two methods, we can fully utilize the advantages of each method and overcome the shortcomings of a single method, thereby providing fruit farmers with more accurate and efficient pest and disease warning and prevention support. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0027] Figure 1 Flow chart of the detection method of the present invention. DETAILED DESCRIPTION
[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0029] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions under which this application can be implemented. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in this application without affecting the efficacy and purpose that can be achieved by this application. Example
[0030] See also Figure 1 The present invention provides a technical solution: a method for detecting early-stage pests and diseases on fruit tree leaves by combining an RGB camera with a portable hyperspectral radiometer, comprising the following steps: S1, spectral data preprocessing; S101, using RP410NIR spectroradiometer to collect spectral data of fruit tree leaves, the collected spectral data is ; As a hyperspectral acquisition device, the RP410 NIR spectroradiometer is equipped with a built-in GPS module, which can obtain geographic location information in real time.
[0031] The RP410NIR spectroradiometer is powered by a rechargeable lithium-ion battery and can operate independently without a PC. Its wavelength range covers the near-infrared (NIR) region from 640nm to 1050nm. It uses a clamp to capture spectral data from fruit tree leaves. By accurately measuring leaf reflectance spectra at different wavelengths, it can obtain NIR spectral information, enabling analysis of leaf growth and potential pests and diseases.
[0032] S102, performing denoising on the spectral data by using a Savitzky-Golay filter to eliminate noise; In the noise removal process, The denoising calculation formula is:
[0033] in, Indicates wavelength.
[0034] After eliminating the noise, is the reflectivity of the blade in different bands.
[0035] S103, performing spectral standardization processing on the spectral data to eliminate the influence of ambient light changes; Spectral normalization The calculation formula is:
[0036] in, is the wavelength The mean difference of the sample data is wavelength The standard deviation of the sample data.
[0037] S104, performing PCA feature extraction from spectral data; The PCA feature extraction calculation formula is:
[0038] in, is the principal component vector.
[0039] S105, using the extracted spectral features to perform classification through a support vector machine (SVM) model to determine the type and occurrence degree of the pests and diseases; The calculation formula for the classification result of the SVM model is:
[0040] in, is the classification result, is the classifier function, is the spectral feature.
[0041] S2, RGB image data preprocessing; S201, using an RGB camera to obtain visible light phenotypic images of fruit tree leaves. Phenotypic images mainly reflect the color, morphology and texture characteristics of the leaves. The collected RGB image data is ; RGB cameras are used to capture visible light phenotypic images of fruit tree leaves. They can capture leaf characteristics such as color, shape, and texture in real time. Aim the camera at the leaf, keeping the lens parallel to the leaf surface as much as possible, to ensure clear and accurate images.
[0042] S202, preprocessing the RGB image, including denoising, contrast enhancement, and color normalization; The calculation formula for contrast enhancement is:
[0043] in, The enhanced image is The pixel value of the position, Indicates that the original image is The pixel value of the position, Indicates the new pixel value after enhancing the original image; S203, extracting phenotypic features of color, morphology and texture from RGB images; The image color normalization calculation formula is as follows:
[0044] in, Represents the color characteristics of the image, Extracting color information from enhanced images; The calculation formula for image texture feature extraction is as follows:
[0045] in, Represents the texture features of the image, Extracting texture information from enhanced images.
[0046] S204, using a convolutional neural network (CNN) to analyze the extracted phenotypic features and identify the type and occurrence of pests and diseases; The formula for calculating the recognition result is:
[0047] in, The enhanced image is The pixel value at the location.
[0048] During data collection, the RP410 NIR spectroradiometer and RGB camera work together to obtain relevant information about fruit tree leaves from both spectral and phenotypic levels. This method combines spectral data with RGB images, merging spectral and phenotypic features to fully utilize their complementarity, and optimizes classification results through weighted averaging, thereby improving detection accuracy and reliability.
[0049] S3. Spectral data and RGB image data are fused using an information fusion method. Hyperspectral data captures the physiological characteristics of leaves, while RGB images intuitively display the phenotypic characteristics of leaves. The combination of the two helps to more accurately identify pests and diseases, improve diagnostic accuracy, and ensure more effective early warning.
[0050] S301, performing feature-level fusion of spectral features and phenotypic features to form a comprehensive feature vector; The calculation formula of the comprehensive eigenvector is as follows:
[0051] in, for dimensional spectral feature vector, is the phenotypic texture feature vector, is the phenotypic color feature vector.
[0052] Comprehensive feature vector It also contains the physiological information (spectrum) and phenotypic information (RGB) of the leaves, thereby improving the accuracy of pest and disease identification.
[0053] S302, performing decision-level fusion on the classification results of the spectral data and the classification results of the RGB image using a weighted average method; The calculation formula of the weighted average method is as follows:
[0054] in, and is the weight coefficient, is the classification result of spectral data, The hyperspectral and RGB data obtained by the two devices are the classification results of the RGB image. The present invention can accurately evaluate the health status of the leaves of the fruit trees in the facility and identify the early symptoms of diseases and pests.
[0055] Combining the portable RP410 NIR hyperspectral radiometer with an RGB camera overcomes the limitations of traditional equipment, which are bulky and complex to operate. Together, the system can simultaneously capture both physiological and phenotypic information on leaves, improving the accuracy and universality of leaf pest and disease early warning systems for fruit trees in greenhouses.
[0056] S4. Early warning of pests and diseases, generating pest and disease warning information to help fruit farmers take corresponding prevention and control measures.
[0057] Using hyperspectral and RGB data acquired by these two devices, this technology can accurately assess the health of fruit tree leaves and identify early signs of pests and diseases. By comparing data changes at different time points, potential pest and disease problems can be detected in advance, providing timely early warning information to fruit farmers and helping them take appropriate preventive measures.
[0058] In summary, the present invention overcomes the limitations of traditional single technologies by combining a portable hyperspectral radiometer with an RGB camera, and provides a more efficient and accurate solution for detecting pests and diseases. Although the hyperspectral imaging technology alone has a high spectral resolution, it is difficult to be widely used in field monitoring due to the large size of the equipment, high price and complex operation; and although the RGB camera is easy to operate, its spectral resolution is low and it is difficult to fully reveal the potential characteristics of pests and diseases. By fusing these two technologies into multimodal data, the physiological characteristics and phenotypic characteristics of fruit tree leaves can be obtained simultaneously, which not only improves the accuracy of pest and disease detection, but also enhances the universality and real-time performance of the system. Therefore, by combining these two methods, we can fully utilize the advantages of each method and overcome the shortcomings of a single method, thereby providing fruit farmers with more accurate and efficient pest and disease warning and prevention support.
[0059] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of the present invention may be made, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of the present invention may be made, without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.
Claims
1. A method for detecting early-stage pests and diseases on fruit tree leaves by combining an RGB camera with a portable hyperspectral radiometer, characterized in that: The following steps are involved: S1, spectral data preprocessing; S2, RGB image data preprocessing; S3, using an information fusion method to fuse the spectral data and the RGB image data; S4. Early warning of pests and diseases, generating pest and disease warning information to help fruit farmers take corresponding prevention and control measures.
2. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 1, characterized in that: In S1, the following sub-steps are included: S101, using a spectroradiometer to collect spectral data of fruit tree leaves, the collected spectral data is ; S102, performing denoising processing on the spectral data; S103, performing spectral standardization processing on the spectral data to eliminate the influence of ambient light changes; S104, performing PCA feature extraction from spectral data; S105. Using the extracted spectral features, classification is performed through a support vector machine (SVM) model to determine the type and occurrence degree of the pests and diseases.
3. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 2, characterized in that: In S102, during the noise removal process, the The denoising calculation formula is: , in, Indicates wavelength.
4. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 2, wherein: In S103, the spectrum normalization process The calculation formula is: , in, wavelength The mean difference of the sample data is wavelength The standard deviation of the sample data.
5. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 2, wherein: In S104, the PCA feature extraction calculation formula is: , in, is the principal component vector.
6. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 2, wherein: In S105, the calculation formula of the classification result of the SVM model is: , in, is the classification result, is the classifier function, is the spectral feature.
7. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 1, wherein: In S2, the following sub-steps are included: S201, using an RGB camera to obtain visible light phenotypic images of fruit tree leaves, wherein the phenotypic images mainly reflect the color, morphology and texture characteristics of the leaves. The collected RGB image data is ; S202, preprocessing the RGB image, including denoising, contrast enhancement, and color normalization; The calculation formula for the contrast enhancement is: , in, The enhanced image is The pixel value of the position, Indicates that the original image is The pixel value of the position, Indicates the new pixel value after enhancing the original image; S203, extracting phenotypic features of color, morphology and texture from RGB images; The image color normalization calculation formula is as follows: , in, Represents the color characteristics of the image, Extracting color information from enhanced images; The image texture feature extraction calculation formula is as follows: , in, Represents the texture features of the image, Extracting texture information from enhanced images; S204, using a convolutional neural network (CNN) to analyze the extracted phenotypic features and identify the type and occurrence of pests and diseases; The recognition result calculation formula is: , in, The enhanced image is The pixel value at the location.
8. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 1, wherein: In S3, the following sub-steps are included: S301, performing feature-level fusion of spectral features and phenotypic features to form a comprehensive feature vector; S302: The classification results of the spectral data and the classification results of the RGB image are fused at the decision level using a weighted average method.
9. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 8, wherein: In S301, the calculation formula of the comprehensive feature vector is as follows: , in, for dimensional spectral feature vector, is the phenotypic texture feature vector, is the phenotypic color feature vector.
10. The method for detecting early-stage pests and diseases on fruit tree leaves using a combination of an RGB camera and a portable hyperspectral radiometer according to claim 8, wherein: In S302, the calculation formula of the weighted average method is as follows: , in, and is the weight coefficient, is the classification result of spectral data, is the classification result of RGB image.