Rapid visual detection method and system for chlorophyll of crop leaves based on visible fluorescence image

Abstract

The invention discloses a crop leaf chlorophyll rapid visual detection method and system based on a visible fluorescence image, and the method comprises the steps: collecting visible and multi-mode fluorescence images, building a mask for the visible image, and applying the mask to the multi-mode fluorescence image to determine a region of interest; pixel intensity statistical characteristics and multi-level depth characteristics of each channel in a region of interest of the multi-mode fluorescence image are extracted, an optimal characteristic subset reflecting chlorophyll information is preferably determined and input into a chlorophyll visual detection model for detection, and detection results comprise a chlorophyll content average value and a spatial distribution visual graph of the leaves. According to the invention, the problem of low precision of chlorophyll content spatial distribution information obtained in the prior art is solved, the detection system is simple in structure and low in manufacturing cost, and visual, high-precision and rapid detection of the chlorophyll content of the crop leaves is realized.

Description

Technical Field

[0001] This invention belongs to the field of rapid visual detection technology of chlorophyll in crop leaves, and particularly relates to a rapid visual detection method and system for chlorophyll in crop leaves based on visible fluorescence images. This detection method and system can be used to rapidly and visually detect the chlorophyll content in crop leaves. Background Technology

[0002] Chlorophyll content in crop leaves is a key indicator for assessing plant growth. Accurate and rapid detection of chlorophyll in crop leaves has significant economic value for optimizing cultivation management. While traditional chemical methods for chlorophyll determination are standardized, their destructive and time-consuming nature makes them unsuitable for rapid, non-destructive testing. Existing chlorophyll detection technologies involve complex equipment, high manufacturing costs, and demanding operator skills, making them difficult to adapt to practical production processes. Furthermore, they offer limited accuracy in obtaining spatial distribution information of chlorophyll content. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, this invention provides a rapid visualization detection method and system for chlorophyll in crop leaves based on visible fluorescence images. The method uses a rapid chlorophyll visualization detection system to acquire multimodal fluorescence images and visible images of crop leaves. A visible image mask is used to determine the region of interest, and the optimal feature subset is extracted and selected before being input into a chlorophyll visualization detection model. The output is a visualization map of the average chlorophyll content and its spatial distribution. This invention aims to solve the problems of existing crop leaf chlorophyll detection technologies, such as destructiveness, low efficiency, high cost, and low accuracy in obtaining spatial distribution information of chlorophyll content.

[0004] The technical solution adopted in this invention is as follows:

[0005] For the crop leaf samples to be tested, a rapid visual detection system for crop leaf chlorophyll was used to excite the samples sequentially with excitation light of a first wavelength (preferably blue light with a center wavelength in the range of 450±10 nm) and a second wavelength (preferably red light with a center wavelength in the range of 650±10 nm). Under each excitation light irradiation, multimodal fluorescence images were acquired by automatically switching different filters. Under blue light excitation, fluorescence images with center wavelengths of 650 nm, 680 nm, and 740 nm were acquired; under red light excitation, fluorescence images with center wavelengths of 680 nm and 740 nm were acquired; and finally, visible images under pure white light were acquired.

[0006] A mask is obtained from the visible image, and the mask is applied to the multimodal fluorescence image. Then, statistical features and multi-level depth features are extracted based on the channel pixel intensity of the multimodal fluorescence image, including but not limited to the mean, standard deviation, maximum value, minimum value and other statistical quantities of each channel. Shallow, middle and deep features are extracted using convolution pooling.

[0007] Subsequently, the optimal subset of features reflecting chlorophyll information is determined by means of the following steps:

[0008] Step 1: Based on the feature importance calculation method, the importance of each feature in the channel statistical features and multi-level deep features of the multimodal fluorescence image to the model prediction output is quantified by calculating the importance value of each feature on all training samples.

[0009] Step 2: Based on the calculated importance values, sort all features in descending order.

[0010] Step 3: Using the sequential forward selection method, starting with the most important feature, add features one by one to a candidate feature subset according to the arrangement order. After each feature is added, retrain the model using the current candidate feature subset and record the model's performance evaluation metrics on the independent validation set.

[0011] Step 4: After traversing all features, based on the change curve of the performance evaluation index, determine a candidate feature subset that contains the fewest features while ensuring optimal or near-optimal model performance, and use this subset as the best feature subset.

[0012] By inputting the optimal subset of features extracted from the leaves of the crop to be tested into the optimal chlorophyll visualization detection model, the average chlorophyll content of the sample can be obtained quickly and accurately.

[0013] To achieve visualization, the ROI of the sample can be divided into multiple pixel blocks, preferably 10 pixels per block. For each unit, the feature extraction process is repeated, and the chlorophyll content is predicted using the final model. Then, the predicted values ​​of all pixel blocks are reconstructed into a two-dimensional data matrix according to their original spatial coordinates within the ROI. Using a preset color mapping table, this matrix is ​​rendered into a pseudo-color image or heatmap, where the color or brightness of the image directly corresponds to the chlorophyll content value, thus intuitively displaying the spatial distribution of chlorophyll within the leaf.

[0014] The aforementioned method and system for rapid visualization detection of chlorophyll in crop leaves based on visible fluorescence images, wherein the optimal chlorophyll visualization detection model is a typical machine learning regression model and a deep learning regression model, including any one or more of GBDT, XGBoost, LightGBM, RF, SVR, PLSR, CNN, and LSTM. To further improve model performance, the Grey Wolf Optimization (GWO) algorithm is preferably used to automatically optimize the model's hyperparameters. The optimization objective is set to maximize the model's coefficient of determination (R²) on the cross-validation set. p () or minimize the root mean square error (RMSE).

[0015] The present invention also provides a system for implementing the above method, the system comprising: a core control module for synchronously controlling the timing and operation of various components of the system, comprising a Raspberry Pi control board, a multi-channel relay, and a constant current driver; an active light source module connected to the core control module, comprising at least one blue light source, a red light source, and a neutral white light source; an image acquisition module comprising a CMOS or CCD image sensor with an infrared cutoff filter removed and a fixed-focus lens; a filter switching module comprising a filter wheel driven by a servo motor and loaded with multiple narrowband filters and infrared cutoff filters of different center wavelengths, the module being disposed in the optical path of the image acquisition module and controlled by the core control module; and a data processing unit storing program instructions to execute the above method.

[0016] The advantages of this invention compared to existing technologies are:

[0017] First, the detection method and system described in this invention can not only predict the average chlorophyll content of leaves, but also generate a visualization of its spatial distribution, intuitively presenting the uneven distribution of chlorophyll in leaves and providing richer information.

[0018] Secondly, the detection system described in this invention has a simple structure, low manufacturing cost, high degree of automation in the detection process, and is integrated into the detection equipment, making it convenient to operate.

[0019] Third, the detection method and system described in this invention achieve high-precision and rapid detection of leaf chlorophyll content, significantly overcoming the problems of traditional chemical methods being highly destructive, having low visualization accuracy, and being inefficient. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0021] Figure 1 This is a schematic diagram of the detection process of the method of the present invention.

[0022] Figure 2 This is a schematic diagram of the training process of the optimal chlorophyll visualization detection model saved in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the structure of the crop leaf chlorophyll rapid visualization detection system for implementing the method of the present invention.

[0024] Figure 4 This is an example of a multimodal fluorescence image acquired in this invention.

[0025] Figure 5 This is an example of a visible image mask acquired by the present invention.

[0026] Figure 6 This is a bar chart showing the global importance ranking of four preferred models in the embodiments of the present invention.

[0027] Figure 7 This is a performance evaluation curve of the sequence forward selection process in an embodiment of the present invention.

[0028] Figure 8 This is a spatial distribution visualization of chlorophyll content generated by the present invention.

[0029] To better understand the accompanying drawings, the following is an explanation. Figure 3 Explanation of the markings appearing in the code: Core control module: 1. Multiplex relay module, 2. Raspberry Pi 5B, 3. Constant current driver; Active light source module: 4. Red light source, 5. Blue light source, 6. Neutral white light source; Image acquisition module: 7. Sony IMX477 camera; Filter switching module: 8. Filter wheel, 9. Filter; Data processing unit: 2. Raspberry Pi 5B Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be fully described below in conjunction with the accompanying drawings and through a detailed, non-limiting preferred embodiment. Those skilled in the art should understand that all equivalent modifications or substitutions made based on this embodiment fall within the protection scope of this invention.

[0031] Example 1: Construction of a rapid visual detection system for chlorophyll in crop leaves

[0032] Image acquisition module: It adopts the official Raspberry Pi HQ camera module, the core of which is a 12.3-megapixel Sony IMX477 CMOS sensor. The camera is connected to a 6mm focal length, large aperture (f / 1.2) CS mount fixed focus lens via a C-CS adapter ring to maximize light throughput and improve the ability to capture weak fluorescence signals.

[0033] The active light source module parameters are as follows: The white light source consists of two 6W flat-panel LED light panels, with a total power of 12W and a color temperature of 6500K. It is used to provide uniform, shadow-free top illumination, mainly for capturing standard color reference images of samples, generating masks, and accurately segmenting regions of interest in multimodal fluorescence images. The blue light source consists of two 12W array-type LED light source panels, with a total power of 24W. The center wavelength range of its emission spectrum is strictly controlled within 450±10 nm. The red light source also consists of two 12W array-type LED light source panels, with a total power of 24W. The center wavelength range of its emission spectrum is strictly controlled within 650±10 nm.

[0034] To improve the stability and lifespan of the light sources, the power supplies for all three light sources are connected to constant current drivers and are connected to the GPIO pins of the Raspberry Pi through a multi-mode relay module. The Raspberry Pi controls the on / off state of the relay by outputting high and low level signals, thereby achieving safe, reliable, and independent programmable switching control of each light source.

[0035] Filter switching module: To separate the target narrowband multimodal fluorescence image from a broadband CMOS sensor, a four-hole filter wheel driven by an SG90 micro servo was designed and 3D printed. The servo receives the PWM signal from the Raspberry Pi and can be precisely positioned to any hole.

[0036] Both the infrared cut-off filter and the narrowband interference filter have a diameter of 30 mm. The infrared cut-off filter receives wavelengths in the range of 350 nm to 650 nm. The center wavelength and full width at half maximum (FWHM) of the narrowband interference filter are 650 nm (FWHM=10 nm), 680 nm (FWHM=20 nm), and 740 nm (FWHM=20 nm), respectively.

[0037] The core control module uses a Raspberry Pi 5B, which uses its general purpose input / output (GPIO) pins to achieve precise timing synchronization control of the active light source module, filter switching module and image acquisition module. The specific functional interface connections are shown in Table 1.

[0038] Table 1 Functional Interface Connection Table

[0039] Example 2: Acquisition and Tagging of Multimodal Fluorescence and Visible Images

[0040] To ensure standardization of the acquisition process, this study employed an automatically controlled image acquisition sequence. The crop leaf chlorophyll rapid visualization detection system was initialized and the filter acquisition module was activated. Subsequently, the program executed the following acquisition tasks in sequence: First, under blue excitation light, the system automatically switched filters to continuously capture multimodal fluorescence images in three bands: 650 nm, 680 nm, and 740 nm. Then, the light source was switched to red excitation light, and multimodal fluorescence images in two bands: 680 nm and 740 nm were acquired. Finally, visible light images were acquired under the action of white light and infrared cutoff filters. After the entire sequence was completed, the system automatically turned off the light source and reset.

[0041] Chlorophyll content was determined using the ethanol (95%) extraction method. After image acquisition, approximately 0.2 g of leaves were cut, avoiding the main vein, and then chopped, mixed, and placed in a reagent tube. Ethanol (95%) was added and the volume was adjusted to 25 mL. The tube was sealed, shaken well, and placed in a light-proof, sealed foam box at room temperature for 48 h. After the extracted leaves turned completely white, they were shaken well again, and the absorbance of the extract at 665 nm and 649 nm was measured using a UV-Vis spectrophotometer (Meipuda UV-1800), and the data were recorded. The formula for calculating chlorophyll content is as follows:

[0042]

[0043]

[0044]

[0045] In the formula: A 665 , A 649 The values ​​are the absorbance (A) of the spectrophotometer at 665 nm and 649 nm, respectively; V is the volume of the extract (mL); and W is the fresh weight of the leaf sample (g).

[0046] Example 3: Optimal Feature Subset Selection and Determination of the Optimal Chlorophyll Visualization Detection Model

[0047] Part A: First, the visible image of the leaf is converted from the preferred color space to the HSV color space; when the pixels satisfy a certain condition, the threshold range can effectively separate the leaf body from the background area and the label, thereby constructing a binary mask. M The data is applied to the acquired multimodal fluorescence images.

[0048]

[0049] For each preprocessed multimodal fluorescence image, the average intensity of all pixels within its ROI is calculated, and this average value is used as the feature value of this sample. Ultimately, each leaf sample corresponds to a set of feature variables, as shown in Table 2 below.

[0050] Table 2 Features extracted from multimodal fluorescence images

[0051] Part b: To obtain a stable and unbiased estimate of the model's generalization ability, a K-fold cross-validation strategy is adopted, where K is set to 5. The entire dataset is divided into five subsets, and through five independent training and validation processes, the comprehensiveness and robustness of the model evaluation are ensured. Within this cross-validation framework, the coefficient of determination (COP) is used. R 2 The model performance is comprehensively evaluated using three metrics: root mean square error (RMSE), and relative prediction error (RPD).

[0052] To build an accurate chlorophyll visualization detection model from the extracted features, four ensemble tree learning models were selected for comparative analysis: XGBoost: an efficient and scalable implementation based on gradient boosting decision trees (GBDT), which optimizes the second-order Taylor expansion of the loss function and introduces L1 and L2 regularization terms to control model complexity and prevent overfitting, demonstrating excellent performance in various data mining competitions and practical applications; Random Forest (RF): an ensemble method that constructs multiple decision trees and averages or votes on their prediction results, exhibiting good anti-overfitting ability; Gradient Boosting Decision Tree (GBDT): adopts a serial approach, where each new tree aims to fit the residual of the previous tree, thereby gradually improving model performance; LightGBM: a GBDT framework derived from XGBoost, employing gradient-based one-sided sampling (GOSS) and mutually exclusive feature binding (EFB) techniques, significantly improving training speed while maintaining accuracy.

[0053] Hyperparameter Intelligent Optimization: To objectively and efficiently determine the optimal hyperparameter combination for the tree model, the Grey Wolf Optimization Algorithm (GWO) is selected. GWO performs a global search in the multidimensional parameter space by simulating the social hierarchy (α, β, δ wolves) and cooperative hunting behaviors (encirclement, pursuit, attack) of a grey wolf population. The parameters of GWO itself are set as follows: wolf pack size is 20, maximum number of iterations is 20, and the optimization objective is to maximize... R 2 p The model results with full feature input are shown in Table 3.

[0054] Table 3 Model results with full feature input

[0055] Part c: Optimizing and determining the best feature subset reflecting chlorophyll information

[0056] The main steps for determining the optimal feature subset by combining the preferred SHAP feature selection method with Sequence Forward Selection (SFS) are as follows.

[0057] Step 1: In one loop, add features sequentially to the candidate subset S. In each step, using the current S, perform 5-fold cross-validation on the training set to train and evaluate the model. Record the average coefficient of determination on the cross-validation set. R 2 p .

[0058] Step 2: Performance curve analysis and plotting. R 2 p The curve showing the change as the number of features increases. Observe the change as the number of features increases from 1 to 15. R 2 p The numerical change.

[0059] Step 3: Select the best-performing and simplest feature combination as the final optimal feature subset.

[0060] Step 4: To further demonstrate the superiority of the SHAP-SFS feature selection method, the feature importance index built into the machine learning benchmark model is used to rank all features by importance. The same sequential forward selection method as SHAP-SFS is used to determine an optimal feature subset based on the FI ranking. On the same test set, the performance of the model trained based on the optimal feature subset of SHAP-SFS and the model trained based on the optimal feature subset of FI-SFS are compared to verify the superiority of the SHAP-SFS method. The specific model results of the two feature selection methods are shown in Tables 4 and 5 below.

[0061] Table 4 Results of the chlorophyll visualization detection model based on FI-SFS

[0062] Table 5 Results of the chlorophyll visualization detection model based on SHAP-SFS

[0063] Comparative analysis reveals the optimal model that utilizes all features. R 2 p =0.8526, RPD=2.6251) and the model using the FI-SFS method ( R 2 pCompared to (=8489, RPD=2.5587), the preferred SHAP-SFS feature selection method of this invention shows certain advantages, significantly reducing feature dimensionality while improving model performance, demonstrating its effectiveness in selecting key information, suppressing noise, and improving model generalization ability. Among them, the preferred GBDT model performs best. R 2 p =0.8552, RPD=2.6483), and saved it as the optimal chlorophyll visualization detection model.

[0064] Example 4: Rapid Visual Detection of Chlorophyll in Crop Leaves

[0065] Step 1: Acquire visible images and multimodal fluorescence images, and input the optimal feature subset selected through Part c of Example 3 into the optimal chlorophyll visualization detection model to output the predicted result of average chlorophyll content.

[0066] Step 2: Pixel-level feature matrix construction: Construct a feature vector for each pixel block (10 pixels per block) within the ROI. Predict pixel-by-pixel. Input the feature vector v(i,j) of each pixel into the final model to obtain the predicted chlorophyll content value C(i,j) of that pixel block. Reconstruct all predicted values ​​C(i,j) into a two-dimensional matrix. Finally, use the Matplotlib library and 'viridis' (color map) to render it into a pseudo-color image and attach color bars indicating the content value. The generated image can clearly and quantitatively reveal the differences in the microscopic distribution of chlorophyll in the leaf, such as the difference in content between the vein and mesophyll regions.

[0067] This invention, through four embodiments, elucidates the operational flow of a rapid visual detection method and system for chlorophyll in crop leaves based on visible fluorescence images, focusing on aspects such as the construction of a rapid visual detection system for chlorophyll in crop leaves, acquisition and labeling of multimodal fluorescence and visible images, optimization of the optimal feature subset and determination of the optimal chlorophyll visual detection model, and rapid visual detection of chlorophyll in crop leaves. This invention utilizes a specific combination of excitation light source and filters to acquire statistical features and multi-level depth features extracted from multimodal fluorescence images of crop leaves. The detection system has a simple structure, low manufacturing cost, and a high degree of automation, integrated into the detection equipment for convenient operation. The detection method and system can not only predict the average chlorophyll content of leaves but also generate a visual map of its spatial distribution, intuitively presenting the uneven distribution of chlorophyll in leaves and providing richer information. It achieves high-precision and rapid detection of chlorophyll content in crop leaves, significantly overcoming the problems of strong destructiveness and low efficiency of traditional chemical methods.

[0068] For the detection of chlorophyll in leaves of other crops, the detection method and system based on visible fluorescence images can be operated with reference to the detection process of the detection method and system proposed in this invention.

[0069] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention and based on the technical essence of the present invention should be included within the scope of the present invention. The patent protection scope of the present invention is defined by the claims.

Claims

1. A rapid visual detection method and system for chlorophyll in crop leaves based on visible fluorescence images, characterized in that... Includes the following steps: (a) Multimodal fluorescence images of the leaves under blue light and red light excitation and visible images under pure white light illumination were acquired using a crop leaf chlorophyll rapid visualization detection system; (b) Create a mask based on the visible image and apply it to the multimodal fluorescence image to determine the region of interest in the multimodal fluorescence image; (c) Extract statistical features and multi-level depth features of the red, green and blue channels of the multimodal fluorescence image within the region of interest, and preferably determine the best feature subset that reflects chlorophyll information; (d) The optimal feature subset that reflects chlorophyll information is selected and input into the chlorophyll visualization detection model for chlorophyll visualization detection. The detection results include the average chlorophyll content of the leaves of the crop to be detected and a spatial distribution visualization map.

2. The method according to claim 1, wherein the steps for acquiring multimodal fluorescence images and visible images are as follows: First, under blue light excitation of 450±10 nm, partial multimodal fluorescence images of the crop leaves are acquired through narrowband filters with center wavelengths of 650 nm, 680 nm, and 740 nm; second, the light source is switched to red light excitation of 650±10 nm, and another part of the multimodal fluorescence images are acquired through narrowband filters with center wavelengths of 680 nm and 740 nm; then, the light source is switched to a pure white light source, and visible images of the crop leaves are acquired through an infrared cutoff filter.

3. The method according to claim 1, wherein the determined channel statistical features and multi-level depth features of the multimodal fluorescence image include any one or more of the mean, standard deviation, maximum value, and minimum value of each channel signal, and the multi-level depth features include any one or more of the features extracted by shallow, medium and deep convolution pooling.

4. The method according to claim 1, wherein the method for optimally determining the best feature subset reflecting chlorophyll information comprises the following steps: Step 1: Based on the feature importance calculation method, the importance of each feature in the channel statistical features and multi-level deep features of the multimodal fluorescence image to the model prediction output is quantified by calculating the importance value of each feature on all training samples. Step 2: Based on the calculated importance values, sort all features in descending order; Step 3: Using the sequential forward selection method, starting from the most important feature, add features one by one to a candidate feature subset according to the arrangement order. After each feature is added, retrain the model using the current candidate feature subset and record the model's performance evaluation metrics on the independent validation set. Step 4: After traversing all features, based on the change curve of the performance evaluation index, determine a candidate feature subset that contains the fewest features while ensuring optimal or near-optimal model performance, and use this subset as the best feature subset.

5. The method according to claim 1, wherein each pixel block of the chlorophyll spatial distribution visualization map has no less than 10 pixels, and the specific steps for generating the spatial distribution visualization map mainly include: First, the predicted chlorophyll content values ​​of all pixel blocks are reconstructed into a two-dimensional data matrix according to their original spatial coordinates within the region of interest. Second, the two-dimensional data matrix is ​​rendered into a pseudo-color image or heatmap using a preset color mapping table, wherein the color or brightness of the image corresponds to the chlorophyll content value.

6. The method according to claim 1, wherein the color space for converting the visible image includes the HSV color space and the LAB color space, a binary mask M is obtained by threshold segmentation, and this mask is applied to the multimodal fluorescence image to determine the region of interest of the multimodal fluorescence image.

7. The method according to claim 1, characterized in that: The chlorophyll visualization detection model is a preferred regression model, including any one or more of GBDT, XGBoost, LightGBM, RF, SVR, PLSR, CNN, and LSTM. The structure and hyperparameters of the chlorophyll visualization detection model are optimized and determined using an intelligent optimization algorithm, with the optimization objective being to maximize the coefficient of determination (R²) on the cross-validation set. p Minimize one or more of the following: , minimize root mean square error (RMSE).

8. A system for implementing the method of claim 1, characterized in that... include: The core control module, used to synchronously control the timing and operation of various components of the system, consists of a Raspberry Pi control board, multiplexers, and a constant current driver; the active light source module, connected to the core control module, includes at least one blue light source, a red light source, and a neutral white light source; the image acquisition module includes a CMOS or CCD image sensor with an infrared cutoff filter removed and a fixed-focus lens; the filter switching module includes a filter wheel driven by a servo motor, loaded with multiple narrowband filters and infrared cutoff filters with different center wavelengths, this module is located in the optical path of the image acquisition module and is controlled by the core control module; and a data processing unit, which stores program instructions, which, when executed, enable the system to perform the steps of the method described in claim 1.

Images