Crop irrigation decision system with multi-channel virtual multi-spectral imaging

Abstract

This invention provides a crop irrigation decision-making system based on multi-channel virtual multispectral imaging, comprising: an image capturing device configured to capture an image of a crop; an image database module configured to store the image of the crop; an image processing module configured to generate a background-free image of the crop using a vegetation index ExGRadj or ExRGB; a multi-channel virtual multispectral module configured to obtain the background-free image of the crop at multiple different wavelengths and convert the background-free image of the crop into multiple multispectral images using a fitting function; a feature analysis module configured to analyze the multiple multispectral images to obtain multiple image feature data; and a water demand module configured to determine the water demand status of the crop using the multiple image feature data.

Description

Technical Field

[0001] This invention relates to a crop irrigation decision-making system that utilizes multi-channel virtual multispectral technology. Prior Technology

[0002] With the development of technology and its application in agriculture, methods for monitoring crop water demand have become more diversified. In areas prone to water scarcity due to climate changes, such as the rice-growing areas of southern Taiwan, ensuring that crops receive timely and adequate water supply is extremely important.

[0003] However, the cost of currently developed multispectral technologies for field application is relatively high, and existing multispectral technologies cannot accurately distinguish between target crops and soil and residues. This results in calculated irrigation water volumes that do not reflect the actual water needs of the crops, leading to water waste or poor crop productivity, and casting doubt on the accuracy and reliability of these technologies. Therefore, it is necessary to provide a more accurate crop irrigation decision-making system to address the problems of conventional technologies. Summary of the Invention

[0004] To address the aforementioned problems, this invention discloses a crop irrigation decision-making system that utilizes multi-channel virtual multispectral technology. By capturing crop data through images and converting the images into multi-channel virtual multispectral images through computation, the system reflects the physiological characteristics of the crops. This allows the multispectral images to reflect changes in the crops' physiological indicators and accurately predict the crops' water requirements before significant variations are observed in their appearance. This establishes stable crop cultivation conditions and quality, while simultaneously achieving energy conservation, carbon reduction, and precision agricultural management benefits.

[0005] To achieve the above objectives, the present invention provides a crop irrigation decision-making system based on multi-channel virtual multispectral imaging, comprising: An image capturing device, configured to capture images of a crop; An image database module is connected to the image capturing device and configured to store images of the crop; An image processing module, connected to the image capturing device or the image database module, is configured to distinguish the crop and background in each crop image using a vegetation index ExGRadj or ExRGB, in order to generate an image with the background crop removed; wherein The ExGRadj = max (0, ExGR) × (ExGR+1) × max (0, y - 180), ExGR = ExG - ExR, ExGB = ExG - ExB, y represents brightness. ExR = 1.4 × r - g, ExG = 2 × g – r - b, ExB = 1.4 × b – g, r represents the red sub-pixel value, g represents the green sub-pixel value, and b represents the blue sub-pixel value. The ExRGB = ExR – ExG – ExB ExR = 1.4 × r - g, ExG = 2 × g – r - b, ExB = 1.4 × b – g, r represents the red sub-pixel value, g represents the green sub-pixel value, and b represents the blue sub-pixel value; A multi-channel virtual multispectral module is connected to the image processing module and configured to obtain images of the crop removed from the background at multiple different wavelengths, and to convert the images of the crop removed from the background at multiple different wavelengths into multiple multispectral images through a fitting function; A feature analysis module, connected to the multi-channel virtual multispectral module, is configured to analyze the multiple multispectral images to obtain multiple image feature data; and A water demand module is connected to the feature analysis module and has a preset crop water demand model. The crop water demand model uses the multiple image feature data to determine the water demand of the crop or whether the crop is in a water shortage state. If the crop is in the water shortage state, the water demand module issues an alert.

[0006] In one embodiment of the present invention, in the image database module, the area of ​​the crop in the image of the crop accounts for more than 80% of the total area of ​​the image.

[0007] In one embodiment of the invention, the plurality of different wavelengths includes at least two of 390 nm, 410 nm, 450 nm, 475 nm, 520 nm, 560 nm, 625 nm, 650 nm, 730 nm, 770 nm, 840 nm, 860 nm, 880 nm, 900 nm, 930 nm, and 960 nm.

[0008] In one embodiment of the present invention, the wavelength is in the range of 390-650 nm. The image of the crop removed at this wavelength is converted into a multispectral image by the following fitting function: a * x 7 + b * x 6 + c * x 5 + d * x 4 + e * x 3 + f * x 2 + g * x + h * 10 5 = y, where a to h are constants, a is -3 to -4, b is 1 to 2, c is -1 to -2, d is 1 to 2, e is -8 to -9, f is 25 to 26, g is -4 to -5, h is 2 to 3, x is the pixel data of the image of the crop removed, and y is the pixel data of the plurality of multispectral images generated corresponding to x.

[0009] In one embodiment of the present invention, the wavelength is in the range of 730-960 nm. The image of the crop removed at this wavelength is converted into a multispectral image by the following fitting function: a' * x 7 + b' * x 6 + c' * x 5 + d' * x 4 + e' * x 3 + f' * x 2 + g' * x + h' * 10 7 = y, where a' to h' are constants, a' is -1 to -2, b' is 1 to 2, c' is -2 to -3, d' is 3 to 4, e' is -30 to -31, f' is 1 to 2, g' is -4 to -5, h' is 5 to 6, x is the pixel data of the image of the crop removed, and y is the pixel data of the multiple multispectral images generated corresponding to x.

[0010] In one embodiment of the present invention, the plurality of image feature data includes the average, maximum, minimum, variance, and standard deviation of the pixel values ​​of each image in the plurality of multispectral images.

[0011] In one embodiment of the present invention, the water demand module further includes classifying the image feature data through an algorithm to determine the water demand or water shortage status of the crop.

[0012] In one embodiment of the present invention, the algorithm includes Random Tree and Function LDA.

[0013] In one embodiment of the present invention, the crop water requirement model utilizes the multiple image feature data for machine learning.

[0014] In one embodiment of the present invention, the crop water requirement model uses the physiological characteristics data of the crop for machine learning, including chlorophyll content, photosynthetic pigment content and vegetation water index.

[0015] The crop irrigation decision system based on multi-channel virtual multispectral imaging of this invention reflects the health status of crops by analyzing changes in crop physiological data under multispectral conditions. Furthermore, to reduce the cost of multispectral imaging instruments, a method for fitting multi-channel virtual multispectral imaging using RGB data is employed. Multispectral images of different bands are constructed through polynomial fitting technology to realize a crop water early warning system, predicting crop water requirements, thereby improving crop yield and quality and reducing resource waste. Simple Explanation of the Diagram

[0016] To gain a more complete understanding of the embodiments and their advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, wherein: Figure 1 is a schematic diagram of the composition of the crop irrigation decision system 10 with multi-channel virtual multispectral imaging according to the present invention, which can be connected to a display device 160; Figures 2A to 2D show the original images of crops from the early growth stage to maturity, the image correction for areas with strong light in the ExGR-removed image, the ExGR-removed image, and the crop image misjudged as background due to light intensity, respectively; and Figures 3A and 3B show the original image of mature crops and the image of mature crops separated from the background after ExRGB processing, respectively. Implementation

[0017] The following describes some specific embodiments of the present invention; however, the present invention may be practiced in many different forms without departing from the spirit of the invention, and the scope of protection of the present invention should not be construed as limited to those set forth in the specification. Furthermore, unless otherwise stated herein, the terms "a," "the," and similar terms used in this specification (especially in the claims described below) should be understood to include both singular and plural forms.

[0018] The main objective of this invention is to provide a system for predicting crop water demand by analyzing changes in crop physiological data under multispectral conditions to reflect the crop's health status. To reduce the cost of multispectral imaging instruments, this invention develops a method for fitting multichannel virtual multispectral imaging using RGB data, and constructs multispectral images of different bands through polynomial fitting techniques to realize a crop water early warning system and predict crop water requirements.

[0019] Referring first to Figure 1, the present invention provides a crop irrigation decision system 10 with multi-channel virtual multispectral imaging, comprising: an image capturing device 100 configured to capture an image of a crop; an image database module 110 connected to the image capturing device 100 and configured to store the crop image; an image processing module 120 connected to the image capturing device 100 or the image database module 110 and configured to distinguish the crop and background in each crop image using a vegetation index ExGRadj or ExRGB, so as to generate an image of the crop without background; and a multi-channel virtual multispectral module 130 connected to the image processing module 120 and configured to obtain... The system collects images of the crop under multiple different wavelengths and converts these images into multiple multispectral images using a fitting function. A feature analysis module 140 is connected to the multichannel virtual multispectral module 130 and configured to analyze the multiple multispectral images to obtain multiple image feature data. A water demand module 150 is connected to the feature analysis module 140 and has a preset crop water demand model. The crop water demand model uses the multiple image feature data to determine the crop's water demand or whether the crop is in a water-scarce state. If the crop is in a water-scarce state, the water demand module 150 issues a warning.

[0020] Preferably, the crop irrigation decision system 10 of the present invention with multi-channel virtual multispectral imaging can also be connected to a display device 160, configured to display the crop's water demand forecast data.

[0021] The ExGR index (ExGR = ExG - ExR, ExG = 2 × g – r - b, ExR = 1.4 × r - g, where r is the red subpixel value, g is the green subpixel value, and b is the blue subpixel value) proposed in previous studies can be used to remove image background for subsequent image analysis. However, in areas with strong light, the ExGR index is less effective at removing image background, potentially mistaking important crops for background, and it also fails to effectively separate mature crops from the background (such as soil and residue), resulting in calculated results that differ significantly from reality. In this image processing module 120, the inventors of this invention propose new indices, ExGRadj and ExRGB, to address the problem of misclassification of crops as background under strong light and at different maturity stages.

[0022] The formula for calculating ExGRadj is as follows:

[0023] [Formula 1] ExGRadj = max (0, ExGR) × (ExGR+1) × max (0, y - 180), Where y is the brightness, ExR = 1.4 × r - g, ExG = 2 × g – r - b, ExB = 1.4 × b – g, r represents the red sub-pixel value, g represents the green sub-pixel value, and b represents the blue sub-pixel value. In this formula, for image correction of strongly lit areas in the ExGR-backgrounded image, regions with brightness less than 180 (max(0, y - 180)) are selected and calculated with the proposed ExGB index. The calculated ExGB index is then corrected, and finally, image areas with a threshold greater than 0 are identified as potential vegetation. This ExGRadj index can effectively distinguish crops from the background in strongly lit areas, especially for crops from the early growth stage to pre-maturity. Figure 2A shows the original image of a crop from the early growth stage to pre-maturity, Figure 2C shows the image after ExGR backgrounding, and Figure 2B shows the image correction for strongly lit areas in the ExGR-backgrounded image. By stitching the image correction found by the above formula ExGRadj back to the image after ExGR backgrounding (i.e., Figure 2C), the crop image that was misjudged as background due to lighting can be well restored (as shown in Figure 2D).

[0024] Regarding mature crops, since the ExGR index proposed in previous studies could not effectively separate mature crops from the background (such as soil and residue) in images, the inventors of this paper propose the ExRGB index to solve this problem. The calculation formula for ExRGB is as follows:

[0025] [Formula 2] ExRGB = ExR – ExG - ExB Where ExR = 1.4 × r - g, ExG = 2 × g – r - b, ExB = 1.4 × b – g, r represents the red subpixel value, g represents the green subpixel value, and b represents the blue subpixel value. This ExRGB index can effectively distinguish mature crops from the background. Experimental results show that, as shown in Figure 3A, the original image of a mature crop can be effectively separated from the background after ExRGB processing (as shown in Figure 3B).

[0026] The image capturing device 100 can be any suitable device, such as a mobile phone, camera, camcorder, USB lens, etc., as long as it can capture clear crop images.

[0027] In one embodiment of the present invention, in the image database module 110, the area of ​​the crop in the image of the crop accounts for more than 80% of the total area of ​​the image.

[0028] In one embodiment of the present invention, the multi-channel virtual multispectral module 130 is configured to obtain images of the crop removed from the background at multiple different wavelengths, and to convert the images of the crop removed from the background at multiple different wavelengths into multiple multispectral images through a fitting function. The multiple different wavelengths may include at least two of 390 nm, 410 nm, 450 nm, 475 nm, 520 nm, 560 nm, 625 nm, 650 nm, 730 nm, 770 nm, 840 nm, 860 nm, 880 nm, 900 nm, 930 nm, and 960 nm, and different fitting functions are used for conversion in different wavelength bands.

[0029] For example, when the wavelength is in the range of 390-650 nm, the image of the crop behind the back at this wavelength can be converted into a multispectral image using the following fitting function: a * x 7 + b * x 6 + c * x 5 + d * x 4 + e * x 3 + f * x 2 + g * x + h * 10 5 = y, where a to h are constants, a is -3 to -4, b is 1 to 2, c is -1 to -2, d is 1 to 2, e is -8 to -9, f is 25 to 26, g is -4 to -5, and h is 2 to 3. For example, the fitting function is –3.639 * 10 -14 * x 7 + 1.304 * 10 -10 * x 6 – 1.992 * 10 -7 * x 5 + 1.679 * 10 -4 * x 4 – 8.463 *10 2 * x 3 + 25.34 * x 2 – 4.197 * 10 3 * x + 2.964 * 10 5 = y, where x is the pixel data of the image of the crop behind it, and y is the pixel data of the multiple multispectral images generated corresponding to x.

[0030] When the wavelength is in the range of 730-960 nm, the image of the crop behind it at this wavelength is converted into a multispectral image using the following fitting function: a' * x 7 + b' * x 6 + c' * x 5 + d' * x 4 + e' * x 3 + f' * x 2 + g' * x + h' * 10 7 = y, where a' to h' are constants, a' is -1 to -2, b' is 1 to 2, c' is -2 to -3, d' is 3 to 4, e' is -30 to -31, f' is 1 to 2, g' is -4 to -5, and h' is 5 to 6. For example, the fitting function is –1.7184 * 10 -13 * x 7 + 1.019 * 10 -9 * x 6 – 2.588 * 10 -6 * x 5 + 3.645 * 10 -3 * x 4-30.75 * x 3+ 1.554 * 10 3* x 2– 4.3588 * 10 5* x + 5.230 * 10 7= y, where x is the pixel data of the image of the crop behind it, and y is the pixel data of the multiple multispectral images generated corresponding to x.

[0031] Specifically, the pixel data of the crop image is calculated using different vegetation indices ExGRadj or ExRGB based on the crop's maturity stage. After the pixel data of the crop image is input into the above-mentioned fitting function according to different wavelength ranges, the pixel data of the corresponding multiple multispectral images can be calculated.

[0032] In one embodiment of the present invention, the plurality of image feature data may include the average, maximum, minimum, variance, and standard deviation of the pixel values ​​of each image in the plurality of multispectral images. The average value = the sum of the pixel values ​​of each point after removing the backing edge / the size of the entire image; the maximum value = the maximum value of the pixel values ​​after removing the backing edge; the minimum value = the minimum value of the pixel values ​​after removing the backing edge; the variance = 2 * (1 / 2 * (each pixel value after removing the backing edge minus the average value) / the number of pixel values ​​in the entire image); and the standard deviation = the square root of the variance.

[0033] In one embodiment of the present invention, the water demand module 150 further includes classifying the image feature data using an algorithm to determine the water demand or water shortage status of the crop. The algorithm may be, for example, Random Tree, Function LDA, etc.

[0034] Specifically, based on the 16 bands of the aforementioned spectrum (i.e., 390 nm, 410 nm, 450 nm, 475 nm, 520 nm, 560 nm, 625 nm, 650 nm, 730 nm, 770 nm, 840 nm, 860 nm, 880 nm, 900 nm, 930 nm, and 960 nm) and the aforementioned five characteristic values ​​(i.e., mean, maximum, minimum, variance, and standard deviation), 80 features are calculated. These 80 features are then used to train the water demand model. The crop water demand model uses these multiple image feature data to determine the crop's water demand or whether the crop is in a water-scarce state.

[0035] In one embodiment of the present invention, the crop water requirement model can be machine-learned using the physiological characteristics data of the crop, including chlorophyll content, photosynthetic pigment content and vegetation water index.

[0036] Table 1 below shows the verification results of the crop irrigation decision system based on multi-channel virtual multispectral imaging of this invention. The cross-training accuracy and test accuracy were analyzed using data from the first and second crop periods. The cross-training accuracy was determined using 10x cross-validation; for example, 1000 data points were divided into 10 groups of 100 data points each. Nine groups were used to train the crop water requirement model, and the remaining 100 data points were used for 10 tests, yielding the average accuracy. The test accuracy refers to the accuracy of the crop water requirement model after training, using, for example, 200 data points outside the initial 1000 data points. Table 1 shows that both the cross-training accuracy and the test accuracy reached over 90%.

[0037] Table 1 Data Collection Cross-training accuracy Test accuracy First phase of data 92.73% 93.33% Second phase of data 91.30% 91.57%

[0038] In summary, the crop irrigation decision system based on multi-channel virtual multispectral imaging of this invention reflects the health status of crops by analyzing changes in crop physiological data under multispectral conditions. Furthermore, to reduce the cost of multispectral imaging instruments, a method for fitting multi-channel virtual multispectral imaging using RGB data is employed. Multispectral images of different bands are constructed through polynomial fitting technology to realize a crop water early warning system, predicting crop water requirements, thereby improving crop yield and quality and reducing resource waste.

[0039] Although the present invention has been disclosed with reference to preferred embodiments, it is not intended to limit the invention. Any person skilled in the art may make various modifications and alterations without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0040] 10: Crop Irrigation Decision System Based on Multi-channel Virtual Multispectral Imaging 100: Image capturing device 110: Image Database Module 120: Image Processing Module 130: Multi-channel virtual multispectral module 140: Feature Analysis Module 150: Water Demand Module 160: Display device

Claims

1. A crop irrigation decision-making system based on multi-channel virtual multispectral imaging, comprising: An image capturing device is configured to capture images of a crop; an image database module is connected to the image capturing device and configured to store images of the crop; an image processing module is connected to the image capturing device or the image database module and configured to distinguish the crop and background in each crop image using a vegetation index ExGRadj or ExRGB, to generate an image of the crop without background; wherein ExGRadj = max (0, ExGR) × (ExGR+1) × max (0, y - 180), ExGR = ExG - ExR, ExGB = ExG - ExB, y is luminance, ExR = 1.4 × r - g, ExG = 2 × g – r - b, ExB = 1.4 × b – g, r is the red subpixel value, g is the green subpixel value, and b is the blue subpixel value; ExRGB = ExR – ExG - ExB, ExR = 1.4 × r - g, ExG = 2 × g – r - b, ExB = 1.4 × b – g, r is the red sub-pixel value, g is the green sub-pixel value, and b is the blue sub-pixel value; A multi-channel virtual multispectral module is connected to the image processing module and configured to obtain images of the crop without backlighting at multiple different wavelengths, and convert the images of the crop without backlighting at multiple different wavelengths into multiple multispectral images through a fitting function; A feature analysis module is connected to the multi-channel virtual multispectral module and configured to analyze the multiple multispectral images to obtain multiple image feature data; and a water demand module is connected to the feature analysis module and has a preset crop water demand model. The crop water demand model uses the multiple image feature data to determine the water demand of the crop or whether the crop is in a water shortage state. If the crop is in a water shortage state, the water demand module issues an alert.

2. The crop irrigation decision system for multi-channel virtual multispectral imaging as described in claim 1, wherein in the image database module, the area of ​​the crop in the image of the crop accounts for more than 80% of the total area of ​​the image.

3. The crop irrigation decision system for multi-channel virtual multispectral imaging as described in claim 1, wherein the plurality of different wavelengths includes at least two of 390 nm, 410 nm, 450 nm, 475 nm, 520 nm, 560 nm, 625 nm, 650 nm, 730 nm, 770 nm, 840 nm, 860 nm, 880 nm, 900 nm, 930 nm, and 960 nm.

4. The crop irrigation decision system for multi-channel virtual multispectral imaging as described in claim 3, wherein the wavelength is in the range of 390-650 nm, then the image of the crop under the backlight at this wavelength is converted into a multispectral image by the following fitting function: a * x 7 + b * x 6 + c * x 5 + d * x 4 + e * x 3 + f * x 2 + g * x + h * 10 5 = y, where a to h are constants, a is -3 to -4, b is 1 to 2, c is -1 to -2, d is 1 to 2, e is -8 to -9, f is 25 to 26, g is -4 to -5, h is 2 to 3, x is the pixel data of the image of the crop under the backlight, and y is the pixel data of the multiple multispectral images generated corresponding to x.

5. A crop irrigation decision system for multi-channel virtual multispectral imaging as described in claim 3, wherein the wavelength is in the range of 730-960 nm, then the image of the crop under the backlight at this wavelength is converted into a multispectral image by the following fitting function: a' * x 7 + b' * x 6 + c' * x 5 + d' * x 4 + e' * x 3 + f' * x 2 + g' * x + h' * 10 7 = y, where a' to h' are constants, a' is -1 to -2, b' is 1 to 2, c' is -2 to -3, d' is 3 to 4, e' is -30 to -31, f' is 1 to 2, g' is -4 to -5, h' is 5 to 6, x is the pixel data of the image of the crop under the backlight, and y is the pixel data of the multiple multispectral images generated corresponding to x.

6. The crop irrigation decision system based on multi-channel virtual multispectral imaging as described in claim 1, wherein the multiple image feature data includes the average, maximum, minimum, variance, and standard deviation of the pixel values ​​of each image in the multiple multispectral images.

7. The crop irrigation decision system based on multi-channel virtual multispectral imaging as described in claim 1, wherein the water demand module further includes classifying the image feature data through an algorithm to determine the water demand or water shortage status of the crop.

8. A crop irrigation decision system for multi-channel virtual multispectral imaging as described in claim 7, wherein the algorithm includes Random Tree and Function LDA.

9. A crop irrigation decision system based on multi-channel virtual multispectral imaging as described in claim 1, wherein the crop water demand model utilizes the multiple image feature data for machine learning.

10. The crop irrigation decision system based on multi-channel virtual multispectral imaging as described in claim 1, wherein the crop water demand model utilizes physiological characteristic data of the crop for machine learning, including chlorophyll content, photosynthetic pigment content, and vegetation water index.

Images