A spectral compensation method for proximal plant canopy based on hyperspectral three-dimensional point cloud

Abstract

The invention discloses a method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud, comprising the following steps: S1: building an experimental platform and completing the debugging of the equipment; S2: respectively collecting data of hyperspectral images and depth images of experimental samples and checkerboards, and preprocessing the collected image data; S3: based on the checkerboard image taken in S2, performing image registration and fusion on the hyperspectral image and depth image of the preprocessed sample in S2, and generating a hyperspectral three-dimensional point cloud of the plant canopy; S4: realizing spectral compensation of the plant canopy based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud in S3, and reducing the influence of two important factors, namely, working distance and leaf angle; S5: respectively performing k-means clustering analysis and average spectral curve comparison on the original and compensated hyperspectral images of the plant canopy in S4, and evaluating the effectiveness of spectral compensation by comparing the results before and after compensation.

Description

Technical Field

[0001] The present invention relates to the field of plant phenotype research, and in particular to a proximal plant canopy spectral compensation method based on hyperspectral three-dimensional point cloud. Background Art

[0002] In recent years, high-throughput plant phenotyping has flourished, which is of great significance for plant breeding and revealing the growth and stress responses of plants under various environments. Among the many phenotyping techniques, proximal hyperspectral imaging is a very promising technology. Because it can simultaneously obtain spectral and spatial information, quickly and non-contactly characterize the physiological and biochemical characteristics of plants, it is widely used in the study of plants in greenhouses, fields and other environments. In the experimental process of plant phenotyping, a standard white plate with a reflectance close to 100% is usually used as a white reference to calculate the reflectance of the sample. At present, most of the research on proximal hyperspectral imaging technology is limited to flattened detached leaves, which greatly reduces its application value in obtaining in situ phenotyping of plant canopies. In fact, hyperspectral imaging phenotyping of proximal plant canopies is more challenging. Due to the interaction between the three-dimensional complex structure of the plant canopy and the light source, the correct calculation of the reflectance of the plant canopy is often not achieved through traditional two-dimensional white plate correction. Specifically, it will be affected by factors such as working distance, leaf angle, shadow and multivariate scattering, which will interfere with or even mask the real spectral information related to plant biochemical characteristics. Therefore, spectral compensation of the proximal plant canopy is crucial to improve the accuracy of the acquired spectral information and reveal the true optical properties of the plant canopy.

[0003] In terms of spectral compensation of proximal canopy hyperspectral images, there are currently several main methods. The most commonly used spectral preprocessing techniques from the field of chemometrics, such as multiple scattering correction (MSC), standard normal variate (SNV), and variable sorting for normalization (VSN), have shown effective spectral compensation effects. However, due to the large performance differences of different spectral preprocessing techniques on different data sets, it is impossible to determine the most effective preprocessing technique for all data sets; the second method is the PROCOSINE model based on physical principles, which is a radiation transfer model that introduces parameters that characterize leaf angles and specular reflections. The estimation of plant physiological and biochemical characteristics at the leaf scale has good robustness, but its feasibility in the plant canopy is still unknown; in addition, deep learning is another spectral compensation method, but deep learning is often based on the training of large data sets, which is usually difficult to obtain in the field of plant phenotyping.

[0004] In recent years, some scholars have found that plant canopy three-dimensional (3D) data and two-dimensional hyperspectral images are highly complementary. The fused hyperspectral three-dimensional point cloud provides the spectrum, depth and local angle information of each point, which has great potential in precise plant phenotyping research and provides feasibility for spectral compensation of proximal canopy hyperspectral images. At present, related research mainly stays on the fusion of three-dimensional data and hyperspectral images. Some scholars capture multispectral images at different angles and align them to the same coordinate system to generate multispectral three-dimensional point clouds, thereby realizing accurate identification of pests; others simultaneously capture multi-view depth images and multispectral images of plant canopies, align the spectral reflectance to the depth image coordinate system, fuse the two, and realize accurate prediction of chlorophyll content. However, there are few reports on the spectral compensation of proximal plant canopies by fusing 3D data. Some scholars have made specific 3D white hemisphere references, used hyperspectral cameras and Kinect depth cameras to shoot 3D white hemisphere images at multiple positions, and combined deep learning to build a 3D white reference library for the canopy, thereby achieving compensation for the canopy spectral reflectance. The compensated results are closer to the standard data measured by the spectrometer. However, this method of constructing a 3D white reference is very complex, time-consuming and labor-intensive, and cannot be actually applied in practice; in addition, due to the low resolution of the Kinect sensor, the generated hyperspectral three-dimensional point cloud is relatively sparse, resulting in insufficient compensation results.

[0005] In summary, the problems existing in the prior art are:

[0006] (1) The performance of spectral compensation methods based on spectral preprocessing technology is unstable, and it is impossible to determine the most effective preprocessing technology that is applicable to all data sets; the spectral compensation method based on the PROCOSINE model is only applicable to the leaf scale, and its feasibility at the canopy scale is still unknown; the compensation method based on deep learning relies too much on the modeling of large data sets, which is difficult to obtain in the field of plant phenotyping.

[0007] (2) The spectral compensation method based on the 3D white reference database is very complex, time-consuming and labor-intensive, lacks applicability in practice, and the generated hyperspectral three-dimensional point cloud is sparse, and the compensation result is still insufficient.

[0008] Therefore, in the field of plant phenotyping, an efficient and accurate proximal plant canopy spectral compensation method is urgently needed to reveal the true optical properties of plants and promote the further development of precise plant phenotyping. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud, and its specific technical solution is as follows:

[0010] A method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud comprises the following steps:

[0011] S1. Experimental platform construction: Build the experimental platform and complete the equipment debugging;

[0012] S2. Image data acquisition and preprocessing: collect data of hyperspectral images and depth images of experimental samples and checkerboards respectively, and perform preprocessing such as background segmentation and noise removal on the collected image data;

[0013] S3. Image registration and fusion: Based on the checkerboard image captured in step S2, the hyperspectral image and the depth image of the preprocessed sample in step S2 are image registered and fused to generate a hyperspectral three-dimensional point cloud of the plant canopy;

[0014] S4. Plant canopy spectral compensation: based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud in step S3, the spectral compensation of the plant canopy is realized, and the influence of two important factors, working distance and leaf angle, is reduced;

[0015] S5. Evaluation of spectral compensation results: Perform k-means cluster analysis and average spectral curve comparison on the original and compensated plant canopy hyperspectral images in step S4, and compare the results before and after compensation to evaluate the effectiveness of spectral compensation.

[0016] The proximal plant canopy spectral compensation method provided by the present invention helps to accurately calculate the canopy spectral reflectance to reveal the true optical properties of the plant canopy. It also has high compensation accuracy, good adaptability and high robustness, and has great promotion value in the field of precise plant phenotyping.

[0017] Preferably, the experimental platform described in step S1 mainly includes a SNAPSCAN VNIR hyperspectral camera (IMEC, Leuven, Belgium), a Raytrix light field camera (GmbH, Kiel, Germany), a 150W ring halogen lamp light source and two 100W LED white light sources, wherein the lens of the hyperspectral camera is 35mm, and the lens of the light field camera is 17mm;

[0018] The SNAPSCANVNIR hyperspectral camera in the experimental platform is different from the traditional linear scanning camera. Its sensor moves inside the camera, so it can ensure the relative stillness between the sample and the camera, which greatly facilitates the acquisition of hyperspectral images.

[0019] The Raytrix light field camera in the experimental platform can obtain information such as color, depth, and 3D point cloud of the target sample through a single shot;

[0020] The annular halogen light source in the experimental platform is mainly used for the hyperspectral camera, and the two LED white light sources are mainly used for the light field camera, wherein the halogen light source is coaxial and in the same direction as the hyperspectral camera;

[0021] In the experimental platform construction described in step S1, the hyperspectral camera and the light field camera are closely attached to each other and fixed on a bracket, about 45 cm from the ground, and the two cameras are about 100 cm away from the experimental sample to ensure that the sample canopy is within the field of view of the two cameras;

[0022] The debugging of the equipment in S1 includes the adjustment of the camera aperture, exposure time and light source intensity. The aperture of the hyperspectral camera is set to 3, the exposure time is set to 32ms, and the aperture of the light field camera is set to 4, and the exposure time is set to 30ms. The above-mentioned equipment debugging is performed to ensure the clear collection of the sample canopy spectral information and depth information, which is convenient for the spectral compensation of the plant canopy.

[0023] Preferably, each experimental sample in step S2 is consistent with the image acquisition environment of its checkerboard, and the specific steps are to first acquire the hyperspectral image and depth image of the sample, then lean the checkerboard against the sample flower pot, and then acquire the two images of the checkerboard;

[0024] The experimental samples in step S2 are normally cultivated perilla seedlings and tea seedlings, which grow at about V5-V7 stages, have a height of about 30 cm, and a canopy depth of about 15 cm;

[0025] In step S2, the chessboard is black and white in color, 9×9 in size, 10×10 cm in size, and 1×1 cm in pattern size;

[0026] The hyperspectral image in step S2 is corrected using white reference and black reference images and the reflectance is calculated. During the correction process, a white board is placed behind the plant canopy. The correction formula is as follows (1):

[0027]

[0028] Where R is the corrected image, R0 is the original hyperspectral image, and R d and R w They are black reference image and white reference image;

[0029] Before the depth image acquisition in step S2, the Raytrix light field camera is self-calibrated to ensure accurate acquisition of subsequent depth information;

[0030] The spatial resolution of the hyperspectral image collected in step S2 is 1500×1024, the spectral range is 610-850nm, and the spatial resolution of the depth image is 1102×766.

[0031] The image preprocessing in step S2 is mainly for hyperspectral images, including background segmentation and noise removal. The threshold band selected for background segmentation is 577nm, and the threshold value is 0.015.

[0032] Preferably, the image registration in step S3 includes two steps, namely, detecting corner points of a chessboard and solving a homography transformation matrix;

[0033] The corner point detection algorithm in step S3 is based on the following formula (2):

[0034]

[0035] Where Ci, Cj are two corner points that are close to each other in the chessboard image. represents the grayscale gradient of point Cj, Ci-Cj represents the direction vector between these two points, and the dot product of the two is always 0;

[0036] The corner point detection in step S3 is implemented based on the OpenCV toolkit, the coding platform is Python 3.7, and the search window size of the corner point is 11×11;

[0037] The solution of the homography transformation matrix in step S3 is based on the following formula (3):

[0038]

[0039] Where P h represents a point in the hyperspectral image, P d ' represents the transformed point in the depth image, (x i ,y i ) represents the image coordinates, and e represents the allowed error. The optimal homography transformation matrix is ​​obtained through iterative calculation;

[0040] The image registration in step S3 is implemented at the pixel level;

[0041] The generation of the plant canopy hyperspectral three-dimensional point cloud in step S3 is obtained by fusing the registered hyperspectral image with the depth image, which includes two steps, namely, obtaining the internal and external parameters of the camera and constructing the hyperspectral three-dimensional point cloud.

[0042] The acquisition of the camera internal and external parameters in S3 is to calibrate the Raytrix light field camera based on the checkerboard image to obtain the internal and external parameter matrix m of the light field camera;

[0043] The construction of the hyperspectral three-dimensional point cloud described in S3 is based on the following formula (4):

[0044]

[0045] Where s is the scaling factor, (u, v) are the pixel coordinates of the depth image, m is the 3*4 light field camera parameter matrix, and (x, y, z) are the homogeneous coordinates of the point in the point cloud.

[0046] Preferably, the spectral compensation of the plant canopy in step S4 is mainly aimed at compensating the working distance and the leaf angle factors. The original spectral reflectance of a certain point in the canopy is expressed as follows:

[0047]

[0048] where λ m is the collected spectral reflectance, λ r is the true reflectivity of the canopy, d w is the distance between the whiteboard and the camera during whiteboard calibration of the hyperspectral camera, d is the distance between the point on the canopy and the camera, and θ is the angle between the normal vector of the point on the canopy and the direction of the incident light. In the formula, the effects of working distance and blade angle on the original spectral reflectance follow the inverse square law and Lambert's cosine law, respectively.

[0049] The spectral compensation of the plant canopy in step S4 is based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud, and the original spectral reflectance of each point in the canopy is compensated by the following formula (6) to reduce the influence of two important factors: working distance and leaf angle:

[0050]

[0051] Among them, θ can estimate the normal vector of each point through the K-Nearest Neighbor (KNN) algorithm, and then calculate the angle between it and the incident light, and the size of K is set to 300;

[0052] The spectrum compensation in step S4 is performed point by point, so as to achieve point by point compensation of the plant canopy spectrum.

[0053] Preferably, the compensation result in step S5 includes the results of distance compensation (DC), inclination compensation (IC) and both compensation (BC);

[0054] The optimal number of clusters for the k-means cluster analysis described in S5 is determined using the elbow method, which is dedicated to reducing the sum of squared clustering errors to determine the optimal number of category clusters k. Compared with the silhouette coefficient method, it is more accurate in determining the optimal number of clusters for different sample data sets. The core idea of ​​the elbow method is that as the number of clusters k increases, the sample division will become more refined, the degree of aggregation of each cluster will be higher, the sum of squared errors will decrease sharply, and then starting from a certain k class, the decrease in the sum of squared errors will become slow, and at this time k is the optimal number of clusters;

[0055] The results of the k-means clustering analysis and average spectral curve evaluation described in S5 showed that most of the pixels in the plant canopy after BC compensation were clustered into one category, and the average spectral curve of the canopy was closer to the standard curve, indicating that the plant canopy spectrum was well compensated, and the adverse effects of working distance and leaf angle on the canopy spectrum were effectively eliminated.

[0056] The beneficial effects of the present invention are:

[0057] (1) The present invention aims to solve the problem that the spectrum of the proximal plant canopy cannot obtain the correct reflectance through the traditional two-dimensional whiteboard correction, and proposes a point-by-point and accurate compensation method for the spectrum of the proximal plant canopy based on the hyperspectral three-dimensional point cloud;

[0058] (2) Compared with the complex spectral compensation method for constructing a 3D white reference database, the method proposed in the present invention is based on the laws of physics and only relies on the precise synchronous acquisition of canopy hyperspectral images and depth maps. This method is efficient and accurate, has high repeatability, good adaptability, and high robustness.

[0059] (3) The proximal plant canopy spectral compensation method proposed in the present invention can be widely applied in the field of plant phenotyping and has great potential in realizing the accurate calculation of the proximal plant canopy spectral reflectance and revealing the true optical properties of the plant canopy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of the spectral compensation method of proximal plant canopy based on hyperspectral three-dimensional point cloud;

[0061] Figure 2 Schematic diagram of the registration results of plant canopy hyperspectral image and depth image ((A) is a single-channel spectral image, (B) and (C) are depth images before and after registration, respectively);

[0062] Figure 3 Schematic diagram of the hyperspectral three-dimensional point cloud of the canopy of Perilla (a) and tea seedling sample (b) at the 795.4nm band;

[0063] Figure 4The k-means clustering results and average spectral curve comparison results of the canopy of Perilla (a) and tea seedling samples (b) before and after compensation; DETAILED DESCRIPTION

[0064] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principles of the present invention, several improvements and modifications may be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0065] In view of the problems existing in the prior art, the present invention provides a method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud. The present invention is described in detail below in conjunction with the accompanying drawings:

[0066] A method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud, the flow chart is as follows Figure 1 As shown, the specific steps include:

[0067] S1. Experimental platform construction: Build the experimental platform and complete the debugging of the equipment. The specific method is: fix the SNAPSCANVNIR hyperspectral camera and the Raytrix light field camera 45cm from the ground, ensure that the two cameras are as close as possible, and at the same time about 100cm away from the experimental sample. Hang the annular halogen light source on the hyperspectral lens, and arrange the two LED white light sources symmetrically on both sides of the sample to ensure that the light source is evenly directed to the sample.

[0068] The annular halogen light source in the experimental platform is mainly used for the hyperspectral camera, and the two LED white light sources are mainly used for the light field camera. During the experiment, the halogen light source and the hyperspectral camera are coaxial and in the same direction;

[0069] The power of the annular halogen light source is 150W, and the power of the two LED white light sources is 100W;

[0070] The hyperspectral lens model is 35mm, and the light field camera lens model is 17mm;

[0071] The aperture of the hyperspectral lens is adjusted to 3, and the aperture of the light field camera lens is adjusted to 4;

[0072] S2. Image data acquisition and preprocessing: Collect hyperspectral images and depth images of experimental samples and checkerboards respectively, and perform preprocessing such as background segmentation and noise removal.

[0073] The experimental samples were normally cultivated perilla seedlings and tea seedlings, which were about V5-V7 growth stages, with a height of about 30 cm and a canopy depth of about 15 cm;

[0074] The hyperspectral image is corrected using white reference and black reference images and the reflectance is calculated. During the correction process, a white board is placed behind the plant canopy, and the correction formula is as follows (1):

[0075]

[0076] Where R is the corrected image, R0 is the original hyperspectral image, and R d and R w They are black reference image and white reference image;

[0077] Before the depth image acquisition described in S2, the Raytrix light field camera is self-calibrated to ensure the accurate acquisition of subsequent depth information. Specifically, the lens of the light field camera is covered with a ring calibration plate, and the light source, aperture, and exposure time are adjusted at the same time, so that the depth and three-dimensional information of the target in the field of view are rich enough;

[0078] Each experimental sample described in S2 is consistent with the image acquisition environment of its checkerboard. The specific steps are to first collect the hyperspectral image and depth image of the sample, then lean the checkerboard against the sample flower pot, and then collect two images of the checkerboard;

[0079] The chessboard is black and white in color, 9×9 in size, 10×10 cm in size, and 1×1 cm in pattern size;

[0080] The spatial resolution of the hyperspectral image is 1500×1024, the spectral range is 610-850nm, and the spatial resolution of the depth image is 1102×766;

[0081] The image preprocessing is mainly aimed at hyperspectral images, including background segmentation and noise removal. The threshold band selected for background segmentation is 577nm, and the threshold value is 0.015.

[0082] Step S3. Image registration and fusion: Based on the checkerboard image captured in step S2, the hyperspectral image and depth image of the preprocessed sample in step S2 are image registered and fused to generate a hyperspectral three-dimensional point cloud of the canopy.

[0083] S3 The image registration includes two steps, namely, detecting the corner points of the checkerboard to determine the feature points in the checkerboard image, and realizing the registration of the sample depth image and the hyperspectral image based on the solution and application of the homography transformation matrix;

[0084] The corner detection algorithm is based on the following formula (2):

[0085]

[0086] Among them C i , Cj are two corner points that are close to each other in the chessboard image. Indicates point C j Gray gradient, C i -C j Represents the direction vector between these two points, and the dot product between the two is always 0;

[0087] The corner point detection is implemented based on the OpenCV toolkit, the coding platform is Python 3.7, and the search window size of the corner point is 11×11;

[0088] The solution of the homography transformation matrix is ​​based on the following formula (3):

[0089]

[0090] Where P h represents a point in the hyperspectral image, P d ' represents the transformed point in the depth image, (x i ,y i ) represents the image coordinates, and e represents the allowed error. The optimal homography transformation matrix is ​​obtained through iterative calculation;

[0091] S3 The image registration is achieved at the pixel level;

[0092] The result of image registration described in S3 is as follows Figure 2 As shown. (A) is a single-channel spectral image, (B) and (C) are depth images before and after registration respectively;

[0093] The generation of the plant canopy hyperspectral three-dimensional point cloud described in S3 is obtained by fusing the registered hyperspectral image with the depth image, which includes two steps, namely, obtaining the internal and external parameters of the camera and constructing the hyperspectral three-dimensional point cloud.

[0094] The camera internal and external parameters are obtained by calibrating the Raytrix light field camera based on the checkerboard image to obtain the internal and external parameter matrix m of the light field camera;

[0095] The construction of the hyperspectral three-dimensional point cloud is based on the following formula (4):

[0096]

[0097] Where s is the scaling factor, (u, v) are the pixel coordinates of the depth image, m is the 3*4 light field camera parameter matrix, and (x, y, z) are the homogeneous coordinates of the point in the point cloud.

[0098] The hyperspectral three-dimensional point cloud generated in step S3 is as follows: Figure 3As shown in Figure 1, (a) shows the hyperspectral 3D point cloud of Perilla frutescens, and (b) shows the hyperspectral 3D point cloud of Tea seedlings.

[0099] S4. Plant canopy spectral compensation: Based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud in step S3, the spectral compensation of the plant canopy is realized to reduce the influence of two important factors: working distance and leaf angle.

[0100] The spectral compensation of the plant canopy is mainly aimed at compensating the working distance and leaf angle factors. The original spectral reflectance of a certain point in the canopy is expressed as follows (5):

[0101]

[0102] where λ m is the collected spectral reflectance, λ r is the true reflectivity of the canopy, d w is the distance between the whiteboard and the camera during whiteboard calibration of the hyperspectral camera, d is the distance between the point on the canopy and the camera, and θ is the angle between the normal vector of the point on the canopy and the direction of the incident light. In the formula, the effects of working distance and blade angle on the original spectral reflectance follow the inverse square law and Lambert's cosine law, respectively.

[0103] The spectral compensation of the plant canopy described in S4 is based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud. The original spectral reflectance of each point in the canopy is compensated by the following formula (6) to reduce the influence of two important factors: working distance and leaf angle:

[0104]

[0105] Among them, θ can estimate the normal vector of each point through the K-Nearest Neighbor (KNN) algorithm, and then calculate the angle between it and the incident light, and the size of K is set to 300;

[0106] The spectrum compensation described in S4 is performed point by point, so as to achieve point by point compensation of the plant canopy spectrum.

[0107] S5. Evaluation of spectral compensation results: Perform k-means cluster analysis and average spectral curve comparison on the original and compensated plant canopy hyperspectral images in step S4, and compare the results before and after compensation to evaluate the effectiveness of spectral compensation.

[0108] The compensation results include distance compensation (DC), inclination compensation (IC) and both compensation (BC).

[0109] The optimal number of clusters for the k-means cluster analysis described in S5 is determined using the elbow method, which aims to reduce the sum of squared clustering errors to determine the optimal number of category clusters k. The core idea of ​​the elbow method is that as the number of clusters k increases, the sample division will become more refined, the degree of aggregation of each cluster will be higher, the sum of squared errors will decrease sharply, and then starting from a certain k class, the decrease in the sum of squared errors will become slow, and at this time k is the optimal number of clusters;

[0110] The results of k-means cluster analysis and average spectral curve comparison described in S5 are shown in Figure 4 As shown, (a) and (b)

[0111] The pictures show the comparison results of Perilla and tea seedlings;

[0112] In the k-means clustering analysis results described in S5, the leaves in the original canopy were clustered into three categories due to different distances and angles; the clustering results after DC did not change much; the clustering results after IC were more uniform, indicating that the canopy spectrum was more affected by the angle; and after BC, most leaves in the canopy were clustered into one category, and the spectral information of the entire canopy became more homogeneous, indicating that the influence of distance and angle on the canopy spectrum has been reduced to a great extent;

[0113] In the spectral curve comparison described in S5, spectra of four in vitro flattened leaves at different distances and angles in each canopy are collected, and the average spectral curve of the four leaves is used as the standard curve;

[0114] In the comparison of the spectrum curves described in S5, the spectrum curves gradually flatten out after 750 nm, so the curves around 800 nm are used as comparison data;

[0115] In the spectral curve comparison results described in S5, the standard spectral curve is approximately 0.52, the original canopy average spectral curve value is approximately 0.45, the average spectral curve value after DC is reduced to below 0.4, the average spectral curve after IC increases to approximately 0.57, and the average spectral curve after BC is approximately 0.5, which is closer to the standard curve, indicating that the influence of distance and angle on the plant canopy spectrum is effectively reduced.

[0116] The results of the k-means clustering analysis and average spectral curve evaluation described in S5 showed that most of the pixels in the plant canopy after BC compensation were clustered into one category, and the average spectral curve of the canopy was closer to the standard curve, indicating that the plant canopy spectrum was well compensated, and the adverse effects of working distance and leaf angle on the canopy spectrum were effectively eliminated.

[0117] In summary, the technical effect of the present invention is remarkable, and it has made a good technical contribution to the development of precise plant phenotyping. It has broad application prospects in the field of proximal plant canopy phenotyping research and considerable economic benefits. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0118] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. All equivalent changes or modifications made according to the structure, characteristics and principles described in the patent application scope of the present invention should be included in the patent application scope of the present invention.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud, characterized in that: The following steps are involved: S1. Experimental platform construction: Build the experimental platform and complete the equipment debugging; S2. Image data acquisition and preprocessing: collect data of hyperspectral images and depth images of experimental samples and checkerboards respectively, and preprocess the collected image data; S3. Image registration and fusion: Based on the checkerboard image captured in step S2, the hyperspectral image and the depth image of the preprocessed sample in step S2 are image registered and fused to generate a hyperspectral three-dimensional point cloud of the plant canopy; S4. Plant canopy spectral compensation: spectral compensation of the plant canopy is achieved based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud in step S3; S5. Evaluation of spectral compensation results: Perform k-means clustering analysis and average spectral curve comparison on the original plant canopy hyperspectral image and the compensated image in step S4, and evaluate the effectiveness of spectral compensation by comparing the results before and after compensation; The image registration of the hyperspectral image and the depth image in step S3 includes two steps, namely, the corner point detection of the chessboard and the solution of the homography transformation matrix; The generation of the plant canopy hyperspectral three-dimensional point cloud in step S3 is obtained by fusing the registered hyperspectral image with the depth image, and the image fusion includes two steps, namely, obtaining the internal and external parameters of the light field camera and constructing the hyperspectral three-dimensional point cloud; The acquisition of the internal and external parameters of the light field camera is to calibrate the Raytrix light field camera based on the checkerboard image to obtain the internal and external parameter matrix m of the light field camera; The construction of the hyperspectral three-dimensional point cloud is based on the following formula (4): Where s is the scaling factor, (u, v) is the pixel coordinates of the depth image, m is the 3*4 light field camera parameter matrix, and (x, y, z) is the homogeneous coordinates of the point in the point cloud; S3 The image registration includes two steps, namely, detecting the corner points of the checkerboard to determine the feature points in the checkerboard image, and realizing the registration of the sample depth image and the hyperspectral image based on the solution and application of the homography transformation matrix; The corner detection algorithm is based on the following formula (2): Where Ci, Cj are two corner points that are close to each other in the chessboard image. represents the grayscale gradient of point Cj, Ci-Cj represents the direction vector between these two points, and the dot product of the two is always 0; The corner point detection is implemented based on the OpenCV toolkit, the coding platform is Python 3.7, and the search window size of the corner point is 11×11; The solution of the homography transformation matrix is ​​based on the following formula (3): Where Ph represents a point in the hyperspectral image, P′ d represents the transformed point in the depth image, (xi, yi) represents the image coordinates, e represents the allowable error, and the optimal homography transformation matrix is ​​obtained by iterative calculation.

2. The method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud according to claim 1, characterized in that: The experimental platform described in step S1 mainly includes a SNAPSCAN VNIR hyperspectral camera, a Raytrix light field camera, a 150W ring halogen lamp light source and two 100W LED white light sources, wherein the lens of the hyperspectral camera is 35mm and the lens of the light field camera is 17mm; The annular halogen light source is mainly used for a hyperspectral camera, and the two LED white light sources are mainly used for a light field camera, wherein the halogen light source is coaxial and in the same direction as the hyperspectral camera; When the experimental platform is built in step S1, the hyperspectral camera and the light field camera are closely attached to each other and fixed on a bracket, and the two cameras are 45 cm away from the ground and 100 cm away from the experimental sample; The debugging of the equipment in step S1 includes adjusting the camera aperture, exposure time and light source intensity. The aperture of the hyperspectral camera is set to 3, and the exposure time is set to 32ms; the aperture of the light field camera is set to 4, and the exposure time is set to 30ms.

3. The method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud according to claim 1, characterized in that: In step S2, each experimental sample and its checkerboard image acquisition environment are consistent, and the specific steps are to first acquire the hyperspectral image and depth image of the sample, then lean the checkerboard against the sample flower pot, and then acquire the two images of the checkerboard; In step S2, the checkerboard color is black and white, the specification is 9×9, the size is 10×10 cm, and the pattern size is 1×1 cm; The image preprocessing in step S2 is mainly for hyperspectral images, and the preprocessing methods mainly include background segmentation and noise removal; the threshold band selected for background segmentation is 577nm, and the threshold size is 0.

015.

4. The method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud according to claim 1, characterized in that: The spectral compensation of the plant canopy in step S4 is mainly aimed at compensating the working distance and the leaf angle factors. The original spectral reflectance of any point in the canopy is expressed as follows: where λ m is the collected spectral reflectance, λ r is the true reflectivity of the canopy, d w is the distance between the white board and the camera during white board calibration of the hyperspectral camera, d is the distance between the point on the canopy and the camera, θ is the angle between the normal vector of the point on the canopy and the direction of the incident light, and in this formula, the effects of the working distance and the blade angle on the original spectral reflectance follow the inverse square law and Lambert's cosine law, respectively; The spectral compensation of the plant canopy in step S4 is based on the depth and angle information of each point in the hyperspectral three-dimensional point cloud, and the original spectral reflectance of each point in the canopy is compensated by the following formula (6): Among them, θ can estimate the normal vector of each point through the K nearest neighbor algorithm, and then calculate the angle between it and the incident light, and the size of K is set to 300; The spectrum compensation in step S4 is performed point by point.

5. The method for spectral compensation of proximal plant canopy based on hyperspectral three-dimensional point cloud according to claim 1, characterized in that: The compensation result in step S5 includes the result of distance compensation, angle compensation and compensation of both; The optimal number of clusters for the k-means cluster analysis in step S5 is determined using the elbow method.

Images