A method for three-dimensional reconstruction and phenotypic feature extraction of hydroponic plant roots
The root image of hydroponic plants is obtained through rotating device and video frame extraction technology, distortion and refraction correction are performed, and three-dimensional reconstruction is achieved by combining the SFM algorithm, which solves the problem of low accuracy caused by the large number of noise points and light refraction in the existing technology, and achieves low cost , high-precision three-dimensional model of plant roots and phenotypic feature extraction.
Patent Information
- Application Number
- CN202410759491.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-06-13
AI Technical Summary
In the prior art, the three-dimensional reconstruction and phenotypic feature extraction methods of hydroponic plant roots have problems such as many noise points, low accuracy, high cost and high operation difficulty, and it is impossible to obtain low-cost and high-precision three-dimensional model. Batch phenotypic feature extraction.
The rotating device is used to carry the culture container for rotation, and the camera takes the root video at a fixed position. The RGB image is obtained through the video extraction frame, the image is divided and the root part is retained, and the distortion and refraction correction is performed. The SFM algorithm is used for three-dimensional reconstruction, and the three-dimensional phenotypic characteristic parameters are extracted.
It improves the accuracy of three-dimensional reconstruction, reduces errors, reduces costs, and realizes rapid acquisition of plant root images, supports large-scale three-dimensional models and phenotypic feature extraction, which facilitates high-throughput analysis.
Smart Images

Figure CN118505784B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision technology and biological feature recognition, and specifically relates to a method for three-dimensional reconstruction of a hydroponic plant root system and extraction of phenotypic features. Background Art
[0002] Three-dimensional reconstruction technology based on computer vision has shown great application potential in the field of plant phenotyping research. Among them, the study of plant root phenotype is of great significance for improving plant varieties, increasing the efficiency of plant absorption and utilization of nutrients, and improving soil quality. Plants can be hydroponically cultivated to directly observe the root configuration of plants. However, in the process of collecting root images, the root images are distorted due to light refraction, which reduces the accuracy of three-dimensional reconstruction. Therefore, a three-dimensional reconstruction method is urgently needed to perform refraction correction on the root images of hydroponic plants and obtain a high-precision three-dimensional root model to facilitate the subsequent extraction of phenotypic features.
[0003] At present, there are few studies on the three-dimensional reconstruction and phenotypic feature extraction of plant roots, and most of them need to rely on high-precision hardware equipment to obtain root configuration and phenotypic characteristics. For example, the publication number CN111288920A proposes a plant root configuration detection and analysis system. Through laser measurement, the plant roots can be located and counted without touching the plant roots. Although this method can complete the root measurement without destroying the plant roots, it requires the use of a large number of laser transmitters and receivers, and can only reconstruct a pseudo three-dimensional root structure. The publication number CN111693551A proposes a three-dimensional trait measurement device and method for rice roots. The rice roots are scanned using CT, and the root tomographic image is obtained and reconstructed using the FDK algorithm, and the three-dimensional trait parameters of the rice roots such as the maximum root length and root volume are extracted. Although this method can reconstruct a visualized three-dimensional model of the rice root system and extract three-dimensional trait parameters, it needs to be implemented with the help of CT equipment, which is costly and has low reconstruction efficiency.
[0004] Comprehensive analysis of existing technologies shows that most of the current research on plant root 3D reconstruction and phenotypic feature extraction requires the use of high-precision equipment, which is costly and difficult to operate, and cannot achieve large-scale plant root 3D reconstruction and phenotypic feature extraction. Therefore, there is an urgent need for a 3D reconstruction method for plant roots to achieve low-cost and high-precision acquisition of plant root 3D models, combined with a phenotypic feature extraction method to achieve large-scale plant root phenotypic feature extraction, which is convenient for high-throughput analysis. Summary of the invention
[0005] The technical problems to be solved by the present invention include the following aspects. In the study of three-dimensional reconstruction of hydroponic plant roots, due to the interference of water and culture containers, the reconstructed three-dimensional model has many noise points, making it difficult to observe the root configuration; at the same time, due to the refraction of light during the imaging process, the collected image has a refraction error, resulting in a low accuracy of the three-dimensional model reconstructed using the original image, which is greatly different from the true configuration of the root system, and the extracted phenotypic characteristics have large errors. A method for three-dimensional reconstruction of plant roots and extraction of phenotypic characteristics is urgently needed. However, the structure of plant roots is complex, and the thickness of the main root and lateral roots varies; at the same time, the root phenotypic characteristics of different plant species and different growth periods are different. The above two points have caused great difficulties in the collection and processing of plant root images. In addition, in the process of collecting images of hydroponic plant roots, the light passes through the container and water respectively, and is refracted twice. Therefore, when refraction correction is performed on the root image, it is necessary to consider the two refractions, which causes difficulties in technical implementation.
[0006] In order to solve the above problems, the present invention provides the following technical solutions, and proposes a method for three-dimensional reconstruction of the root system of hydroponic plants and extraction of phenotypic characteristics, as follows.
[0007] A method for three-dimensional reconstruction and feature extraction of hydroponic plant roots, characterized in that it comprises the following steps:
[0008] Step S1, planting the plant in a cylindrical transparent glass culture container containing a nutrient solution in a hydroponic manner;
[0009] Step S2, using a rotating device to carry the culture container, the rotating device drives the culture container to rotate, and the rotation center axis coincides with the cylinder center axis of the culture container; using a fixed position camera to shoot a video of the hydroponic plant root system rotating for one week from the side of the culture container, and then obtaining the plant root system RGB image in a video frame extraction manner;
[0010] Step S3, segmenting the collected root system RGB image to retain only the root system part in the image;
[0011] Step S4, performing distortion correction and refraction correction on the root system partial image;
[0012] Step S5, based on the corrected image, use the SFM algorithm to perform three-dimensional reconstruction to obtain a point cloud model;
[0013] Step S6, extracting three-dimensional phenotypic characteristic parameters of the hydroponic plant root system based on the point cloud model.
[0014] Preferably, in step S1, in the culture container, a KT plate is used to fix the plant rhizome at the center of a circular surface on the upper surface of the nutrient solution.
[0015] Preferably, in step S2, the rotation speed of the rotating device is set to 18 seconds / revolution, the length of the video shot by the camera is set to 18 seconds, the video frame rate is set to 25 frames / second, the frame interval of the video frame extraction is set to 2 frames, and a total of 150 plant root RGB images are obtained in one rotation.
[0016] Preferably, in step S3, the specific steps of segmenting the root system RGB image are as follows:
[0017] Step S31, performing ROI extraction on the root system RGB image, where the ROI area is the underwater part of the plant root system;
[0018] Step S32, performing global threshold segmentation on the ROI area image, setting the image background to black, and copying the image color information;
[0019] Step S33, converting the image into a grayscale image, extracting the outline of the plant root system based on the canny edge detection algorithm, and removing small connected domains;
[0020] Step S34, mapping the image color information copied in step S32 to the image obtained in step S33.
[0021] Preferably, in step S4, the specific steps of performing distortion correction and refraction correction on the image are as follows:
[0022] Step S41, calculating the radial distortion coefficient and the tangential distortion coefficient of the camera based on Zhang calibration method;
[0023] Step S42, establishing a distortion correction mapping table based on the radial distortion coefficient and the tangential distortion coefficient of the camera, mapping the pixels of the root part obtained in step S3 to the corrected position, and completing the distortion correction;
[0024] Step S43, calculate the refraction error of a certain point of the plant root in the vertical direction according to formula (1): :
[0025] (1)
[0026] According to formula (2), the refraction error at a certain point of the plant root in the horizontal direction is calculated. :
[0027] (2)
[0028] In the formula, t is the thickness of the cylindrical glass culture container, L1 is the distance between the camera lens and the outer wall of the cylindrical glass culture container, L2 is the distance between the inner wall of the cylindrical glass culture container and the center of the plant root system, Y is the vertical distance between a certain point of the plant root system and the optical center of the camera, X is the horizontal distance between a certain point of the plant root system and the optical center of the camera, and L is the horizontal distance between the center of the plant root system and the camera lens. , , are the refractive indices of light in air, glass, and water respectively;
[0029] Step S44, subtract the refraction error from the coordinate values of all the pixels on the root system image and then map them to new positions to complete the refraction correction.
[0030] Preferably, in step S5, the SFM algorithm is used to perform three-dimensional reconstruction to obtain a point cloud model, and the specific steps are as follows:
[0031] Step S51, extracting feature points from the root system image obtained in step S4 based on a scale-invariant feature conversion method;
[0032] Step S52, using a KD tree algorithm to match the feature points extracted in step S51;
[0033] Step S53, estimating the camera pose based on a triangulation method to obtain a sparse point cloud model of the plant root system;
[0034] Step S54, based on the dense matching method of hash image clustering and patch model, the sparse point cloud model is densified to obtain a dense point cloud model of the plant root system, and the point cloud model is output as a result.
[0035] Preferably, in step S6, the specific steps of extracting the three-dimensional phenotypic characteristic parameters of the plant root system are as follows:
[0036] Step S61, segmenting the three-dimensional point cloud model of the plant root system based on a region growing algorithm, and only retaining the taproot point cloud of the plant root system;
[0037] Step S62, counting the number of taproots, that is, the number of taproots in the three-dimensional phenotypic characteristics of the plant root system;
[0038] Step S63, calculating the distance between the topmost point cloud and the bottommost point cloud in the point cloud model obtained in step S61, which is the root depth in the three-dimensional phenotypic characteristics of the plant root system;
[0039] Step S64, skeletonizing the point cloud model obtained in step S61, fitting the skeletonized lines into n continuous curves based on the B-spline curve fitting method, calculating the lengths of all curves based on the differential element method, and taking the maximum value thereof, which is the maximum root length in the three-dimensional phenotypic characteristics of the plant root system;
[0040] Step S65, for each main root in the point cloud model obtained in step S61, three cross sections are cut, the diameter of the minimum enveloping circle of the cross section is calculated, and the average value is obtained, which is the main root diameter in the three-dimensional phenotypic characteristics of the plant root system;
[0041] Step S66, performing point cloud patch processing on the point cloud model obtained in step S5, calculating the total area of all patches, and converting it into an area unit, which is the root surface area in the three-dimensional phenotypic characteristics of the plant root system;
[0042] Step S67, performing point cloud voxelization processing on the point cloud model obtained in step S5, calculating the number of all voxels, and converting them into volume units, namely, the root volume in the three-dimensional phenotypic characteristics of the plant root system.
[0043] Compared with the prior art, the present invention has at least the following positive technical effects.
[0044] (1) Most existing methods for acquiring two-dimensional images of plant roots use multiple cameras to shoot at multiple angles simultaneously or a single camera to shoot continuously. The former is more expensive and the latter has a slow imaging speed. The present invention designs an image acquisition scheme that uses a fixed-position camera to shoot a video of the process of hydroponic plant roots rotating for one week from the side of a culture container, and then obtains the RGB image of the plant root system by video frame extraction, so that the time for acquiring 150 images of the root system of a single plant is shortened to 18 seconds.
[0045] (2) Most existing computer vision-based plant root 3D reconstruction methods directly use the original root images. Since there are a large number of pixels and noise points that are not related to the root system in the original image, the accuracy of the reconstructed model is seriously affected, and the root system configuration cannot be directly observed. The present invention designs a root RGB image segmentation method. Through three processing steps, only the root part in the image is retained. This design improves the accuracy of the method for 3D reconstruction of plant roots.
[0046] (3) The existing image refraction correction method only performs a single correction for the refraction of water, ignoring the refraction caused by light passing through the container wall, resulting in a large error between the three-dimensional reconstruction result and the actual plant root configuration. The present invention designs a correction method for hydroponic plant root images. On the basis of performing distortion correction on the root image, two refraction corrections are performed respectively for the light refraction caused by the plant culture container and the water in the container, thereby reducing the error between the three-dimensional reconstruction model and the actual root configuration. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is the overall technical scheme diagram of the present invention.
[0048] Figure 2This is a flow chart of root system image segmentation of the present invention.
[0049] Figure 3 It is a schematic diagram of refraction correction of the present invention.
[0050] Figure 4 This is a flow chart of the three-dimensional reconstruction technology of the present invention.
[0051] Figure 5 These are three-dimensional point cloud model images of the root system reconstructed in three dimensions by the present invention. DETAILED DESCRIPTION
[0052] The present invention is further described below with reference to the accompanying drawings and implementation examples.
[0053] A method for 3D reconstruction and phenotypic feature extraction of hydroponic plant roots. The overall technical scheme is shown in the figure below. Figure 1 As shown, it is characterized by comprising:
[0054] Step S1, making a culture device suitable for hydroponic plant growth, transplanting plant seeds into the hydroponic culture device after germination, and placing them under suitable temperature and light conditions for growth'
[0055] Step S2, using a camera at a fixed position to shoot a video of the hydroponic plant root system rotating for one week from the side of the culture container, and then obtaining an RGB image of the plant root system by video frame extraction;
[0056] Step S3, based on ROI extraction, global threshold segmentation, canny edge detection and small connected domain segmentation methods, the root system RGB image obtained in step S2 is segmented, the image background is set to pure black, irrelevant pixels are removed, and only some pixels of the root system are retained. The image segmentation process and effect are shown as follows Figure 2 As shown;
[0057] Step S4, based on Zhang's calibration method, the segmented image is subjected to distortion correction, and based on the image correction algorithm, the image subjected to distortion correction is subjected to refraction correction. The schematic diagram of the image refraction correction algorithm is shown in FIG. Figure 3 As shown;
[0058] Step S5, based on the SFM algorithm, a sparse 3D point cloud model of the plant root system is obtained, and based on the CMVS / PMVS algorithm, the 3D point cloud model is densified. The 3D reconstruction process of the plant root system is as follows: Figure 4 As shown in Figure 2, the final three-dimensional point cloud model is as follows: Figure 5 As shown, the figure shows three different three-dimensional point cloud models of root systems;
[0059] Step S6, extracting relevant phenotypic feature data based on the three-dimensional point cloud model of the plant root system obtained in step S5.
[0060] More specifically, the plant hydroponic device in step S1 is composed as follows: (1) A cylindrical glass container is used as a culture container for the growth of plant roots; (2) A nutrient solution is placed in the culture container, and a fixable KT board is placed on the upper surface of the nutrient solution; (3) A hole is dug in the center of the KT board, and a sponge is used to clamp the plant rhizome and fix it in the hole to prevent the upper part of the plant from shaking or displacement.
[0061] More specifically, the steps for collecting the RGB image of the plant root in step S2 are as follows: (1) placing the plant hydroponic container on the optional device, and the rotating device drives the culture container to rotate, and the rotation speed of the optional device is set to 18 seconds / circle, and the rotation center axis coincides with the cylindrical center axis of the culture container; (2) using a fixed-position camera to shoot a video of the hydroponic plant root rotation process of one circle from the side of the culture container, and setting the video length to 18 seconds and the video frame rate to 25 frames / second; (3) extracting 150 plant root RGB images from the video by extracting 1 frame at intervals of 2 frames.
[0062] More specifically, the root image segmentation process in step S3 is as follows: (1) the area from the plant seed point to the bottom of the root system is selected as the ROI area, and the plant root ROI image is extracted; (2) the root ROI image is segmented based on global threshold segmentation to obtain a plant root threshold segmentation map; (3) the root threshold segmentation map is processed based on canny edge detection to extract the plant root contour and remove small connected areas to obtain a plant root segmentation map.
[0063] More specifically, the calculation process of refraction correction in step S4 is as follows: (1) Measure the distance L1 from the camera lens to the outer wall of the culture container, the distance L2 from the inner wall of the culture container to the center point of the plant rhizome, and the glass thickness t; (2) Based on Snell's law and the relationship between trigonometric functions, derive the calculation formula for the refraction error of a certain point on the image; (3) Read the coordinate value of a certain pixel point on the plant root system image, subtract the horizontal and vertical coordinate values of the center pixel point of the image, and obtain the corresponding X and Y values; (4) Substitute the X value into the formula
[0064]
[0065] Substitute the Y value into the formula
[0066]
[0067] Calculate the refraction errors ∆x and ∆y in the horizontal and vertical directions of the pixel point; (5) Use the coordinate value of the pixel point to subtract the refraction errors ∆x and ∆y respectively to obtain the coordinate value of the pixel point after refraction correction.
[0068] More specifically, the process of reconstructing the three-dimensional point cloud model of the plant root system in step S5 is as follows: (1) calculating the camera intrinsic parameter matrix by calling the relevant OpenCV function; (2) extracting feature points based on the scale-invariant feature transformation method; (3) matching feature points based on the KD tree; (4) initializing the three-dimensional model of the plant root system based on the triangulation method; (5) reading the calculated camera intrinsic parameters, restoring the three-dimensional coordinates of the feature points through the triangulation method, and reducing the error based on the bundle adjustment method to improve the three-dimensional model; (6) repeating the calculation process of (5) until all feature points of all images are restored, iteratively restoring the three-dimensional structure of the plant root system, and obtaining a sparse three-dimensional point cloud model; (7) densifying the sparse three-dimensional point cloud model obtained in (6) through hash image clustering and dense matching method based on the patch model.
[0069] More specifically, the steps of extracting plant root phenotypic characteristic data in step S6 are as follows: (1) performing faceting processing on the plant root three-dimensional point cloud model, calculating the total area of all facets, converting it into area units, and obtaining the root surface area; (2) performing voxel processing on the plant root three-dimensional point cloud model, calculating the number of all voxels, converting it into volume units, and obtaining the root volume; (3) calculating the distance from the top point cloud to the bottom point cloud in the root three-dimensional point cloud model, and obtaining the root depth; (4) processing the root three-dimensional point cloud model based on the region growing algorithm, and only retaining the main root part of the root system; (5) counting the number of main roots in the three-dimensional point cloud model that only retains the main root part of the root system, and obtaining the number of main roots; (6) The three-dimensional point cloud model of the taproot of the root system is skeletonized, and the skeletonized lines are fitted into n continuous curves based on the B-spline curve fitting method. The lengths of all curves are calculated based on the differential element method, and the maximum value among them is taken to obtain the maximum root length; (7) For each taproot in the three-dimensional point cloud model of the taproot of the root system, three cross sections are taken, and the diameter of the minimum enveloping circle of the cross section is calculated. The average value is calculated to obtain the taproot diameter.
[0070] The specific examples described in the application are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific examples described in the present invention, or replace them in a similar manner, but they will not deviate from the spirit of the present invention or exceed the scope defined by the attached claims.
Claims
1. A method for three-dimensional reconstruction and feature extraction of hydroponic plant roots, characterized in that: The steps include: Step S1, planting the plant in a cylindrical transparent glass culture container containing a nutrient solution in a hydroponic manner; Step S2, using a rotating device to carry the culture container, the rotating device drives the culture container to rotate, and the rotation center axis coincides with the cylinder center axis of the culture container; using a fixed position camera to shoot a video of the hydroponic plant root system rotating for one week from the side of the culture container, and then obtaining the plant root system RGB image in a video frame extraction manner; Step S3, segmenting the collected root system RGB image to retain only the root system part in the image; Step S4, performing distortion correction and refraction correction on the root system partial image; Step S5, based on the corrected image, use the SFM algorithm to perform three-dimensional reconstruction to obtain a point cloud model; Step S6, extracting three-dimensional phenotypic characteristic parameters of the hydroponic plant root system based on the point cloud model; The specific steps of step S4 for performing distortion correction and refraction correction on the image are as follows: Step S41, calculating the radial distortion coefficient and the tangential distortion coefficient of the camera based on Zhang's calibration method; Step S42, establishing a distortion correction mapping table based on the radial distortion coefficient and the tangential distortion coefficient of the camera, mapping the pixels of the root part obtained in step S3 to the corrected position, and completing the distortion correction; Step S43, calculate the refraction error Δy in the vertical direction of a certain point of the plant root system according to formula (1): (1) According to formula (2), the refraction error Δx at a certain point of the plant root in the horizontal direction is calculated: (2) In the formula, t is the thickness of the cylindrical glass culture container, L1 is the distance between the camera lens and the outer wall of the cylindrical glass culture container, L2 is the distance between the inner wall of the cylindrical glass culture container and the center of the plant root system, Y is the vertical distance between a certain point of the plant root system and the optical center of the camera, X is the horizontal distance between a certain point of the plant root system and the optical center of the camera, and L is the horizontal distance between the center of the plant root system and the camera lens. , , are the refractive indices of light in air, glass, and water respectively; Step S44, subtract the refraction error from the coordinate values of all the pixels on the root system image and then map them to new positions to complete the refraction correction.
2. A method for three-dimensional reconstruction and feature extraction of hydroponic plant roots according to claim 1, characterized in that: In step S1, in the culture container, a KT plate is used to fix the plant rhizome at the center of the circular surface on the upper surface of the nutrient solution.
3. The method for three-dimensional reconstruction and feature extraction of hydroponic plant roots according to claim 1, characterized in that: In step S2, the rotation speed of the rotating device is set to 18 seconds / revolution, the length of the video shot by the camera is set to 18 seconds, the video frame rate is set to 25 frames / second, the frame interval of the video frame extraction is set to 2 frames, and a total of 150 plant root RGB images are obtained in one rotation.
4. The method for three-dimensional reconstruction and feature extraction of hydroponic plant roots according to claim 1, characterized in that: In step S3, the specific steps for segmenting the root system RGB image are as follows: Step S31, performing ROI extraction on the root system RGB image, where the ROI area is the underwater part of the plant root system; Step S32, performing global threshold segmentation on the ROI area image, setting the image background to black, and copying the image color information; Step S33, converting the image into a grayscale image, extracting the outline of the plant root system based on the canny edge detection algorithm, and removing small connected domains; Step S34, mapping the image color information copied in step S32 to the image obtained in step S33.
5. The method for three-dimensional reconstruction and feature extraction of hydroponic plant roots according to claim 1, characterized in that: In step S5, the SFM algorithm is used to perform 3D reconstruction to obtain a point cloud model. The specific steps are as follows: Step S51, extracting feature points from the root system image obtained in step S4 based on a scale-invariant feature conversion method; Step S52, using a KD tree algorithm to match the feature points extracted in step S51; Step S53, estimating the camera pose based on a triangulation method to obtain a sparse point cloud model of the plant root system; Step S54, based on the dense matching method of hash image clustering and patch model, the sparse point cloud model is densified to obtain a dense point cloud model of the plant root system, and the point cloud model is output as a result.
6. The method for three-dimensional reconstruction and feature extraction of hydroponic plant roots according to claim 1, characterized in that: Step S6: Extracting plant root three-dimensional phenotypic characteristic parameters. Specific steps are as follows: Step S61, segmenting the three-dimensional point cloud model of the plant root system based on a region growing algorithm, and only retaining the taproot point cloud of the plant root system; Step S62, counting the number of taproots, that is, the number of taproots in the three-dimensional phenotypic characteristics of the plant root system; Step S63, calculating the distance between the topmost point cloud and the bottommost point cloud in the point cloud model obtained in step S61, which is the root depth in the three-dimensional phenotypic characteristics of the plant root system; Step S64, skeletonizing the point cloud model obtained in step S61, fitting the skeletonized lines into n continuous curves based on the B-spline curve fitting method, calculating the lengths of all curves based on the differential element method, and taking the maximum value thereof, which is the maximum root length in the three-dimensional phenotypic characteristics of the plant root system; Step S65, for each main root in the point cloud model obtained in step S61, three cross sections are cut, the diameter of the minimum enveloping circle of the cross section is calculated, and the average value is obtained, which is the main root diameter in the three-dimensional phenotypic characteristics of the plant root system; Step S66, performing point cloud patch processing on the point cloud model obtained in step S5, calculating the total area of all patches, and converting it into an area unit, which is the root surface area in the three-dimensional phenotypic characteristics of the plant root system; Step S67, performing point cloud voxelization processing on the point cloud model obtained in step S5, calculating the number of all voxels, and converting them into volume units, namely, the root volume in the three-dimensional phenotypic characteristics of the plant root system.
Citation Information
Patent Citations
Plant fibrous root system configuration detection and analysis system
CN111288920A
Rice plant and root system three-dimensional character nondestructive measurement device and method
CN111693551A
Test device and method used for researching root system three-dimensional architecture
CN104186239A
Plant three-dimensional reconstruction method, terminal and storage medium
CN115375842A
Binocular vision and IMU-based underwater scene three-dimensional reconstruction method, and device
WO2024045632A1
