Phenotyping system based on three-dimensional reconstruction of cotton using RGB helix camera

Abstract

The application discloses a kind of based on RGB helical camera's cotton three-dimensional reconstruction's phenotypic detection system, it is related to image processing system technical field.The system includes: rotating camera module, for multi-angle rotating shooting cotton plant image;Weighing module is uniformly distributed at the bottom of rotating camera module, for measuring the weight of cotton plant;Data transmission module is uploaded to cloud or local server by wireless network after the data handled by processor module;Processor module, main processor is used for three-dimensional reconstruction algorithm operation, phenotype parameter comprehensive analysis, weighing and image data multiple processing, while managing the task coordination of entire system and peripheral communication;Auxiliary processor is used for signal analysis and data preliminary calibration of real-time image pre-processing weighing module of helical camera module.The system can ensure the integrity of cotton image information, realize the construction of high-precision three-dimensional cotton model.

Description

Technical Field

[0001] This invention relates to the field of image processing methods, and in particular to a phenotypic detection system for three-dimensional reconstruction of cotton based on RGB spiral imaging. Background Technology

[0002] As an important economic crop, cotton's breeding, field management, and yield and quality improvement all heavily rely on accurate phenotypic data. Phenotypic parameters (such as plant height and leaf area) are not only core standards for evaluating cotton growth status but also key criteria for selecting high-quality varieties suitable for machine harvesting, early maturity, and high yield. However, traditional and existing cotton phenotypic testing technologies have long faced multiple bottlenecks, making it difficult to meet the development needs of modern intelligent agriculture and high-quality, high-yield breeding. Manual testing is inefficient and prone to significant errors: traditional cotton phenotypic testing relies primarily on manual measurement, requiring personnel to measure and record data on-site. This method not only consumes significant manpower and time, making it unsuitable for the testing needs of large-scale cotton fields or breeding bases, but is also susceptible to the influence of personnel experience, operating techniques, and subjective judgment, leading to large data errors and difficulty in providing reliable data support. Furthermore, manual measurement has stringent requirements for environmental conditions such as light, further exacerbating data instability.

[0003] With the development of 3D imaging technology, laser scanning, depth camera imaging, and other technologies are gradually being applied to crop phenotypic detection. While these technologies can achieve a certain level of accuracy in 3D reconstruction, they have significant drawbacks: First, the equipment is expensive and complex to operate, requiring highly skilled operators, making large-scale field application difficult. Second, some technologies use contact or semi-contact measurement, which can easily damage cotton leaves and young bolls during the growing season, affecting normal plant growth and subsequent long-term observation. Third, most 3D detection schemes have blind spots, insufficient coverage of the cotton canopy interior and plant base, leading to partial gaps in the reconstructed model and making it difficult to meet the accuracy requirements for detailed phenotypic parameter detection. Insufficient integration of high-throughput detection and data standardization: Modern cotton breeding requires rapid screening of massive samples, making the demand for "high-throughput" phenotypic detection increasingly urgent. However, existing technologies generally suffer from long processing times for single-plant detection and low batch processing efficiency. Furthermore, data collected from different detection devices and batches often have inconsistent formats, making direct aggregation and analysis difficult and requiring additional data preprocessing, further reducing detection efficiency.

[0004] Furthermore, existing technologies are largely limited to controlled laboratory environments. In the natural field setting, they are easily affected by factors such as terrain undulations, light variations, and weed interference, leading to a significant decrease in modeling accuracy. This makes it impossible to accurately evaluate the stress resistance and suitability for machine harvesting of new cotton varieties under natural growing conditions. Population detection and individual segmentation are challenging: In cotton population planting scenarios, leaves are closely connected and overlapping is common. Additionally, cotton fields often have ridge-furrow structures and undulating terrain caused by irrigation. Traditional detection techniques struggle to accurately segment individual plants and organ point clouds from the population, resulting in insufficient accuracy in extracting individual phenotypic parameters and failing to meet the needs of phenotypic analysis at the population scale. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a phenotypic detection system that can ensure the integrity of cotton image information and realize the construction of a high-precision three-dimensional cotton model.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a phenotypic detection system for three-dimensional reconstruction of cotton based on RGB spiral imaging, comprising: Rotating camera module: used for multi-angle rotating capture of cotton plant images; Weighing module: evenly distributed at the bottom of the rotating camera module, used to measure the weight of cotton plants; The data transmission module uploads the data processed by the processor module to the cloud or local server via a wireless network. Processor module: Includes main processor and auxiliary processor. The main processor is used for 3D reconstruction algorithm calculation, comprehensive analysis of phenotypic parameters, and multiple processing of weighing and image data. It also manages the task coordination and peripheral communication of the entire system. The auxiliary processor is used for real-time image preprocessing of the spiral camera module, signal analysis and preliminary data calibration of the weighing module, and motion control command output of the rotating camera module.

[0007] A further technical solution is as follows: the rotating camera module includes a housing with an openable side panel. A rotating gimbal is installed inside the housing, and a weighing module is located at the bottom of the rotating gimbal. A rotating track is fixed on the upper side of the rotating gimbal. The path of the camera's spiral imaging is fixed via the preset rotating track. The camera is fixed on the rotating track and is used to collect cotton image information. A stepper motor is controlled by the processor module and is used to drive the camera to move under the control of the processor module, enabling the camera to rotate horizontally and vertically. A fixed light source is located inside the housing for supplementary lighting to reduce the impact of the environment on the 3D reconstruction process. A human-computer interaction module is bidirectionally connected to the processor module for inputting control commands and displaying the output data.

[0008] A further technical solution is that the three-dimensional reconstruction algorithm includes: Image preprocessing is used to denoise and align the acquired spiral sequence images; Helical trajectory position estimation, based on the calculation of camera attitude using helical constraints; Sparse point cloud reconstruction: 3D point recovery based on spiral multi-view constraints; Dense point cloud optimization: Spiral multi-view matching based on image color; Phenotypic parameter extraction: Based on the optimized 3D mesh model, the core phenotypic parameters of cotton are extracted, and 3D reconstruction of cotton is performed.

[0009] The beneficial effects of adopting the above technical solution are as follows: The system uses a controllable rotating gimbal to drive a camera to acquire cotton images from multiple angles. Combined with image processing and 3D modeling technology, it achieves automated and accurate monitoring of phenotypic parameters such as plant height and leaf area, providing data support for cotton phenotypic research. By adopting a spiral camera mode, the camera moves around the cotton plant in a spiral trajectory, covering all parts of the plant from multiple angles and dimensions, effectively eliminating blind spots and ensuring the integrity of image information. Combined with the fusion processing of RGB images and depth information and filtering algorithms, the accuracy of feature point matching is greatly improved, background noise and outlier interference are reduced, and a high-precision 3D model is constructed, laying the foundation for more accurate phenotypic parameter extraction. Attached Figure Description

[0010] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0011] Figure 1 This is a schematic block diagram of the system described in the embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the rotating camera module in the system described in the embodiment of the present invention; Figure 3 This is a flowchart of phenotypic processing in the system described in the embodiments of the present invention; The components include: 1. Box body; 2. Rotating gimbal; 3. Weighing module; 4. Rotating track; 5. Camera; 6. Stepper motor; 7. Fixed light source; 8. Human-computer interaction module. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0013] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0014] Overall, such as Figure 1 As shown in the figure, this invention discloses a phenotypic detection system for three-dimensional reconstruction of cotton based on RGB spiral imaging. The phenotypic detection system includes: Rotating camera module: used for multi-angle rotating capture of cotton plant images; Weighing module: evenly distributed at the bottom of the rotating camera module, used to measure the weight of cotton plants; The data transmission module uploads the data processed by the processor module to the cloud or local server via a wireless network. Processor module: Includes main processor and auxiliary processor. The main processor is used for 3D reconstruction algorithm calculation, comprehensive analysis of phenotypic parameters, and multiple processing of weighing and image data. It also manages the task coordination and peripheral communication of the entire system. The auxiliary processor is used for real-time image preprocessing of the spiral camera module, signal analysis and preliminary data calibration of the weighing module, and motion control command output of the rotating camera module.

[0015] like Figure 2 As shown, the rotating camera module includes a housing 1 with an openable side panel. A rotating gimbal 2 is housed inside the housing 1. A weighing module 3 is located at the bottom of the rotating gimbal 2. A rotating track 4 is fixed to the upper side of the rotating gimbal 2, and the path of the spiral camera 5 is fixed via the preset rotating track 4. The camera 5 is fixed on the rotating track and is used to collect cotton image information. A stepper motor 6, controlled by the processor module, drives the camera to rotate horizontally and vertically. A fixed light source 7 is located inside the housing for supplementary lighting, reducing the impact of the environment on the 3D reconstruction process. A human-computer interaction module 8 is bidirectionally connected to the processor module for inputting control commands and displaying output data. The camera is driven by a stepper motor with an accuracy of ±0.1°, supporting 360° horizontal rotation and 180° vertical rotation.

[0016] Furthermore, the side of the housing 1 can be opened, and the rotating gimbal 2 can be pulled out via the push-pull base to place the cotton plant on the rotating gimbal 2. The rotating gimbal 2 is used to control the entire cotton plant to rotate in the opposite direction to the camera's rotation direction, improving the accuracy of shooting and 3D reconstruction. Four weighing modules 3 at the bottom of the rotating gimbal 2 use high-precision strain gauge pressure sensors to measure the weight of the cotton plant. The rotating track 4 fixes the path of the camera's spiral imaging through a preset rotating track, enhancing the accuracy of image imaging. A high-precision RGB camera 5 is cheaper and easier to use than 3D acquisition data such as LiDAR and spectroscopy. A stepper motor 6 drives the camera; the stepper motor's accuracy can reach ±0.1°, allowing the camera 5 to rotate 360° horizontally and 180° vertically. A fixed light source 7 reduces the impact of the environment on the 3D reconstruction process, making the imaging more accurate. The human-computer interaction module 8 allows operation of the camera's rotation rate and shooting frequency, as well as subsequent 3D reconstruction and data analysis.

[0017] Data transmission module: Uploads processed data to the cloud or local server via wireless network; Weighing module: Employs four high-precision strain gauge pressure sensors, evenly distributed at the bottom of the rotating gimbal. The sensors achieve an accuracy of ±0.01%FS and feature an anti-electromagnetic interference coating. This coating prevents electromagnetic interference from the RGB camera module from affecting the sensor signals. The four sensors address single-point weighing eccentricity errors, improving weighing accuracy and adapting to the rotating environment of the helical camera.

[0018] Processor Module: The processor module includes a main processor and a coprocessor. The main processor is the core of the system's computing power, responsible for the calculation of the core algorithm for 3D reconstruction, comprehensive analysis of phenotypic parameters, and multiple processing of weighing and image data. It also manages the task coordination and peripheral communication of the entire system. The coprocessor is responsible for the real-time image preprocessing (such as noise reduction) of the RGB spiral camera module, the signal analysis and preliminary data calibration of the weighing module, and the output of motion control commands for the rotating gimbal, camera shooting angle, and other equipment.

[0019] The processor module includes an image processing module and an intelligent control module. The image processing module performs noise reduction, stitching, and 3D reconstruction on the acquired images, extracting phenotypic parameters such as cotton plant height and leaf area index. It also uses deep learning algorithms to identify and segment cotton organs, generating a 3D point cloud model by combining images taken from various perspectives to obtain phenotypic parameters. The intelligent control module maintains a certain background light source and automatically controls the camera's shooting frequency and rotation rate according to a preset time.

[0020] Furthermore, such as Figure 3 As shown, the 3D reconstruction algorithm includes: 1) Image preprocessing: To address potential issues such as motion blur, uneven lighting, and noise interference during helical imaging, a multi-step preprocessing workflow is designed to provide high-quality image data for subsequent reconstruction. Adaptive noise suppression: An improved bilateral filtering algorithm is employed, dynamically adjusting the core filtering parameters based on cotton leaf texture features. (1) Spatial domain weights: ; (2) Gray-scale weights: ; (3) Among them These are the row and column coordinates of the pixels to be processed. These are the row and column coordinates of the neighboring pixels of the pixel to be processed. Represents the grayscale value of the image. (Spatial standard deviation) is adaptively adjusted based on the edge density of cotton leaves in the image (when the edge density is high). Use 1.5-2.0, or 2.0-2.5 when it's low. The grayscale standard deviation is fixed at 20 to balance noise reduction and detail preservation.

[0021] Light equalization: Using CLAHE (Adaptive Histogram Equalization with Limiting Contrast), the image is divided into 8x8 pixel local blocks, and the contrast gain is limited to 2.0, which can effectively avoid overexposure in the shadow areas of the leaves or loss of detail in the highlight areas.

[0022] Spiral frame alignment correction: Based on SIFT feature matching, the homography matrix H between adjacent frames is calculated, and images with slight displacement are further aligned to better ensure the spatial continuity of the spiral sequence. (1) Extract SIFT feature points from adjacent frames and use FLANN matcher for the first step of matching.

[0023] (2) Remove outliers based on RANSAC algorithm (iteration number = 1000, inlier value = 2.0 pixels).

[0024] (3) Solve for the homography matrix The perspective transformation and alignment are performed on the next frame image.

[0025] 2) Spiral trajectory position estimation: Camera attitude calculation based on helical constraints: The trajectory of a helical camera has the inherent characteristics of "uniform upward movement along the axis + uniform rotation around the circumference". This constraint is used to optimize the accuracy of camera position estimation and solve the drift problem of traditional SfM (structure of motion recovery) in long sequences. Initial position and attitude estimation: Preliminary calculations were performed using the improved ORB-SLAM2 algorithm. (1) Feature extraction: ORB features (number of key points = 2000, number of pyramid layers = 8) are used to improve real-time performance and robustness. (2) Tracking thread: Combined with the continuity of the spiral trajectory, "pose constraint of the first two frames" is added to reduce tracking loss caused by occlusion.

[0026] (3) Local map optimization: Minimize reprojection error through bundle adjustment (BA): ; in Indicates the first The actual pixel coordinates of a 3D point on a 3D image plane These are camera-intra-camera parameters. For the camera pose, For three-dimensional points, This is the projection function.

[0027] Spiral trajectory constraint optimization: (1) Define the mathematical model of the spiral trajectory: Let the spiral axis be the world coordinate system. The coordinates of the camera center on the axis satisfy: ; ; ; in Where is the helix radius. Angular velocity, For axial velocity, For initial phase and height (2) Based on the initial position and attitude, the results are estimated, and the rotation parameters are optimized using the Levenberg-Marquardt algorithm. This minimizes the error between the actual position of the camera center and the spiral model. (3) Use the optimized spiral model to calibrate the position and attitude of the camera and eliminate the drift error caused by the accumulation of long sequences. 3) Sparse point cloud reconstruction: 3D point recovery based on helical multi-view constraints: By leveraging the full-view coverage of spiral trajectories, the integrity and accuracy of sparse point clouds are improved: Feature matching and triangulation: (1) Cross-frame feature matching: ORB feature matching is performed on key frames in the spiral sequence at 15° intervals (circular direction) to ensure the feature correspondence under different viewpoints. (2) Triangulation solution: For the successfully matched feature point combination, the three-dimensional coordinates are calculated using the camera pose through the triangulation formula: Let the image coordinates of two points be on camera 1 and camera 2. , Corresponding camera matrix and Then three-dimensional points satisfy: ; in , The calculation method is similar; it is obtained by solving the system of linear equations. To, among them The intrinsic parameter matrix describes the camera's own optical characteristics. This is for Rotation matrix (describes the camera's pose). for Translation vector (describes the camera's position).

[0028] Point Cloud Selection and Optimization: (1) Remove invalid points: Remove points with reprojection error greater than 3.0 pixels and depth values ​​exceeding the reasonable range (0.5). 3.0m (set according to cotton planting density) three-dimensional point (2) Global BA optimization: Using all key poses and 3D points as optimization variables, a global reprojection error function is constructed. The overall accuracy of the point cloud is improved through iterative optimization using the BA algorithm. 4) Dense point cloud optimization: Color-based spiral multi-view matching: To accurately reproduce the details on the surface of cotton, a dense matching algorithm based on color consistency is designed: Image pyramid construction: A four-layer Gaussian pyramid is constructed on the preprocessed RGB image. Dense matching is performed progressively from the top layer (low resolution) to the bottom layer (high resolution) to improve matching efficiency and accuracy. Stereoscopic matching from a spiral perspective: (1) Cost calculation for matching: A hybrid cost function of color difference + gradient difference is used: ; in The pixel coordinates of the reference frame For parallax, and Images for reference frames and matching frames, For image gradient, (Balancing the information between color and structure); Generation and denoising of dense point clouds: (1) Transformation of disparity map into 3D point cloud: Using the camera's intrinsic parameters and pose, the optimized disparity map is transformed into a dense point cloud. (2) Statistical filtering for noise reduction: Calculate the mean distance between each point and its 50 nearest neighbors, and remove points whose mean distance is greater than twice the standard deviation. (3) Reconstruction of Poisson surface: Fit Poisson surface to dense point cloud to generate a smoother three-dimensional mesh model of cotton, while preserving detailed structures such as leaves.

[0029] 5) Phenotypic parameter extraction: The automated calculation of cotton phenotypic characteristics is based on an optimized three-dimensional mesh model, extracting the core phenotypic parameters of cotton to meet the needs of breeding and cultivation. Plant height detection: (1) Coordinate normalization: Set the lowest point (root system) of the 3D model as Origin of the axis. (2) Extraction of extreme values: obtained through the parameters of the mesh model. The maximum value of the axis coordinates is the cotton plant height. ; Leaf area measurement: (1) Leaf segmentation, based on color characteristics (leaves are green, Threshold: and texture features (leaf edge complexity) ), dividing the blade area, (2) Area calculation: For the mesh model of each blade, calculate its surface area (sum of the areas of the triangular patches), and the total blade area. .

[0030] 6) Phenotypic results output: Output the obtained results such as plant height and leaf area.

[0031] This invention is based on an RGB camera-based system. Compared to high-end 3D inspection equipment such as laser scanning, it significantly reduces hardware costs and its automated operation process requires no professional skills, offering advantages such as low cost, ease of operation, and suitability for large-scale promotion. The spiral camera mode covers the plant from multiple angles and dimensions, eliminating blind spots in the canopy and plant bottom caused by fixed perspectives or simple surround shooting in conventional 3D inspection. Therefore, it has the advantages of high model integrity and no local defects. The non-contact acquisition design avoids damage to the plant, and the multi-view image fusion processing reduces environmental interference such as field lighting and terrain. It can be adapted to multiple scenarios such as laboratories and fields without additional auxiliary equipment, and has the advantages of non-destructive testing, strong adaptability to multiple scenarios, and high detection stability.

[0032] It should be noted that although the above embodiments have been described in this document, those skilled in the art can make other changes and modifications to these embodiments once they understand the basic inventive concept. Therefore, the above description is only an embodiment of the present invention and does not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A phenotypic detection system for three-dimensional reconstruction of cotton based on RGB spiral imaging, characterized in that... include: Rotating camera module: used for multi-angle rotating capture of cotton plant images; Weighing module: evenly distributed at the bottom of the rotating camera module, used to measure the weight of cotton plants; The data transmission module uploads the data processed by the processor module to the cloud or local server via a wireless network. Processor module: Includes main processor and auxiliary processor. The main processor is used for 3D reconstruction algorithm calculation, comprehensive analysis of phenotypic parameters, and multiple processing of weighing and image data. It also manages the task coordination and peripheral communication of the entire system. The auxiliary processor is used for real-time image preprocessing of the spiral camera module, signal analysis and preliminary data calibration of the weighing module, and motion control command output of the rotating camera module.

2. The phenotypic detection system for three-dimensional reconstruction of cotton based on RGB spiral imaging as described in claim 1, characterized in that: The rotating camera module includes a housing (1), the side panel of which can be opened, a rotating gimbal (2) is provided inside the housing (1), the weighing module (3) is located at the bottom of the rotating gimbal (2), and a rotating track (4) is fixed on the upper side of the rotating gimbal (2). The path of the camera (5) for spiral imaging is fixed by the preset rotating track (4). The camera (5) is fixed on the rotating track and is used to collect cotton plant image information. The stepper motor (6) is controlled by the processor module and is used to drive the camera to move under the control of the processor module, so that the camera (5) can rotate horizontally and vertically. The fixed light source (7) is located inside the housing and is used to supplement light and reduce the influence of the environment on the three-dimensional reconstruction process. The human-computer interaction module (8) is bidirectionally connected to the processor module and is used to input control commands and display the output data.

3. The phenotypic detection system for cotton three-dimensional reconstruction based on RGB spiral imaging as described in claim 2, characterized in that: The weighing module (3) includes four pressure sensors, which are evenly distributed at the bottom of the rotating platform.

4. The phenotypic detection system for three-dimensional reconstruction of cotton based on RGB spiral imaging as described in claim 1, characterized in that, The 3D reconstruction algorithm includes: Image preprocessing is used to denoise and align the acquired spiral sequence images; Helical trajectory position estimation, based on the calculation of camera attitude using helical constraints; Sparse point cloud reconstruction: 3D point recovery based on spiral multi-view constraints; Dense point cloud optimization: Spiral multi-view matching based on image color; Phenotypic parameter extraction: Based on the optimized 3D mesh model, the core phenotypic parameters of cotton are extracted, and 3D reconstruction of cotton is performed.

5. The phenotypic detection system for cotton three-dimensional reconstruction based on RGB spiral imaging as described in claim 4, characterized in that, The specific implementation method of image preprocessing includes the following steps: Adaptive noise suppression: An improved bilateral filtering algorithm is used, and the core filtering parameters are dynamically adjusted based on the texture features of cotton leaves. (1) Spatial domain weights: ; (2) Gray-scale weights: ; (3) Among them These are the row and column coordinates of the pixels to be processed. These are the row and column coordinates of the neighboring pixels of the pixel to be processed. Represents the grayscale value of an image, spatial standard deviation Adaptively adjust the grayscale standard deviation based on the density of cotton leaf edges in the image. Set to 20 to balance noise reduction and detail preservation; Illumination equalization: An adaptive histogram equalization algorithm with limited contrast is used to divide the image into 8x8 pixel local blocks and limit the contrast gain to 2.

0. Spiral frame alignment correction: Based on SIFT feature matching, the homography matrix H between adjacent frames is calculated, and images with slight displacement are further aligned. (1) Extract SIFT feature points from adjacent frames and use the FLANN matcher for the first step of matching; (2) Remove outliers based on the RANSAC algorithm; (3) Solve for the homography matrix The perspective transformation and alignment are performed on the next frame image.

6. The phenotypic detection system for cotton three-dimensional reconstruction based on RGB spiral imaging as described in claim 4, characterized in that, The phenotypic parameter extraction method includes the following steps: The automated calculation of cotton phenotypic characteristics is based on an optimized three-dimensional mesh model, extracting the core phenotypic parameters of cotton to meet the needs of breeding and cultivation. Plant height detection: (1) Coordinate normalization: Set the lowest point of the 3D model as Axis origin; (2) Extraction of extreme values: obtained through the parameters of the mesh model. The maximum value of the axis coordinates is the cotton plant height. ; Leaf area measurement: (1) Leaf segmentation: Based on color features, segment the leaf regions; (2) Area calculation: For the mesh model of each blade, calculate its surface area and the total blade area. .

Images