A corn canopy vertex height measurement system and method based on a sprinkler-mounted downward-looking depth camera
By installing a 45° oblique angle depth camera and IMU on the sprinkler irrigation machine, and combining deep learning models and spatial triangulation correction, the problems of accuracy and environmental adaptability in maize plant height monitoring were solved, and efficient and low-cost maize canopy apex height measurement was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWEST A & F UNIV
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-07
AI Technical Summary
In precision corn planting management, existing technologies struggle to acquire plant height data accurately and at high frequency in field environments. Traditional methods are inefficient and have poor measurement accuracy, and vertical-view camera height measurement suffers from limited monitoring range and inability to acquire lateral structural information.
Using a 45° oblique perspective method, the vertical plant height of maize is obtained by accurately locating the apex of the maize canopy through spatial coordinate system rotation and depth gradient processing, combined with depth camera and IMU attitude data. The maize leaf core region is identified by a deep learning object detection model and corrected by a spatial triangular geometric model.
It significantly improves measurement accuracy and data reliability and usability, enables real-time monitoring with low power consumption, can offset the effects of mechanical vibration, adapts to unstructured farmland environments, and meets the needs of mobile sprinkler irrigation operations.
Smart Images
Figure CN122347673A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of smart agriculture and computer vision monitoring technology, specifically relating to a corn canopy apex height measurement system and method based on a sprinkler irrigation airborne down-view depth camera. Background Technology
[0002] In precision maize planting management, continuous monitoring of plant height is crucial for assessing crop growth and guiding variable irrigation. However, obtaining plant height data frequently, accurately, and at low cost in field environments still presents numerous technical challenges. Traditional manual sampling is inefficient, and ultrasonic sensors are prone to misjudgment and have poor measurement accuracy. Existing mainstream algorithms for depth camera height measurement are mostly based on a vertical perspective, which easily leads to a "crowning effect." Furthermore, with vertical installation, the camera's field of view is limited, and it cannot acquire information about the crop's lateral structure. Therefore, developing an intelligent monitoring system that can expand the monitoring range through an oblique perspective and accurately capture the apex of maize plant height through spatial geometric compensation is of great significance. Summary of the Invention
[0003] To address the issues of limited monitoring range and low measurement accuracy caused by the "regional averaging algorithm" in existing vertical viewing methods, a method based on a 45° oblique viewing angle is proposed. This method, through spatial coordinate system rotation and depth gradient processing, accurately locates the physical vertices of the maize canopy on resource-constrained terminals.
[0004] To achieve the above objectives, the present invention provides the following solution: A corn canopy apex height measurement system based on a sprinkler irrigation vehicle-mounted down-view depth camera includes: The data acquisition module is used to install a depth camera under the sprinkler truss to acquire RGB and depth images of the field from an oblique perspective, and to obtain the IMU attitude data of the depth camera. The ROI localization module is used to identify and locate the corn leaf core region in the RGB image based on a deep learning object detection model, and map the corn leaf core region to the corresponding depth image to obtain the depth map ROI region; The distance calculation module is used to perform depth extraction based on the ROI region of the depth map to obtain the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core; The distance correction module is used to parse the IMU attitude data and construct a spatial triangular geometric model to perform vertical projection correction on the oblique straight-line distance to obtain the vertical plant height of the corn.
[0005] Preferably, the data acquisition module includes: The sensor angle adjustment unit is used to adjust the direction of the optical axis of the depth camera so that the optical axis forms a 45° angle with the gravity plumb line and points obliquely downwards toward the crop row; The data acquisition unit is used to continuously acquire RGB images and depth maps of the field from a 45° oblique perspective using a depth camera, and simultaneously record Euler angle attitude data output by the inertial measurement unit (IMU) built into the depth camera. The data synchronization unit is used to ensure that each frame of RGB image, depth image and Euler angle pose data corresponds in time and space within the edge computing terminal through a unified timestamp reference.
[0006] Preferably, the ROI positioning module includes: The leaf core feature recognition unit is used to perform leaf core feature recognition on the RGB image using a deep learning object detection model to obtain the leaf core region bounding box. The depth mapping unit is used to map the ROI region in the corresponding depth image based on the geometric center coordinates of the leaf core region bounding box.
[0007] Preferably, the distance calculation module includes: The neighborhood extraction unit is used to extract a 3×3 pixel neighborhood of the leaf core center within the ROI region of the depth map. The depth calculation unit is used to perform statistical filtering on the depth pixel values in the 3×3 pixel neighborhood of the leaf core center, remove outliers, and calculate the average depth value. The average depth value represents the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core.
[0008] Preferably, the distance correction module includes: The attitude data parsing unit is used to parse the IMU attitude data to obtain the current pitch and roll angles of the depth camera. The actual total tilt angle calculation unit is used to obtain the actual total tilt angle of the camera optical axis relative to the gravity plumb line based on the algebraic sum of the 45° preset angle and the pitch angle. The plant height calculation unit is used to construct a spatial triangular geometric model based on the roll angle, the actual total tilt angle, and the installation height of the depth camera, and to perform vertical projection correction on the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core to obtain the vertical plant height of the corn.
[0009] This invention also provides a method for measuring the height of the corn canopy apex based on a sprinkler irrigation vehicle-mounted down-view depth camera. The system includes: A depth camera is installed under the sprinkler truss to capture RGB and depth images of the field from an oblique perspective, and to obtain the IMU attitude data of the depth camera. Based on a deep learning object detection model, the corn leaf core region in the RGB image is identified and located, and the corn leaf core region is mapped to the corresponding depth image to obtain the depth map ROI region; Based on the ROI region of the depth map, depth extraction is performed to obtain the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core; The IMU attitude data is analyzed, and a spatial triangular geometric model is constructed to perform vertical projection correction on the oblique straight-line distance to obtain the vertical plant height of the corn.
[0010] Preferably, the data acquisition methods include: Adjust the optical axis of the depth camera so that it forms a 45° angle with the vertical line of gravity and points obliquely downwards toward the crop row; Using a depth camera, RGB images and depth maps of the field are continuously acquired from a 45° oblique perspective, and Euler angle attitude data output by the inertial measurement unit (IMU) built into the depth camera are recorded simultaneously. Within the edge computing terminal, a unified timestamp benchmark ensures that each frame of RGB image, depth image, and Euler angle pose data corresponds spatiotemporally.
[0011] Preferred methods for obtaining the ROI region of a depth map include: A deep learning object detection model is used to identify leaf core features in the RGB image to obtain the bounding box of the leaf core region. Based on the geometric center coordinates of the bounding box of the leaf core region, the ROI region is mapped in the corresponding depth image.
[0012] Compared with existing technologies, the beneficial effects of this invention are as follows: By abandoning the traditional regional mean method, this invention innovatively utilizes the edge gradient features of depth maps to accurately locate the canopy apex (tassel or leaf tip), thus overcoming the "cropping effect" commonly found in visual measurements and significantly improving measurement accuracy and data usability. Furthermore, thanks to its embedded edge computing architecture design, the system can achieve low-power localized data processing without relying on unstable external networks or expensive backend servers, meeting the real-time monitoring needs of mobile sprinkler operations. In addition, by integrating an IMU-based dynamic attitude compensation mechanism, this invention can effectively offset the mechanical vibrations generated when large sprinkler machines travel on unstructured farmland surfaces, ensuring strong environmental adaptability. Attached Figure Description
[0013] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a system architecture diagram of an embodiment of the present invention; Figure 2This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0017] Example 1: like Figure 1 As shown, a corn canopy apex height measurement system based on a sprinkler irrigation airborne down-view depth camera includes: a data acquisition module, an ROI positioning module, a distance calculation module, and a distance correction module.
[0018] The data acquisition module is used to install a depth camera (such as ZED 2i) under the sprinkler truss via a customized angle-adjustable bracket to acquire RGB and depth images of the field from an oblique perspective, and to obtain the IMU attitude data of the depth camera.
[0019] A further implementation method includes a data acquisition module comprising: The sensor angle adjustment unit is used to adjust the direction of the optical axis of the depth camera so that the optical axis forms a 45° angle with the vertical line of gravity and points obliquely downwards towards the crop row; to ensure that the viewing angle can penetrate the obstruction of the upper leaves and directly see the internal structure of the corn plant heart; a low-power embedded edge computing terminal is used as the core computing power unit, which is powered by the sprinkler irrigation machine.
[0020] The data acquisition unit is used to continuously acquire RGB images and depth maps of the field from a 45° oblique perspective using a depth camera, and simultaneously record Euler angle attitude data output by the inertial measurement unit (IMU) built into the depth camera. The data synchronization unit is used to ensure that each frame of RGB image, depth image and Euler angle pose data corresponds in time and space within the edge computing terminal through a unified timestamp reference.
[0021] The ROI localization module is used to identify and locate the corn leaf core region in an RGB image based on a deep learning object detection model, and map the corn leaf core region to the corresponding depth image to obtain the depth map ROI region.
[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the following will be used in conjunction with... Figure 2The present invention will be further described below.
[0023] A further implementation method includes an ROI positioning module comprising: The leaf core feature recognition unit is used to perform leaf core (i.e., the spiral divergence center of the leaf) feature recognition on RGB images by a lightweight deep learning object detection model (such as YOLOv8-Nano) accelerated by TensorRT edge inference, and obtain the leaf core region bounding box; compared with the canopy outline from the vertical view, the leaf core features from the oblique view are more stable and less affected by wind swaying; the model generates a bounding box that is close to the leaf core region in the image.
[0024] The depth mapping unit is used to map the ROI region in the corresponding depth image based on the geometric center coordinates of the leaf core region bounding box.
[0025] The distance calculation module is used to extract depth based on the ROI region of the depth map and obtain the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core.
[0026] A further implementation method includes a distance calculation module comprising: The neighborhood extraction unit is used to extract a 3×3 pixel neighborhood of the leaf core center within the ROI region of the depth map. Specifically, within the mapped ROI region of the depth map, considering the noise that may exist in the field environment (such as flying insects or reflections), instead of directly taking the depth of a single point, a 3×3 pixel neighborhood of the leaf core center is extracted.
[0027] The depth calculation unit performs statistical filtering on the depth pixel values within a 3×3 pixel neighborhood of the leaf core center, removes outliers, and then calculates the average depth value Z. measure Average depth value Z measure This represents the oblique straight-line distance from the camera's optical center to the corn leaf core growth point. Compared to traditional algorithms that find the highest point, directly locking onto the "leaf core" effectively avoids inaccurate or inaccurate measurements caused by excessive growth or drooping of the top leaves, thus locking onto the most accurate biomass growth node of the crop.
[0028] The distance correction module is used to parse IMU attitude data and construct a spatial triangular geometric model to perform vertical projection correction on the oblique straight-line distance to obtain the vertical plant height of corn.
[0029] A further embodiment of the method is that the distance correction module includes: The attitude data parsing unit is used to parse IMU attitude data to obtain the current pitch angle (Pitch, α) and roll angle (Roll, β) of the depth camera. The actual total tilt angle calculation unit is used to obtain the actual total tilt angle Φ of the camera optical axis relative to the gravity plumb line based on the algebraic sum of the 45° preset angle and the pitch angle (α). The plant height calculation unit is used to construct a spatial triangular geometric model based on the roll angle, actual total tilt angle, and depth camera installation height. It then performs vertical projection correction on the oblique straight-line distance from the camera's optical center to the corn leaf core growth point to obtain the vertical plant height of the corn. Specifically, the calculation process is as follows: H=H cam - Z measure ·cos(45°+α)·cos(β), Where: H is the calculated vertical plant height of maize; Z measure H represents the oblique straight-line distance measured by the camera; α represents the real-time pitch angle deviation of the sprinkler's forward direction measured by the IMU; β represents the lateral roll angle deviation of the sprinkler measured by the IMU. cam Determine the camera mounting height.
[0030] The corrected plant height data, along with the operation timestamp and geographic coordinates, are encapsulated and sent to the sprinkler control system in real time via a communication interface.
[0031] The system of the present invention is compared with the traditional method, and the comparison results are shown in Table 1 and Table 2.
[0032] Example 2: like Figure 2 As shown, the present invention also provides a method for measuring the height of the corn canopy apex based on a sprinkler irrigation vehicle-mounted down-view depth camera, using the system of Embodiment 1, comprising: A depth camera is installed below the sprinkler truss to acquire RGB and depth images of the field from an oblique perspective, and to obtain the IMU attitude data of the depth camera. A further implementation method includes: adjusting the optical axis of the depth camera so that the optical axis forms a 45° angle with the vertical line of gravity and points obliquely downwards towards the crop row; using the depth camera, continuously acquiring RGB images and depth maps of the field from a 45° oblique perspective, and simultaneously recording the Euler angle attitude data output by the inertial measurement unit (IMU) built into the depth camera; within the edge computing terminal, using a unified timestamp reference, ensuring that each frame of RGB image, depth image, and Euler angle attitude data corresponds spatiotemporally.
[0033] Specifically, the system is embedded and deployed at a 45° angle. A ZED 2i binocular camera is mounted on the sprinkler truss using a shock-absorbing bracket with a dial, precisely adjusting the camera's pitch angle so that its optical axis forms a 45° angle with the plumb line. The Jetson Orin Nano development board is housed in an IP65-rated waterproof box and powered from the sprinkler's main cable via a DC-DC wide-voltage module. The camera connects to the development board via a USB 3.0 interface, enabling high-speed transmission of large amounts of data. The system is configured for automatic startup, automatically loading environmental parameters.
[0034] Based on a deep learning object detection model, the corn leaf core region in an RGB image is identified and located, and the corn leaf core region is mapped to the corresponding depth image to obtain the depth map ROI region; a further implementation method is that the method for obtaining the depth map ROI region includes: using a deep learning object detection model to perform leaf core feature recognition on the RGB image to obtain the leaf core region bounding box; based on the geometric center coordinates of the leaf core region bounding box, the ROI region is mapped in the corresponding depth image.
[0035] Specifically, when the sprinkler system starts running, the development board calls the ZED SDK to obtain the left-eye RGB image and depth map. Using a pre-trained YOLOv8-Nano engine quantized to FP16 half-precision, it quickly detects "corn leaf cores (Whorl)" in the RGB image. The model training set contains a large number of corn seedling to jointing stage samples from a 45° overhead view, ensuring the ability to distinguish between leaf edges and leaf core centers. The system extracts the corresponding depth data block based on the center (u, v) of the detection box and, based on the 45° slope distance characteristic, removes invalid detection boxes with excessively large depth values (falling into the background soil).
[0036] Based on the ROI region of the depth map, depth extraction is performed to obtain the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core. In step S3, in order to obtain the most accurate growth point height, the system no longer traverses the contour, but directly extracts the depth value Z from the center of the leaf core detection box. measure To eliminate edge effects, a median filtering algorithm is used to process the depth pixels in the central 3×3 region: Z measure = Median(D u-1 , v-1 , . . .,D u+1 , v+1 ) This step directly obtains the oblique Euclidean distance from the camera to the corn growing point, effectively avoiding measurement noise caused by leaf tip oscillation in traditional algorithms. u-1 , v-1 This represents the depth observation value of the top-left neighboring pixel in the depth image, with the center coordinates (u, v) of the leaf core detection box as the origin.
[0037] IMU attitude data was analyzed, and a spatial triangular geometric model was constructed to perform vertical projection correction on the oblique straight-line distance to obtain the vertical plant height of the corn. Considering the slight shift in the camera's optical axis caused by vibration during sprinkler operation, an IMU-based coordinate system transformation model was established. Let the camera installation height H be... cam =300cm, the real-time pitch angle deviation output by the IMU is α, and the roll angle is β. Corrected corn plant height H plant The calculation model is as follows: H plant =H cam - Z measure ·cos(45°+α)·cos(β) Among them, Z measure • cos(45°+α) projects the slope distance onto the vertical plane, and cos(β) further corrects for the cosine error caused by lateral sway. The final output is the true plant height value to guide variable irrigation.
[0038] In summary, this invention combines an embedded edge computing terminal with a large sprinkler irrigation platform and replaces the traditional vertical layout with a depth camera installed at a 45° angle on its truss. It uses a deep learning model to accurately identify the "corn leaf core (growing point)" features exposed from a 45° angled view, directly obtains the oblique distance of the leaf core center, and performs dynamic triangulation compensation by combining the attitude parameters calculated in real time by the IMU built into the vision sensor. The oblique observation value of the leaf core is then projected in real time to restore the true plant height in the vertical direction of gravity. In principle, this avoids the obstruction of the top leaves and wind interference, and achieves accurate and vibration-resistant measurement of the core growth height of the crop.
[0039] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A corn canopy apex height measurement system based on a sprinkler irrigation vehicle-mounted down-view depth camera, characterized in that, include: The data acquisition module is used to install a depth camera under the sprinkler truss to acquire RGB and depth images of the field from an oblique perspective, and to obtain the IMU attitude data of the depth camera. The ROI localization module is used to identify and locate the corn leaf core region in the RGB image based on a deep learning object detection model, and map the corn leaf core region to the corresponding depth image to obtain the depth map ROI region; The distance calculation module is used to perform depth extraction based on the ROI region of the depth map to obtain the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core; The distance correction module is used to parse the IMU attitude data and construct a spatial triangular geometric model to perform vertical projection correction on the oblique straight-line distance to obtain the vertical plant height of the corn.
2. The system according to claim 1, characterized in that, The data acquisition module includes: The sensor angle adjustment unit is used to adjust the direction of the optical axis of the depth camera so that the optical axis forms a 45° angle with the gravity plumb line and points obliquely downwards toward the crop row; The data acquisition unit is used to continuously acquire RGB images and depth maps of the field from a 45° oblique perspective using a depth camera, and simultaneously record Euler angle attitude data output by the inertial measurement unit (IMU) built into the depth camera. The data synchronization unit is used to ensure that each frame of RGB image, depth image and Euler angle pose data corresponds in time and space within the edge computing terminal through a unified timestamp reference.
3. The system according to claim 1, characterized in that, The ROI location module includes: The leaf core feature recognition unit is used to perform leaf core feature recognition on the RGB image using a deep learning object detection model to obtain the leaf core region bounding box. The depth mapping unit is used to map the ROI region in the corresponding depth image based on the geometric center coordinates of the leaf core region bounding box.
4. The system according to claim 1, characterized in that, The distance calculation module includes: The neighborhood extraction unit is used to extract a 3×3 pixel neighborhood of the leaf core center within the ROI region of the depth map. The depth calculation unit is used to perform statistical filtering on the depth pixel values in the 3×3 pixel neighborhood of the leaf core center, remove outliers, and calculate the average depth value. The average depth value represents the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core.
5. The system according to claim 4, characterized in that, The distance correction module includes: The attitude data parsing unit is used to parse the IMU attitude data to obtain the current pitch and roll angles of the depth camera. The actual total tilt angle calculation unit is used to obtain the actual total tilt angle of the camera optical axis relative to the gravity plumb line based on the algebraic sum of the 45° preset angle and the pitch angle. The plant height calculation unit is used to construct a spatial triangular geometric model based on the roll angle, the actual total tilt angle, and the installation height of the depth camera, and to perform vertical projection correction on the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core to obtain the vertical plant height of the corn.
6. A method for measuring the height of the corn canopy apex based on a sprinkler irrigation vehicle-mounted down-view depth camera, using the system described in any one of claims 1-5, characterized in that, include: A depth camera is installed under the sprinkler truss to capture RGB and depth images of the field from an oblique perspective, and to obtain the IMU attitude data of the depth camera. Based on a deep learning object detection model, the corn leaf core region in the RGB image is identified and located, and the corn leaf core region is mapped to the corresponding depth image to obtain the depth map ROI region; Based on the ROI region of the depth map, depth extraction is performed to obtain the oblique straight-line distance from the optical center of the camera to the growth point of the corn leaf core; The IMU attitude data is analyzed, and a spatial triangular geometric model is constructed to perform vertical projection correction on the oblique straight-line distance to obtain the vertical plant height of the corn.
7. The method according to claim 6, characterized in that, Data collection methods include: Adjust the optical axis of the depth camera so that it forms a 45° angle with the vertical line of gravity and points obliquely downwards toward the crop row; Using a depth camera, RGB images and depth maps of the field are continuously acquired from a 45° oblique perspective, and Euler angle attitude data output by the inertial measurement unit (IMU) built into the depth camera are recorded simultaneously. Within the edge computing terminal, a unified timestamp benchmark ensures that each frame of RGB image, depth image, and Euler angle pose data corresponds spatiotemporally.
8. The method according to claim 6, characterized in that, Methods for obtaining the ROI region of a depth map include: A deep learning object detection model is used to identify leaf core features in the RGB image to obtain the bounding box of the leaf core region. Based on the geometric center coordinates of the bounding box of the leaf core region, the ROI region is mapped in the corresponding depth image.