Wheat canopy geometry parameter extraction method, system, device and storage medium

Abstract

The application discloses a wheat crown layer geometric parameter extraction method, system, equipment and a storage medium, and comprises the following steps: constructing a multi-modal sensor integrated acquisition system; adopting an ESO-LQR-based pitch axis disturbance control strategy for the acquisition system; collecting point cloud data and image data of the wheat by using the acquisition system, extracting two-dimensional traits based on color indexes by using the image data, and extracting three-dimensional crown layer structure distribution traits by using the point cloud data and the image data. The ESO-LQR-based pitch axis disturbance control strategy comprises the following steps: establishing a pitch axis dynamics model and performing disturbance analysis; expanding the lumped disturbance into a system state, designing a third-order extended state observer (ESO), and generating a disturbance estimation value; constructing a state vector for an optimal controller (LQR), designing a cost function, solving an optimal feedback matrix through a Riccati equation, and generating a control amount; and compensating the disturbance estimation value into the control amount to generate a final control input.

Description

Technical Field

[0001] This invention relates to the field of agricultural testing technology, and in particular to methods, systems, equipment, and storage media for extracting geometric parameters of wheat canopy. Background Technology

[0002] Wheat is one of the world's major food crops, and its planting area and yield are crucial to farmers' economic benefits and national food security. Therefore, rapidly and accurately obtaining wheat growth phenotypic data has become a hot topic in modern agricultural research. Geometric parameters such as wheat canopy coverage, flatness, and yellowing index are core indicators for assessing crop growth status.

[0003] Traditional measurement methods rely on manual visual inspection or a single sensor (such as a handheld laser rangefinder), which suffers from low efficiency, limited data dimensions, and poor dynamic adaptability, making it difficult to meet the demands of modern agriculture for efficient and accurate data. Furthermore, complex field environments, such as gravel ground and vehicle vibrations, often affect the stable operation of sensors, hindering efficient data acquisition. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method, system, device, and storage medium for extracting geometric parameters of wheat canopy. It utilizes a multimodal sensor integrated acquisition system and employs an ESO-LQR-based pitch axis disturbance rejection control strategy, overcoming the deficiencies of existing technologies that use a single sensor, resulting in limited data dimensions and inaccurate data acquisition due to field vibrations.

[0005] To achieve the above objectives, the present invention provides a method for extracting geometric parameters of wheat canopy, comprising the following steps: constructing a multimodal sensor integrated acquisition system, the acquisition system including an image-stabilized acquisition device and multiple sensors, the multiple sensors including at least a camera, a tightly coupled lidar, and an inertial measurement unit (IMU), the image-stabilized acquisition device including a three-axis imaging gimbal and a vibration damping and fixing device, the multiple sensors being mounted on the three-axis imaging gimbal; employing an ESO-LQR-based pitch axis disturbance rejection control strategy for the acquisition system; acquiring point cloud data and image data of wheat using the acquisition system; performing two-dimensional trait extraction based on color index using the image data; and performing three-dimensional canopy structure distribution trait extraction using the point cloud data and the image data.

[0006] The pitch axis disturbance rejection control strategy based on ESO-LQR further includes the following steps: establishing a pitch axis dynamic model and performing disturbance analysis, wherein the pitch axis dynamic model includes lumped disturbances, which include low-frequency disturbances, high-frequency disturbances, and coupled disturbances; expanding the lumped disturbances into system states, designing a third-order extended state observer (ESO), and generating disturbance estimates; constructing a state vector for the optimal controller (LQR), designing a cost function, and solving for the optimal feedback matrix using the Riccati equation to generate control inputs; and compensating the disturbance estimates into the control inputs to generate the final control inputs.

[0007] The pitch axis disturbance rejection control strategy based on ESO-LQR for the acquisition system further includes the following steps: angle initialization, which includes: manual angle initialization and automatic angle initialization, wherein:

[0008] The manual angle initialization includes: controlling the pitch axis angle by sending a PWM signal or an S.Bus signal via a remote controller, and setting and locking the pitch axis and yaw axis angles based on the angle values ​​fed back by the inertial measurement unit (IMU);

[0009] The automatic angle initialization includes: setting the target angle of the pitch axis, obtaining the current angle of the pitch axis, automatically adjusting the pitch angle of the acquisition system based on the angle difference between the current angle and the target angle, and entering the angle locking mode when the angle difference reaches a predetermined error range.

[0010] The method of extracting two-dimensional traits based on color index using the image data further includes the following steps: image data preprocessing, including: brightness correction, using homomorphic filtering to separate the illumination component and reflection component of the image data; noise elimination, using anisotropic diffusion filtering to suppress salt-and-pepper noise; image registration, using Hough transform to detect the main row direction of the wheat, calculating the tilt angle and performing rotation correction.

[0011] The extraction of two-dimensional features based on color indices using the image data further includes the following steps: green area index extraction, comprising: multi-color space synergistic enhancement, introducing the LAB color space to enhance the first supergreen index I. GAI Optimization was performed, including HSV spatial background removal; dynamic threshold segmentation and morphological optimization were carried out using Otsu adaptive threshold segmentation and region filtering; and the green area index (GAI) was calculated by converting the mask coverage area and imaging resolution to the actual green proportion.

[0012] The extraction of two-dimensional traits based on color indices using the image data further includes the following steps: canopy yellowing index extraction, comprising:

[0013] Multispectral feature enhancement includes: converting the image data into the LAB color space, extracting the a channel and normalizing the a channel, defining the initial form of the canopy yellowing index CYI using the statistical extreme values ​​of the a channel in the early stage of grain filling and the statistical extreme values ​​of the a channel in the mature stage; calculating the second supergreen index ExG based on the image data and generating an ExG image; applying Otsu thresholding to the ExG image to generate a first vegetation mask, setting a fixed threshold for the a channel to generate a second vegetation mask, and merging the first vegetation mask and the second vegetation mask to generate a third vegetation mask;

[0014] Texture feature fusion includes: extracting texture features in the wheat ear region using the third vegetation mask; calculating contrast and correlation using the gray-level co-occurrence matrix; constructing a multiple linear regression model by combining the canopy yellowing index CYI, the contrast, and the correlation; and using the multiple linear regression model to predict the degree of yellowing.

[0015] The extraction of three-dimensional canopy structure distribution characteristics using the point cloud data and the image data further includes the following step: canopy coverage calculation, including:

[0016] Point cloud projection and rasterization include: preprocessing the point cloud data, projecting the preprocessed point cloud data onto a horizontal plane, and dividing it into raster grids; calculating the proportion of vegetation points for each raster grid; if the number of point clouds in a raster grid is less than a first set value, marking the raster grid as a low-density area and triggering interpolation processing.

[0017] The sliding window weighted optimization includes: using a 3×3 grid as the sliding window, performing a local coverage weighted average on all grids within each sliding window; and removing outlier grids whose local coverage is greater than a first percentage value or less than a second percentage value.

[0018] Global canopy coverage calculation and verification include: averaging the grid coverage of all sliding windows that have undergone the weighted optimization of the sliding window to generate global canopy coverage; and performing linear regression analysis between the global canopy coverage and the actual measured canopy coverage to verify reliability.

[0019] The extraction of three-dimensional canopy structure distribution characteristics using the point cloud data and the image data further includes the following step: canopy flatness calculation, including:

[0020] Point cloud segmentation and neighborhood construction: The point cloud data is divided into multiple cubic units, and the neighborhood radius of each cubic unit is selected according to the growth stage to generate a point cloud set.

[0021] Covariance matrix and PCA analysis: For each cubic cell, the centroid of the point cloud set is calculated, the covariance matrix is ​​constructed, and the covariance matrix is ​​decomposed into eigenvalues ​​to generate eigenvalues ​​e1, e2 and e3 and corresponding eigenvectors, where e1≥e2≥e3;

[0022] Flatness index calculation and optimization: If the number of point clouds in a cubic cell is less than a second preset value, the cubic cell is marked as an invalid cell. The flatness index is defined after normalizing the eigenvalues ​​e1, e2, and e3. Set a constraint range for the flatness index CFI, and truncate it if it exceeds the constraint range;

[0023] Spatial interpolation and visualization: For the invalid cells, inverse distance weighted interpolation is used, and the flatness index CFI is mapped according to the color gradient.

[0024] In another aspect, the present invention provides a wheat canopy geometric parameter extraction system, employing the above-described wheat canopy geometric parameter extraction method, comprising:

[0025] A multimodal sensor integrated acquisition system, the acquisition system including an image stabilization acquisition device and multiple sensors, the multiple sensors including at least a camera, a tightly coupled lidar and an inertial measurement unit (IMU), the image stabilization acquisition device including a three-axis imaging gimbal and a vibration reduction and fixing device, the multiple sensors being mounted on the three-axis imaging gimbal;

[0026] The control module is used to employ an ESO-LQR-based pitch axis disturbance rejection control strategy for the acquisition system.

[0027] The extraction module is used to collect point cloud data and image data of wheat using the acquisition system, perform two-dimensional trait extraction based on color index using the image data, and perform three-dimensional canopy structure distribution trait extraction using the point cloud data and the image data.

[0028] The steps of the pitch axis disturbance rejection control strategy based on ESO-LQR specifically include: establishing a pitch axis dynamic model and performing disturbance analysis, wherein the pitch axis dynamic model includes lumped disturbances, which include low-frequency disturbances, high-frequency disturbances, and coupled disturbances; expanding the lumped disturbances into system states, designing a third-order extended state observer (ESO), and generating disturbance estimates; constructing a state vector for the optimal controller (LQR), designing a cost function, and solving for the optimal feedback matrix using the Riccati equation to generate control inputs; and compensating the disturbance estimates into the control inputs to generate the final control inputs.

[0029] In another aspect, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the wheat canopy geometric parameter extraction method described above.

[0030] As can be seen from the above solutions, the advantages of the present invention are:

[0031] A multi-modal sensor integrated acquisition system is adopted to achieve stable and efficient acquisition of multi-sensor data in the field. By using an ESO-LQR-based pitch axis disturbance rejection control strategy, the extended state observer (ESO) is used to estimate and compensate for nonlinear disturbances (such as low-frequency vibration, high-frequency vibration and coupling interference) in real time. Combined with the state feedback optimization of the LQR optimal control law, the disturbance rejection capability and robustness of the acquisition system are significantly improved.

[0032] By employing a fusion GAI computing framework, which integrates color space transformation, dynamic threshold segmentation, and morphological optimization, the accurate extraction of green areas in wheat canopy under complex field conditions is achieved.

[0033] A method for determining crop yellowing degree using Lab color space conversion is employed, and non-destructive monitoring is achieved through threshold segmentation and texture analysis.

[0034] By using canopy cover to quantify differences in wheat canopy structure, this study combines three-dimensional point cloud processing, rasterization analysis, and spatial optimization techniques. Through multi-level spatial optimization and physical constraints, it significantly improves the robustness and farmland applicability of canopy cover measurement, providing reliable vegetation structure parameters for precision agriculture.

[0035] By utilizing canopy flatness to quantify the geometric undulation characteristics of wheat canopy surfaces, and based on point cloud blocks and PCA analysis, an index calculation system integrating physical properties and statistical models was constructed, providing irreplaceable decision support for smart agriculture. Attached Figure Description

[0036] Figure 1 This is a flowchart of the wheat canopy geometric parameter extraction method of the present invention;

[0037] Figure 2A This is a perspective view of the data acquisition system of the present invention;

[0038] Figure 2B This is a side view of the data acquisition system of the present invention;

[0039] Figure 2C This is a front view of the data acquisition system of the present invention;

[0040] Figure 3 This is a structural diagram of the sensor support base of the present invention;

[0041] Figure 4 for Figure 1 Flowchart of step S20;

[0042] Figure 5A This is a schematic diagram of the LQR controller structure in the prior art;

[0043] Figure 5B This is a schematic diagram of the ESO_LQR composite control structure of the present invention;

[0044] Figure 6 for Figure 1 Another flowchart for step S20;

[0045] Figure 7 for Figure 6 Flowchart of step S202;

[0046] Figure 8 for Figure 1 Flowchart of step S30;

[0047] Figure 9 for Figure 8 Flowchart of step S300;

[0048] Figure 10 for Figure 8 Flowchart of step S310;

[0049] Figure 11 for Figure 8 Another flowchart of step S310;

[0050] Figure 12 for Figure 8 Flowchart of step S320;

[0051] Figure 13 for Figure 8 Flowchart of step S330;

[0052] Figure 14 for Figure 8 Flowchart of step S340;

[0053] Figure 15 This is a schematic diagram of the canopy coverage of the present invention;

[0054] Figure 16A This is a color gradient diagram of canopy flatness in this invention;

[0055] Figure 16B This is a canopy flatness distribution diagram of the present invention;

[0056] Figure 17 This is a structural diagram of the wheat canopy geometric parameter extraction system of the present invention;

[0057] Figure 18 This is a structural diagram of the electronic device of the present invention;

[0058] In the attached figures, the following labels are used:

[0059] 100 - Method for extracting geometric parameters of wheat canopy; 10 - Acquisition system;

[0060] 1-L-type adapter mounting plate; 2-Three-axis imaging gimbal;

[0061] 20 - Roll axis motor; 21 - Yaw axis motor;

[0062] 22-Pitch axis motor; 3-Sensor support base;

[0063] 30 - First long groove; 31 - Support plate;

[0064] 32 - Second long slot; 4 - Camera;

[0065] 5-T-type mounting components; 6-Shock-absorbing balls;

[0066] 7 - Inertial Measurement Unit (IMU); 8 - LiDAR;

[0067] 200 - System for extracting geometric parameters of wheat canopy; 201 - Control module;

[0068] 202 - Extraction module; 300 - Electronic device;

[0069] 301 - Processor; 302 - Memory;

[0070] 303 - Computer program; 33 - Sensor;

[0071] M1 - Direction of movement of the support plate; M2 - Direction of movement of the sensor;

[0072] S10~S30,S200~S240,S201,S202,S300~S303,S310~S313,S320~S322,S330~S333,S340~S344 - Steps. Detailed Implementation

[0073] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments to further understand the purpose, solution and effect of the present invention, but it is not intended to limit the scope of protection of the appended claims.

[0074] This invention focuses on phenotypic measurement in wheat breeding plots in the field, and designs and develops a collaborative intelligent data acquisition system. A ground-based mobile platform carrying multiple information sensors is used, employing a stabilized image acquisition device. This platform utilizes a planar array lidar to acquire structural parameters such as canopy height and crop row width, while simultaneously using an RGB camera to capture various color traits of the crop.

[0075] This invention proposes a method for collecting wheat field information based on multimodal sensor acquisition technology. It designs an acquisition system mounted on a self-propelled acquisition platform, aiming to construct a high-precision three-dimensional crop map and two-dimensional color traits in the field while in motion, and proposes a corresponding dynamic method for geometric parameters.

[0076] like Figure 1 As shown, an embodiment of the present invention provides a method 100 for extracting geometric parameters of wheat canopy, comprising the following steps:

[0077] S10: Construct a multimodal sensor integrated acquisition system;

[0078] S20: The pitch axis disturbance rejection control strategy based on ESO-LQR is adopted for the acquisition system;

[0079] S30: Use the acquisition system to collect point cloud data and image data of wheat, use the image data to extract two-dimensional traits based on color index, and use the point cloud data and image data to extract three-dimensional canopy structure distribution traits.

[0080] In step S10, the multimodal sensor integrated acquisition system mainly includes an image stabilization acquisition device and multiple sensors. The multiple sensors include at least a camera, a tightly coupled lidar, and an inertial measurement unit (IMU). The image stabilization acquisition device includes a three-axis imaging gimbal and a shock-absorbing fixing device. The multiple sensors are mounted on the three-axis imaging gimbal. The image stabilization acquisition device is set on the top of the self-propelled vehicle.

[0081] Specifically, such as Figures 2A to 2C The diagram shows the structure of the data acquisition system 10. The system includes an L-shaped adapter plate 1, a three-axis imaging gimbal 2, a sensor mounting base 3, a camera 4, a T-shaped mounting component 5, a shock-absorbing ball 6, an inertial measurement unit (IMU) 7, and a lidar 8. The L-shaped adapter plate 1, sensor mounting base 3, T-shaped mounting component 5, and shock-absorbing ball 6 constitute a shock-absorbing and fixing device. The camera 4, IMU 7, and lidar 8 are mounted on the three-axis imaging gimbal 2 via the sensor mounting base 3. The three-axis imaging gimbal 2 includes a roll axis, a pitch axis, and a yaw axis. The entire data acquisition system 10 utilizes a ground-based data acquisition vehicle (e.g., a self-propelled vehicle) as a mobile platform and the three-axis imaging gimbal 2 as a stabilization platform, achieving stable and efficient data acquisition from multiple sensors in the field.

[0082] In the actual construction process, self-propelled vehicles adapted to different field environments are selected as mobile carriers. For example, in the relatively flat and open wheat fields of the northern plains, large four-wheel drive self-propelled vehicles can be selected to ensure driving stability and range. In the hilly and small farmland areas of the south, small and lightweight self-propelled vehicles are selected to facilitate flexible movement in the fields.

[0083] The mechanical architecture design of the data acquisition system 10 fully integrates the differences in characteristics between ground and aerial platforms. Firstly, ground-based data acquisition vehicles have drawbacks such as smooth walls without mounting points, a hollow center but limited internal space, and severe sensor obstruction. Therefore, this embodiment adopts a strategy of mounting the data acquisition system 10 to the outer wall of the vehicle. The three-axis imaging gimbal 2 should be installed vertically, and connected via an L-shaped adapter plate 1. The hardware structure of the data acquisition system 10 is made entirely of aluminum alloy components. The main components include three major axis drive units, a central control module, and a mounting structure. The drive motors of the three major axis drive units, including the roll axis motor 20, yaw axis motor 21, and pitch axis motor 22, are high-efficiency integrated motors, small in size, high in torque, and low in noise. The yaw axis motor 21 can achieve 360° continuous yaw axis rotation, equipped with a 17-bit magnetic encoder and a harmonic reducer (reduction ratio 1:100), with a repeatability accuracy of ±0.05°, meeting the heading control requirements for continuous scanning between rows in large fields. The pitch axis motor 22 has a pitch adjustment range of 0 to 90°, and its high torque characteristics can drive the sensor mounted on the carrier plate to control the pitch angle.

[0084] The three-axis imaging gimbal 2 of the image stabilization acquisition device needs to have high-precision angle adjustment capabilities, with a horizontal rotation accuracy of ±0.01° and pitch and roll accuracy controlled within a very small range to ensure that the mounted sensor can stably acquire data. In addition to the shock-absorbing ball 6, the vibration damping and fixing device can also adopt a special rubber shock-absorbing pad and spring combination structure to effectively absorb vibrations during vehicle operation and reduce the impact on the sensor.

[0085] like Figure 3 As shown, the sensor support base 3 includes a first elongated groove 30 and a support plate 31 that moves along the first elongated groove 30. A second elongated groove 32 perpendicular to the first elongated groove 30 is formed on the support plate 31, and various sensors are mounted on the support plate 31. For example, the length of the second elongated groove 32 is 5cm, and the range of movement of the support plate 31 along the first elongated groove 30 is 5cm. In this way, the sensor 33 (including the camera 4, the inertial measurement unit IMU7, and the lidar 8) can be mounted on the sensor support base 3 within 25cm. 2 The position can be flexibly adjusted within the planar space to optimize the layout and imaging space of the acquisition system 10. M1 represents the movable direction of the support plate 31, and M2 represents the movable direction of the sensor 33.

[0086] The image stabilization acquisition device is bolted to the top center of the front of the ground acquisition vehicle (e.g., a self-propelled vehicle). Considering the mounting position on the vehicle and the imaging characteristics of the sensor, the height of the image stabilization acquisition device is set at 2.2m. The lidar 8, inertial measurement unit (IMU) 7, and camera 4 are all fixed to the image stabilization acquisition device via sensor support base 3. The lidar 8 is installed upside down, with the IMU 7 closely attached to the bottom of the lidar 8, together forming the radar inertial measurement system. The camera 4 is installed directly below the sensor support base 3. During installation, the relative positions must be strictly calibrated to ensure the consistency and accuracy of the measurement data. After installation, the camera 4 and the radar inertial measurement system are calibrated multiple times using professional calibration equipment and software to ensure that the radar inertial measurement system error is within the allowable range. The lidar 8, for example, is a fixed-direction non-repeating scanning radar, which features high resolution and a large measurement range. In agricultural environments, it can effectively measure distances up to 100 meters, with an angular resolution accurate to 0.5°. The mounting structure, consisting of the T-shaped mounting component 5 and the L-shaped adapter fixing plate 1, is 30cm high and has a load-bearing capacity of 1.5kg.

[0087] The main parameters of the acquisition system 10 are shown in Table 1. It can acquire multi-dimensional sensor information. The size, structure, power supply and other parameters are shown in the table.

[0088] Table 1 Main parameters of data acquisition system 10

[0089]

[0090] The data acquisition system 10 can be equipped with a high-precision BeiDou positioning system, achieving centimeter-level positioning accuracy and precisely recording vehicle trajectories. Before actual operation, based on the shape, size, and terrain of the farmland, a pre-set "S"-shaped or round-trip data acquisition path is used using professional path planning software. For example, for rectangular wheat fields, a round-trip path is preferred to ensure comprehensive and efficient data acquisition; for irregularly shaped farmland, an "S"-shaped path is used to ensure coverage of all areas to be measured.

[0091] The vehicle speed is adjusted based on the crop growth status in the field and the sensor performance, generally within 2 m / s, with 1.5 m / s typically chosen to ensure efficiency and mapping accuracy. In the early stages of crop growth, when the plants are shorter, the speed can be appropriately increased; in the later stages of crop growth, the speed is reduced to ensure data accuracy.

[0092] In step S20, a pitch axis disturbance rejection strategy based on ESO-LQR control is used to address the vibration disturbance problem of the mobile platform in the field. By using an extended state observer (ESO) to estimate nonlinear disturbances in real time and compensate them to the optimal controller LQR, the pitch axis attitude stability and disturbance rejection capability are significantly improved.

[0093] like Figure 4As shown, step S20 further includes the following steps:

[0094] S210: Establish a pitch axis dynamic model and perform disturbance analysis. The pitch axis dynamic model includes lumped disturbances, which include low-frequency disturbances, high-frequency disturbances, and coupled interference.

[0095] S220: Expand the lumped disturbance into the system state, design a third-order extended state observer (ESO), and generate disturbance estimates;

[0096] S230: Construct the state vector for the optimal controller LQR, design the cost function, and solve the optimal feedback matrix through the Riccati equation to generate the control quantity;

[0097] S240: Compensate the disturbance estimate into the control input to generate the final control input.

[0098] Specifically, such as Figure 5A The diagram shown is a schematic of a traditional LQR controller in the prior art. The traditional LQR controller is designed based on the state feedback principle, and its state equations are described by the following equations (1) and (2):

[0099]

[0100] y = Cx (2),

[0101] in, A state vector is used to describe the system state, such as angle and angular velocity. Control input vectors are used to describe things like motor drive voltages. The system state matrix is ​​used to describe the internal dynamic characteristics of the system. The input matrix is ​​used to describe the effect of control inputs on the state. The output vector describes the observable physical quantities of the system, namely sensor measurements. The output matrix is ​​used to map the state to the observable output. The optimal feedback matrix K is obtained by solving the Riccati equation, which minimizes the cost function. However, traditional LQR controllers have limited ability to suppress disturbances caused by internal coupling torques (such as nonlinear friction of motors, inertial coupling, etc.) of the three-axis imaging gimbal 2.

[0102] Pitch axis dynamics modeling and disturbance analysis: First, establish the dynamic equations. The pitch axis dynamics model can be simplified as follows:

[0103]

[0104] In the formula: J p τ is the moment of inertia of the pitch axis. d,p The total disturbance includes external vibration, wind load moment, and frame coupling torque; Cp θ is the pitch axis damping coefficient; p The pitch angle represents the current pitch attitude of the system. The first derivative of the pitch angle, i.e., the pitch angular velocity; K is the second derivative of the pitch angle, i.e., the pitch angular acceleration; t,p u is the pitch axis moment constant; p To control the input voltage.

[0105] From the perspective of disturbance characteristics, low-frequency disturbances (<5Hz) are mainly caused by carrier vibration, with amplitude fluctuations ranging from ±0.6Nm; high-frequency disturbances (510Hz) are caused by motor cogging effect and nonlinear friction of gear transmission, exhibiting periodic harmonic characteristics; coupled interference occurs when the roll axis and yaw axis motions are transmitted to the pitch axis disturbance through the frame stiffness of the image stabilization acquisition device. Maintaining the pitch axis angle is crucial during field data acquisition; therefore, this embodiment uses the pitch axis for ESO-LQR composite control design. The schematic diagram of the ESO-LQR composite control structure is shown below. Figure 5B .

[0106] ESO perturbation observer will lumped perturbation τ d,p Extending to the system state, a third-order extended state observer (ESO) is designed as follows (4):

[0107]

[0108] Where z1 is the pitch angle θ p The estimated value, z2 is the angular velocity. The estimated values ​​are z3 and β1, β2, and β3, respectively, which are the gain parameters. These parameters are set using the bandwidth parameterization method (e.g., β1 = 150, β2 = 7500, β3 = 125000, corresponding to the observer bandwidth ω). o =50 rad / s).

[0109] LQR optimal control law, constructing state vector The cost function is given by equation (5):

[0110]

[0111] Where q1 is the pitch angle θ p The weighting coefficient is used to adjust the angle tracking accuracy; q2 is the pitch angular velocity. The weighting coefficient is used to adjust the response speed; r is the control input u. p The weighting coefficient is used to suppress excessive control input; t is time.

[0112] The optimal feedback matrix, K = [k1, k2], is obtained by solving the Riccati equation. The compensated control input is given by equation (6):

[0113]

[0114] Among them, K t,p J is the pitch axis moment constant; p The moment of inertia of the pitch axis (kg·m) 2 );θ ref The target pitch angle (rad).

[0115] like Figure 6 As shown, step S20 further includes the following step: angle initialization S200, which includes: manual angle initialization S201 and automatic angle initialization S202, wherein:

[0116] Step S201 includes: sending a PWM signal or an S.Bus signal via a remote controller to control the pitch axis angle, and setting and locking the pitch axis and yaw axis angles based on the angle values ​​fed back by the inertial measurement unit (IMU).

[0117] Step S202 includes: setting the target angle of the pitch axis, acquiring the pitch angle of the lidar or the current angle of the pitch axis, automatically adjusting the pitch angle of the acquisition system based on the angle difference between the pitch angle of the lidar or the current angle and the target angle, and entering the angle locking mode when the angle difference reaches the predetermined error range.

[0118] Specifically, in step S201, the angle manual initialization control method uses the PWM signal of S.Bus to control the angle. A specific angle can be fixed by manual remote control. By observing the angle feedback value of the inertial measurement unit IMU7 attached to the surface of the lidar 8, the pitch and yaw angle control of the sensor can be realized. At the same time, the sensor support base 3 has a feedback function, which can compensate for any instantaneous offset.

[0119] In step S202, automatic angle initialization only requires inputting the acquisition parameters, and the acquisition system 10 will automatically rotate to the appropriate acquisition position and enter the angle locking mode. For example... Figure 7The diagram shows the automatic angle initialization control flowchart, which is as follows: Start; Power on the acquisition system 10; Read the current pitch angle θ1 of the lidar 8; Set the installation height, plant (i.e., wheat) height, field of view size, and field of view acquisition length of the acquisition system 10; Calculate the target pitch angle θ2; Calculate the difference between the target pitch angle θ2 and the current pitch angle θ1; Determine if θ2-θ1 is greater than zero; When θ2-θ1 is greater than zero, the acquisition system 10 issues a control command, and the three-axis imaging gimbal 2 "tilts up"; When θ2-θ1 is less than or equal to zero, the acquisition system 10 issues a control command, and the three-axis imaging gimbal 2 "tilts down"; Calculate whether the angle difference between the target pitch angle θ2 and the current pitch angle θ1 reaches the predetermined error range of 5%; When the angle difference reaches the predetermined error range of 5%, enter the gimbal angle locking mode; otherwise, return to calculate the difference between the target pitch angle θ2 and the current pitch angle θ1; End.

[0120] The current pitch angle θ1 is calculated by the inertial measurement unit (IMU7) attached to the top of the lidar 8, and sent to the microcontroller via the serial port of the acquisition computer. The target pitch angle θ2 is the aforementioned θ. ref The control commands use the S.BUS protocol output by the microcontroller. The microcontroller uses an Arduino Uno for S.BUS data parsing and command redefinition, configured with a 100kHz baud rate, 8 data bits, even parity, 2 stop bits, and no current control. The protocol format contains 25 bytes, where data 1 to 22 represent 11-bit values ​​for the 16 channels. During hardware circuit construction, the S.BUS signal needs to be inverted before being connected to the microcontroller for proper reading.

[0121] The automatic angle control adopts the PID (Proportion Integration Differentiation) control strategy to provide feedback control for the pitch angle. The microcontroller sends the angle rotation command to the pitch axis motor 22 of the three-axis imaging gimbal 2 to complete the response and angle locking. This process can solve the problem of inaccurate initial angle setting of the remote control.

[0122] In order to collect data on the wheat area to be measured, the lidar 6 is mounted obliquely downwards by the three-axis imaging gimbal 2, which also allows for free switching of the acquisition angle. By setting the initial acquisition angle value, the angle locking mode of the three-axis imaging gimbal 2 can be used to achieve stable acquisition at the locked angle during subsequent acquisition processes.

[0123] In step S30,

[0124] Based on the collected point cloud and image raw data, this study addresses the problems of low efficiency in parsing three-dimensional phenotypic features of crops and insufficient multi-dimensional data fusion in complex field environments. By researching multi-dimensional data preprocessing techniques and image color feature extraction algorithms, two-dimensional trait extraction of crops based on color indices is achieved. Based on the characteristics of wheat point clouds, the influence of search radius selection on phenotypic imaging is analyzed in depth, and point cloud feature extraction methods are used to mine and verify three-dimensional traits of wheat.

[0125] Dynamic extraction of canopy geometric information includes two-dimensional trait extraction based on color index and three-dimensional canopy structure distribution trait extraction. Through multi-dimensional analysis of point cloud and image data, accurate quantification of geometric parameters such as coverage (CC), flatness (CFI), and yellowing index (CYI) is achieved.

[0126] Before extracting the 3D canopy structure distribution using point cloud and image data, the point cloud data needs to be preprocessed. This includes: during coordinate transformation, based on the IMU attitude information recorded before acquisition, using coordinate transformation formulas to transform the point cloud data from the radar coordinate system to the world coordinate system. In the region of interest extraction stage, using geographic information data of farmland boundaries, combined with the range of point cloud data acquired by lidar, a pass-through filter is used for cropping. For example, for a specific farmland, the x-axis cropping range is set to [50, 50], and the y-axis cropping range is set to [50, 50], removing invalid point cloud data outside the farmland boundary.

[0127] In the Octree-based downsampling method, the number of octree segmentation levels and the number of points in each child node are reasonably set according to the initial density of the point cloud data and the subsequent processing requirements. During the noise filtering stage, the parameter n and the standard deviation threshold n_sigma for statistical filtering are determined through multiple experiments and data analysis.

[0128] Point cloud normal vector calculation is based on PCA analysis. During the calculation process, the neighborhood radius r is reasonably selected according to the distribution characteristics of the crop point cloud. For example, for wheat crops, the neighborhood radius r is set to 0.15 in the early stage of growth; in the later stage of growth, due to the denser crop canopy, the neighborhood radius r is appropriately adjusted to 0.3 to ensure accurate calculation of the normal vector.

[0129] like Figure 8 and Figure 9 As shown, the two-dimensional trait extraction based on color index includes: S300: Image data preprocessing, including:

[0130] S301: Brightness correction, using homomorphic filtering to separate the illumination component and reflection component of image data;

[0131] S302: Noise cancellation, based on anisotropic diffusion filtering to suppress salt-and-pepper noise;

[0132] S303: Image registration, detecting the main direction of the wheat rows through Hough transform, calculating the tilt angle and performing rotation correction.

[0133] Specifically, to address issues such as uneven lighting and noise interference in field images, a multi-stage enhancement and correction process is employed, including:

[0134] Brightness correction is performed by using homomorphic filtering to separate the illumination and reflection components of the image, suppressing the interaction between canopy overexposure and soil shadow, as shown in equation (7):

[0135]

[0136] In the formula: γ H γ is the high-frequency gain of the image after Fourier transform; L γ is the base frequency gain of the image after Fourier transform; D0 is the filter cutoff frequency; u, v are coordinate variables in the frequency domain; c is the steepness coefficient of the ramp function; n is the order of the filter; and γ H =0.5, γ L =0.25, D0=20.

[0137] Noise cancellation is achieved by suppressing salt-and-pepper noise based on anisotropic diffusion filtering (PeronaMalik model) using a gradient threshold function, as shown in equation (8):

[0138]

[0139] Wherein, the derivative function With parameter K=30, this noise reduction method can improve the sharpness of the blade edges while preserving sufficient texture details.

[0140] Image registration was performed based on the characteristics of wheat row sowing. The main direction of the rows in the field was detected by Hough transform, and the tilt angle θ was calculated. RGB And perform rotational correction. Tilt angle θ RGB The calculation formula is as follows (9):

[0141]

[0142] Where, Δy i Δx i The vertical and horizontal displacements (in pixels) of the i-th wheat row detected by the Hough transform in the image.

[0143] like Figure 10 As shown, the two-dimensional trait extraction based on color index includes S310: Green Area Index (GAI) extraction, which includes:

[0144] S311: Multi-color space synergistic enhancement, introducing the LAB color space for the first super green index I. GAI Optimize and remove background in HSV space;

[0145] S312: Dynamic threshold segmentation and morphological optimization, using Otsu adaptive threshold segmentation and region filtering for dynamic threshold segmentation and morphological optimization;

[0146] S313: Green Area Index Calculation. The Green Area Index (GAI) is calculated by converting the mask coverage area and imaging resolution to the actual green area ratio.

[0147] The Green Area Index (GAI) is defined as the total lateral area of ​​green vegetation elements per unit of land surface. It is a core agronomic parameter characterizing the density of green tissue cover in crop canopies, and is defined as the proportion of the projected area of ​​green photosynthetic organs (leaves, stems, etc.) of crops per unit land area. Traditional measurements rely on destructive sampling and manual measurement, which are inefficient and difficult to implement for dynamic field monitoring. With the development of phenotyping platforms and the popularization of imaging technologies, computer vision-based rapid GAI extraction technology has become a hot topic in precision agriculture research.

[0148] This invention employs a fusion of GAI (Graphical Application Intelligence) computing frameworks, combining color space transformation, dynamic threshold segmentation, and morphological optimization to achieve accurate extraction of green regions in wheat canopies under complex field conditions. The specific process is as follows: Figure 11 .

[0149] After the image data is input, the first step is a multi-color space collaborative enhancement mechanism. Using multi-channel feature extraction, and targeting the green characteristics of wheat in the early growth stage, the dynamic response characteristics of the a channel (red-green axis) of the LAB color space are introduced to construct an improved super-green index, as shown in the following formula (10):

[0150] I GAI =2.5L-1.5a-0.2b (10),

[0151] In the formula: L is the luminance component (L channel) in the LAB color space; a is the red-green axis component (a channel) in the LAB color space; b is the yellow-blue axis component (b channel) in the LAB color space.

[0152] L represents the overall brightness and darkness information of the image. The light characteristics of wheat leaves are enhanced by a weighting coefficient of 2.5, highlighting the brightness contrast of green areas. a reflects the color change characteristics from red (negative value) to green (positive value). In Equation (10), a negative weight of 1.5 is used to suppress non-green interference (such as soil or withered yellow areas) and enhance the dynamic response sensitivity of wheat green leaves. b represents the color change characteristics from yellow (positive value) to blue (negative value). In Equation (10), a small negative weight of 0.2 is used to weaken yellow background noise (such as early yellowing leaves of wheat) and optimize the yellow-green gradient separation effect.

[0153] Next, background removal in HSV space is performed: combining the dual threshold constraints of hue (H) and saturation (S), a background mask is defined as follows (11):

[0154]

[0155] In the formula, H is the hue component of the HSV color space; S is the saturation component of the HSV color space.

[0156] HSV spatial background removal can effectively separate wilted tissue (H<30°) from moist soil regions in wheat plants (V<0.1).

[0157] After spatial color enhancement, due to the complexity of the field environment, data fluctuation areas easily exist between the soil and leaf gaps. Dynamic threshold segmentation and morphological optimization can be performed using Otsu adaptive threshold segmentation and region filtering to effectively remove residual soil noise. Specifically, the threshold is determined using the Otsu adaptive method, and for I... GAI The grayscale image is binarized, and the gaps and pores between leaves are filled by morphological closing operation (5×5 elliptical kernel). For residual noise in the soil, area filtering is performed by region screening, and the main structure of the canopy is preserved by connected component analysis, where the threshold is a single connected component area ≥ 100 pixels.

[0158] The GAI index is calculated by converting the mask coverage area and imaging resolution to the actual green percentage, as shown in the following formula (12):

[0159]

[0160] In the formula, S pixel d represents the actual area of ​​a single pixel (based on camera calibration parameters). GSD The ground sampling distance is θ, and the camera's pitch angle is θ4, which can be obtained through the pre-acquisition system 10 (to eliminate perspective distortion); W image H image The width and height (in pixels) of the image; Mask final This represents the total number of pixels in the final vegetation mask.

[0161] like Figure 12 As shown, the two-dimensional trait extraction based on color index includes S320: canopy yellowing index extraction, which includes:

[0162] S321: Multispectral feature enhancement, including: converting image data to LAB color space, extracting and normalizing the a channel, defining the initial form of the canopy yellowing index CYI using the statistical extreme values ​​of the a channel in the early stage of grouting and the statistical extreme values ​​of the a channel in the mature stage; calculating the second supergreen index ExG based on the image data and generating an ExG image; applying Otsu thresholding to the ExG image to generate a first vegetation mask, setting a fixed threshold for the a channel to generate a second vegetation mask, and merging the first and second vegetation masks to generate a third vegetation mask;

[0163] S322: Texture feature fusion, including: extracting texture features in the wheat ear area using a third vegetation mask, calculating contrast and correlation using the gray-level co-occurrence matrix; constructing a multiple linear regression model by combining the canopy yellowing index CYI, contrast, and correlation, and using the multiple linear regression model to predict the degree of yellowing.

[0164] In studies of wheat color traits based on RGB images, wheat at different growth stages exhibits intuitive color differences, specifically a change from green to yellow. To distinguish and quantify these differences, a Lab color space conversion method for assessing yellowing was employed for the later stages of wheat growth. This method utilizes thresholding and texture analysis for non-destructive monitoring. The core principle lies in capturing the color migration patterns and structural changes in the wheat ear from the grain-filling stage to maturity. The specific process is as follows:

[0165] First, color space conversion and feature enhancement are performed. The RGB image is converted to the Lab color space, and the basic response parameters are constructed using the sensitivity of the a channel (red and green axes) to the wheat yellowing process. The interference of light difference is eliminated by normalization, and the Canopy Yellowing Index (CYI) is defined as follows (13):

[0166]

[0167] In the formula: a max This represents the statistical extreme value of channel a during the initial stage of grouting; a min This represents the statistical extreme value of channel a during the mature stage.

[0168] Color space fusion: Simultaneously extract the ExG (Super Green Index) and the a channel of the Lab space from the RGB image; Dynamic threshold segmentation: For the ExG image, the Otsu algorithm is used to adaptively determine the vegetation mask. For the a channel, a threshold a≥5 is set (an empirical value to suppress soil background).

[0169] Meanwhile, in order to continuously transform the yellowing degree, a multiple linear regression model is constructed by combining the color index CYI and texture features. The gray-level co-occurrence matrix texture parameters need to be extracted from the wheat ear region, including contrast (as shown in equation (14)) and correlation (as shown in equation (15):

[0170] Contrast=∑ i,j |ij| 2 ·P(i,j) (14),

[0171] Contrast=∑ i,j |ij| 2 ·P(i,j) (15),

[0172] Where P(i,j) is the probability distribution of the gray-level co-occurrence matrix, and μ and σ are the mean and standard deviation, respectively. Gray-level co-occurrence matrix (GLCM): Window size: 15×15 pixels (corresponding to actual field size of 5cm×5cm).

[0173] Combined with the color index CYI origin Based on texture features, a multiple linear regression model is constructed as shown in equation (16):

[0174] CYI = 0.65CYI origin +0.28Contrast-0.07Correlation (16),

[0175] Multiple linear regression model optimization: Training data: Two hundred wheat canopy samples were collected (covering the grain-filling stage to maturity), and the yellowing level (0-1) was manually labeled. Model fitting: Ridge regression was used to prevent overfitting, and the final weight coefficients are as described above.

[0176] This multiple linear regression model determines the weight coefficients through principal component analysis, which can quantify the continuous change in the degree of yellow color from 0 (early stage of grouting) to 1 (fully mature).

[0177] In step S30, as Figure 13 The extraction of three-dimensional canopy structure distribution characteristics using point cloud data and image data also includes the following steps: S330: Canopy coverage calculation, including:

[0178] S331: Point cloud projection and rasterization, including: preprocessing point cloud data, projecting the preprocessed point cloud data onto a horizontal plane, and dividing it into raster grids; calculating the proportion of vegetation points for each raster grid; if the number of point clouds in a raster grid is less than a first set value, marking the raster grid as a low-density area and triggering interpolation processing.

[0179] S332: Sliding window weighted optimization, including: using a 3×3 grid as a sliding window, performing a local coverage weighted average on all grids within each sliding window; and removing outlier grids whose local coverage is greater than a first percentage value or less than a second percentage value.

[0180] S333: Global coverage calculation and verification, including: averaging the grid coverage of all sliding windows after sliding window weighted optimization to generate global canopy coverage; performing linear regression analysis between global canopy coverage and actual measured canopy coverage to verify reliability.

[0181] like Figure 14 As shown, the extraction of three-dimensional canopy structure distribution characteristics using point cloud data and image data also includes the following steps: S340: Canopy flatness calculation, including:

[0182] S341: Point cloud segmentation and neighborhood construction: Divide the point cloud data into multiple cubic units, select the neighborhood radius for each cubic unit according to the growth stage, and generate a point cloud set.

[0183] S342: Covariance matrix and PCA analysis. Calculate the centroid of the point cloud set for each cubic cell, construct the covariance matrix, perform eigenvalue decomposition on the covariance matrix, and generate eigenvalues ​​e1, e2 and e3 and their corresponding eigenvectors, where e1≥e2≥e3.

[0184] S343: Flatness Index Calculation and Optimization. If the number of point clouds in a cubic cell is less than a second set value, the cubic cell is marked as an invalid cell. After normalizing the eigenvalues ​​e1, e2, and e3, the flatness index is defined as follows: Set a constraint range for the flatness index CFI, and truncate it if it exceeds the constraint range;

[0185] S344: Spatial interpolation and visualization. For invalid cells, inverse distance weighted interpolation is used, and the flatness index CFI is mapped according to the color gradient.

[0186] Specifically, during the growth process of wheat crops, the growth status of each breeding plot differs, not only in vertical structure but also in canopy horizontal structure. To evaluate the differences in canopy distribution among crops, canopy coverage and canopy flatness are used to quantify the differences in canopy structure between crops. This embodiment constructs a dual-index calculation system based on the geometric features of LiDAR point clouds, integrating physical attributes and statistical models, including:

[0187] (1) Calculation of canopy coverage (CC)

[0188] CC quantization is based on point cloud classification and rasterization. First, region cropping and elevation thresholding are performed on the LiDAR point cloud to extract crop points. Specifically, this includes:

[0189] Grid cell division: The 3D point cloud is projected onto a horizontal plane to generate 5cm×5cm grid cells (this size has been verified in field trials to balance accuracy and computational efficiency). The average row spacing of wheat is 15-20cm, and a 5cm grid can capture the cover differences of 2-3 plants within a single row.

[0190] Point cloud classification rules: Vegetation point determination: Elevation threshold segmentation method, points with H≥5cm above the ground are defined as vegetation points (excluding soil surface noise).

[0191] Density correction: If the total number of points n in the grid is less than 10 (first set value), it is marked as a low confidence region and interpolation processing is triggered.

[0192] The percentage of vegetation points within each grid cell (CC) i The calculation formula is as follows (17):

[0193]

[0194] In the formula, The number of cloud points on the plant; This represents the total number of point clouds in the region.

[0195] Then, sliding window weighted optimization is performed, including:

[0196] Window size: A 3×3 grid (15cm×15cm) sliding window is used to simulate the distribution characteristics of wheat populations. The weight of high-density areas is close to 1, while the weight of low-density areas is inherited from the neighboring 8 grids by bilinear interpolation (interpolation radius 15cm).

[0197] Anomaly filtering: Remove outlier rasters with CC > 95% (first percentage value) or CC < 5% (second percentage value) (possibly caused by point cloud noise or fallen plants).

[0198] Finally, global coverage is calculated, including:

[0199] Global coverage is then achieved by integrating local raster results through a sliding window weighted average, as shown in equation (18):

[0200]

[0201] In the formula, M is the number of grid cells; w i Adjust the weights for density.

[0202] Where the weight w i The grid point density is dynamically adjusted, and neighbor interpolation is used to optimize low-density areas. For example... Figure 15 This is a schematic diagram of canopy cover (CC). Different colors represent point clouds at different locations of crops, with red and blue representing areas of concentrated crop cover and areas of sparse crop cover, respectively.

[0203] Manual sampling comparison: 10 1m samples were randomly selected. 2 The percentage of projected area of ​​manually counted plants in a given region was linearly regressed with the CC calculation results (R²). 2 ≥0.92).

[0204] (2) Calculation of Canopy Flatness (CFI)

[0205] The quantification of canopy flatness (CFI) relies on principal component analysis (PCA) to reveal the geometric distribution characteristics of the canopy surface. Specifically, it includes the following steps:

[0206] Point cloud segmentation and neighborhood construction include: Analysis unit division: dividing the canopy point cloud into 20cm×20cm cubic units (covering a single plant and up to 3 clumps of wheat); Neighborhood radius selection: early growth stage (plant height <30cm): neighborhood radius r = 0.1m (to avoid cross-plant interference), mid-to-late growth stage (plant height ≥30cm): neighborhood radius r = 0.2m (to adapt to the close contact characteristics of the canopy).

[0207] Covariance matrix and PCA analysis include: calculating the centroid of the point cloud set P within each analysis unit, constructing the covariance matrix, solving for Cov, and obtaining the eigenvalues ​​e1, e2, e3 and the corresponding eigenvectors.

[0208] Flatness index calculation and optimization, including: after solving for the eigenvalues ​​e1≥e2≥e3, the flatness index CFI is defined as follows (19):

[0209]

[0210] Noise suppression strategy, effective point cloud threshold: if the number of points n in a cell is less than 20, it is marked as invalid data (to avoid eigenvalue distortion caused by sparse point cloud); eigenvalue normalization: divide e1, e2, e3 by the point cloud spatial scale to eliminate the shadow of absolute scale; output range correction: enforce constraint CFI∈[0,1]: if the calculation result exceeds the range, it is truncated.

[0211] Formula (19) quantifies the undulation of the canopy surface by normalizing the variance difference between the secondary principal direction and the weakest direction. When e1 is significantly greater than e2 and e3, the CFI approaches 0, indicating that the canopy is planar; conversely, if the difference between e2 and e3 increases, the CFI increases, reflecting three-dimensional heterogeneity. This method is consistent with the mathematical core of least squares plane fitting, but extends to three-dimensional space to capture complex canopy structures.

[0212] Spatial interpolation and visualization include: missing cell filling: for invalid cells (n<20), inverse distance weighted interpolation (IDW) is used; visualization mapping: CFI values ​​are mapped according to color gradients (red: CFI=1, severe canopy undulation; blue: CFI=0, completely flat). For example... Figure 16A and Figure 16B This is a schematic diagram of canopy flatness. Flatness is divided using a color gradient, with the value normalized to between 0 and 1. Red and blue represent the endpoints of regions with high and low flatness, respectively.

[0213] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0214] like Figure 17 The diagram shown is a structural diagram of a wheat canopy geometry parameter extraction system 200 provided in another embodiment of the present invention, comprising:

[0215] The acquisition system 10 is a multimodal sensor integrated system. The acquisition system 10 includes an image stabilization acquisition device and multiple sensors. The multiple sensors include at least a camera 4, a tightly coupled lidar 8, and an inertial measurement unit (IMU) 7. The image stabilization acquisition device includes a three-axis imaging gimbal 2 and a vibration reduction and fixing device. The multiple sensors are mounted on the three-axis imaging gimbal 2.

[0216] Control module 201 is used to implement an ESO-LQR-based pitch axis disturbance rejection control strategy for the acquisition system 10;

[0217] The extraction module 202 is used to collect point cloud data and image data of wheat using the acquisition system, extract two-dimensional traits based on color index using the image data, and extract three-dimensional canopy structure distribution traits using the point cloud data and image data.

[0218] The steps of the pitch axis disturbance rejection control strategy based on ESO-LQR specifically include:

[0219] Establish a pitch axis dynamic model and perform disturbance analysis. The pitch axis dynamic model includes lumped disturbances, which include low-frequency disturbances, high-frequency disturbances, and coupled interference.

[0220] The lumped disturbance is extended to the system state, and a third-order extended state observer (ESO) is designed to generate disturbance estimates.

[0221] Construct the state vector for the optimal controller LQR, design the cost function, and solve the optimal feedback matrix through the Riccati equation to generate the control quantity;

[0222] The disturbance estimate is compensated into the control quantity to generate the final control input.

[0223] Furthermore, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the wheat canopy geometric parameter extraction system 200 described herein can be referred to the corresponding process in the aforementioned wheat canopy geometric parameter extraction method 100 embodiment, and will not be repeated here.

[0224] Furthermore, another embodiment of the present invention discloses an electronic device 300, such as... Figure 18 As shown, the content in the figure should not be considered as any limitation on the scope of use of the present invention.

[0225] Figure 18 This is a schematic diagram of the structure of an electronic device 300 provided in an embodiment of the present invention. The electronic device 300 specifically includes at least one processor 301 and at least one memory 302. The memory 302 stores a computer program 303, which is loaded and executed by the processor 301 to implement the relevant steps in the wheat canopy geometric parameter extraction method 100 disclosed in any of the foregoing embodiments, such as... Figure 1 The steps are shown.

[0226] In addition, the memory 302, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, and the storage method can be temporary storage or permanent storage.

[0227] In addition to including a computer program capable of performing the wheat canopy geometry parameter extraction method 100 executed by electronic device 300 as disclosed in any of the foregoing embodiments, computer program 303 may further include a computer program capable of performing other specific tasks.

[0228] Another embodiment of the present invention discloses a computer storage medium storing computer-executable instructions. When these computer-executable instructions are loaded and executed by a processor, they implement the 100 steps of the wheat canopy geometric parameter extraction method disclosed in any of the foregoing embodiments, such as... Figure 1The steps are shown.

[0229] In summary, the present invention has the following advantages:

[0230] Flexible equipment and easy data collection: The data collection system 10 of this invention is compact and flexible, and can be easily mounted on various field walking platforms. No additional field targets are required during data collection, reducing the complexity of preliminary preparations and saving resources for subsequent point cloud matching, greatly improving collection efficiency and making the entire data collection process more convenient and efficient.

[0231] Precise and reliable trait extraction: The proposed method for extracting wheat canopy geometric parameters achieves 100% accuracy, stably and accurately acquiring wheat canopy information. This is of great significance for assessing wheat growth status, helping farmers to promptly identify growth anomalies, thus providing a basis for precision agriculture operations and contributing to improved wheat yield and quality.

[0232] The algorithms are highly adaptable and stable: the series of algorithms used in point cloud data processing and canopy information extraction are highly adaptable to complex field environments. They can effectively overcome problems such as ground undulations, crop shading, and noise interference. Even in the event of unexpected situations such as sensor errors, they can operate stably and ensure reliable data processing results.

[0233] Wide range of applications: This invention is not limited to wheat; its principles and methods can be extended to monitoring the growth of various crops. In scenarios such as corn and rice cultivation, orchard and tea garden management, and urban greening plant monitoring, this system can acquire three-dimensional plant information, providing innovative solutions for plant growth research and management in different fields.

[0234] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms fall within the scope of protection of the present invention.

Claims

1. A method for extracting geometric parameters of wheat canopy, characterized in that, A multimodal sensor integrated acquisition system is constructed. The acquisition system includes an image-stabilized acquisition device and multiple sensors. The multiple sensors include at least a camera, a tightly coupled lidar, and an inertial measurement unit (IMU). The image-stabilized acquisition device includes a three-axis imaging gimbal and a vibration damping and fixing device. The multiple sensors are mounted on the three-axis imaging gimbal. The acquisition system employs an ESO-LQR-based pitch axis disturbance rejection control strategy. The acquisition system is used to acquire point cloud data and image data of wheat. The image data is used to extract two-dimensional traits based on color index. The point cloud data and the image data are used to extract three-dimensional canopy structure distribution traits. The pitch axis disturbance rejection control strategy based on ESO-LQR includes the following steps: A pitch axis dynamic model is established and a disturbance analysis is performed. The pitch axis dynamic model includes lumped disturbances, which include low-frequency disturbances, high-frequency disturbances, and coupled interference. The lumped disturbance is extended to the system state, and a third-order extended state observer (ESO) is designed to generate disturbance estimates. Construct the state vector for the optimal controller LQR, design the cost function, and solve the optimal feedback matrix through the Riccati equation to generate the control quantity; The disturbance estimate is compensated into the control quantity to generate the final control input; The extraction of two-dimensional traits based on color indices using the image data further includes the following steps: canopy yellowing index extraction, comprising: Multispectral feature enhancement, including: The image data is converted into the LAB color space, the a channel is extracted and normalized, and the initial form of the canopy yellowing index CYI is defined using the statistical extreme values ​​of the a channel in the early stage of grouting and the statistical extreme values ​​of the a channel in the mature stage. The second super-green index ExG is calculated based on the image data, and an ExG image is generated; The Otsu threshold segmentation is applied to the ExG image to generate a first vegetation mask. A fixed threshold is set for the a channel to generate a second vegetation mask. The first vegetation mask and the second vegetation mask are merged to generate a third vegetation mask. Texture feature fusion, including: Texture features are extracted from the wheat ear region using the third vegetation mask, and contrast and correlation are calculated using the gray-level co-occurrence matrix. A multiple linear regression model is constructed by combining the canopy yellowing index CYI, the contrast ratio (Contrast), and the correlation (Correlation), and the degree of yellowing is predicted using the multiple linear regression model.

2. The method for extracting geometric parameters of wheat canopy according to claim 1, characterized in that, The adoption of an ESO-LQR-based pitch axis disturbance rejection control strategy for the acquisition system further includes the following steps: Angle initialization includes: manual angle initialization and automatic angle initialization, wherein: The manual angle initialization includes: controlling the pitch axis angle by sending a PWM signal or an S.Bus signal via a remote controller, and setting and locking the pitch axis and yaw axis angles based on the angle values ​​fed back by the inertial measurement unit (IMU); The automatic angle initialization includes: setting the target angle of the pitch axis, obtaining the current angle of the pitch axis, automatically adjusting the angle of the pitch axis by the acquisition system based on the angle difference between the current angle and the target angle, and entering the angle locking mode when the angle difference reaches a predetermined error range.

3. The method for extracting wheat canopy geometric parameters according to claim 1, characterized in that, The extraction of two-dimensional traits based on color index using the image data further includes the following steps: image data preprocessing, including: Brightness correction is performed by using homomorphic filtering to separate the illumination and reflection components of the image data. Noise cancellation is achieved by suppressing salt-and-pepper noise using anisotropic diffusion filtering. Image registration involves detecting the main direction of the wheat rows using Hough transform, calculating the tilt angle, and performing rotation correction.

4. The method for extracting wheat canopy geometric parameters according to claim 1, characterized in that, The extraction of two-dimensional features based on color indices using the image data further includes the following steps: green area index extraction, including: Multi-color space synergistic enhancement, introducing the LAB color space to enhance the first super green index I GAI Optimize and remove background in HSV space; Dynamic threshold segmentation and morphological optimization are performed using Otsu adaptive threshold segmentation and region filtering. The Green Area Index (GAI) is calculated by converting the mask coverage area and imaging resolution to determine the actual green area ratio.

5. The method for extracting wheat canopy geometric parameters according to claim 1, characterized in that, The extraction of three-dimensional canopy structure distribution characteristics using the point cloud data and the image data further includes the following step: canopy coverage calculation, including: Point cloud projection and rasterization, including: The point cloud data is preprocessed, the preprocessed point cloud data is projected onto a horizontal plane, and then divided into grids. Calculate the vegetation point ratio for each grid cell. If the number of point clouds in a grid cell is less than a first set value, mark the grid cell as a low-density area and trigger interpolation processing. Sliding window weighted optimization includes: A 3×3 grid is used as a sliding window, and a local coverage weighted average is performed on all grids within each sliding window. Remove outlier grid cells whose local coverage is greater than the first percentage value or less than the second percentage value; Global coverage calculation and verification, including: The average grid coverage of all sliding windows that have undergone the aforementioned sliding window weighted optimization is used to generate the global canopy coverage. The reliability was verified by performing a linear regression analysis between the global canopy coverage and the actual measured canopy coverage.

6. The method for extracting wheat canopy geometric parameters according to claim 1, characterized in that, The extraction of three-dimensional canopy structure distribution characteristics using the point cloud data and the image data further includes the following step: canopy flatness calculation, including: Point cloud segmentation and neighborhood construction: The point cloud data is divided into multiple cubic units, and the neighborhood radius of each cubic unit is selected according to the growth stage to generate a point cloud set. Covariance matrix and PCA analysis are performed. For each cubic cell's point cloud set, the centroid is calculated, and a covariance matrix is ​​constructed. Eigenvalue decomposition is then performed on the covariance matrix to generate eigenvalues ​​e1, e2, and e3 and their corresponding eigenvectors. ; Flatness index calculation and optimization: If the number of point clouds in a cubic cell is less than a second preset value, the cubic cell is marked as an invalid cell. The flatness index is defined after normalizing the eigenvalues ​​e1, e2, and e3. A constraint range is set for the flatness index CFI, and if it exceeds the constraint range, it is truncated. Spatial interpolation and visualization: For the invalid cells, inverse distance weighted interpolation is used, and the flatness index CFI is mapped according to the color gradient.

7. A system for extracting geometric parameters of wheat canopy, characterized in that, The method for extracting wheat canopy geometric parameters according to any one of claims 1 to 6 includes: A multimodal sensor integrated acquisition system, the acquisition system including an image stabilization acquisition device and multiple sensors, the multiple sensors including at least a camera, a tightly coupled lidar and an inertial measurement unit (IMU), the image stabilization acquisition device including a three-axis imaging gimbal and a vibration reduction and fixing device, the multiple sensors being mounted on the three-axis imaging gimbal; The control module is used to employ an ESO-LQR-based pitch axis disturbance rejection control strategy for the acquisition system. The extraction module is used to collect point cloud data and image data of wheat using the acquisition system, perform two-dimensional trait extraction based on color index using the image data, and perform three-dimensional canopy structure distribution trait extraction using the point cloud data and the image data. The steps of the pitch axis disturbance rejection control strategy based on ESO-LQR specifically include: A pitch axis dynamic model is established and a disturbance analysis is performed. The pitch axis dynamic model includes lumped disturbances, which include low-frequency disturbances, high-frequency disturbances, and coupled interference. The lumped disturbance is extended to the system state, and a third-order extended state observer (ESO) is designed to generate disturbance estimates. Construct the state vector for the optimal controller LQR, design the cost function, and solve the optimal feedback matrix through the Riccati equation to generate the control quantity; The disturbance estimate is compensated into the control quantity to generate the final control input.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the wheat canopy geometric parameter extraction method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs; when the computer programs are executed by a processor, they implement the wheat canopy geometric parameter extraction method as described in any one of claims 1 to 6.

Images