A multi-terrain all-weather plant height measurement device and plant height measurement method

Abstract

The present invention relates to the field of non-contact, automated measurement of crop structural information, and discloses a multi-terrain all-weather plant height measurement device and a plant height measurement method thereof, which utilizes a plant height estimation sensor based on binocular laser vision and a calibration plate structure with a gyroscope to perform image acquisition and sensor body parameter calibration, and performs plant height measurement based on the calibrated sensor; during calibration, the parameters are calibrated separately according to the ground observation station or the airborne case, and during plant height measurement, the plant height is calculated separately according to the ground observation station or the airborne case, and different methods are selected to determine the plant height according to the light intensity. Compared with the prior art, the present invention can work all-weather, and uses a binocular model to estimate the plant height of crops when the light is strong, and uses a structured light model to estimate the plant height of crops when the light is weak; and the present invention is suitable for planes and inclined planes, and is applicable to measurement in various terrain environments.

Description

Technical Field

[0001] The invention relates to the field of non-contact and automatic measurement of crop structural information, and in particular to a multi-terrain and all-weather plant height measurement device and a plant height measurement method thereof. Background Art

[0002] Plant height is the distance between the base of the plant and the top of the main diameter (growth point), and is a key indicator for judging the growth and growth period of the plant, as well as for predicting yield. Due to the large differences in measurement environments and the influence of various plant species, it is currently mainly measured manually. For example, Chinese invention patents CN117128874A, CN116718094A, CN111678409B, CN109099812A, etc. disclose different measuring rulers to assist surveyors in completing plant height measurement. In terms of automated measurement, Chinese invention patents CN112665539B and CN108592832B both use the ultrasonic ranging principle to measure the height of seedlings. However, ultrasonic technology is susceptible to environmental interference in large-scale measurements, and it is difficult to achieve real-time measurement of crops with large plant height growth ranges, such as real-time measurement of the entire growth cycle of crops such as corn and sugarcane. The development of visual technology provides technical support for high-precision real-time measurement of plant height. For example, Chinese patent CN115995010B discloses a plant height measurement method based on point cloud data. By collecting image sequences at different angles, the three-dimensional point clouds of the bottom surface and the plant are extracted respectively, and then the plant height of a single plant is extracted; Chinese patent CN117128874A discloses a plant height measurement method based on a depth camera. This method calculates the depth through two upper and lower depth cameras mounted on a tracked vehicle. Both of the above measurement methods work in a visible light environment, and it is difficult to achieve real-time measurement in a weak light environment. Chinese patent CN106871799B discloses a plant height measurement method based on structured light. This method solves the problem of plant height measurement on plains in a weak light environment, but does not consider plant height measurement in special terrains such as mountains.

[0003] Therefore, existing plant height measurement devices are generally susceptible to environmental interference, and it is difficult to achieve real-time measurement when light changes greatly. In addition, it is difficult to accurately measure the plant height of plants in areas with complex terrain. Summary of the invention

[0004] Purpose of the invention: In response to the problems pointed out in the background technology, the present invention discloses a multi-terrain all-weather plant height measurement device and a plant height measurement method thereof. For plains and mountainous terrains, all-weather plant height measurement is performed based on binocular multi-line laser vision technology, which is suitable for real-time and high-precision measurement under the needs of plant height measurement in ground observation stations and airborne systems.

[0005] Technical solution: The present invention discloses a multi-terrain all-weather plant height measurement device, including a plant height measurement sensor body and a calibration plate with a gyroscope, wherein the sensor body includes a sensor housing, a camera A, a camera B, a photosensitive module and a laser; the sensor housing is provided with a camera A opening, a camera B opening and a laser photosensitive module opening; the device also includes a power supply module, an input / output module and a processing controller, wherein the power supply module supplies power to each device, the camera A, the camera B, the photosensitive module and the laser are connected to the input / output module via a connecting line, and the input / output module is connected to the processing controller;

[0006] When the calibration plate calibrates the sensor parameters, it is placed in the common field of view of camera A and camera B. The sensor body collects images with and without the laser turned on, and the calibration plate calibrates the sensor body parameters.

[0007] Furthermore, the calibration board is an assembled calibration board, including four calibration board modules and a gyroscope; the four calibration board modules are connected by using plug-in columns and plug-in holes, and a checkerboard is arranged on the calibration board module, and the checkerboard has an even number of corner points in one direction and an odd number of corner points in one direction; the gyroscope is arranged in the calibration board module.

[0008] Furthermore, the photosensitive module collects light intensity and transmits it to the processing controller through a connecting line. The processing controller determines whether the light intensity is greater than a threshold value. If the light intensity is greater than the threshold value, the processing controller controls camera A and camera B to form a binocular measurement system. If the light intensity is less than or equal to the threshold value, the processing controller controls the laser, camera A and camera B to form a binocular structured light measurement system. When the binocular structured light measurement system is started, the controller controls to turn on the laser, and the laser emits a red line laser and a blue line laser. After a certain delay, the processing controller controls camera A and camera B to synchronously collect images with laser bars. After the image collection is completed, the laser is turned off.

[0009] The present invention also discloses a plant height measurement method based on a multi-terrain all-weather plant height measurement device, wherein a camera is arranged on a ground observation station, and the method comprises the following steps:

[0010] Step 1: Calibrate the sensor parameters under the flat or inclined measurement terrain;

[0011] Step 2: When the light intensity is greater than the threshold, the binocular measurement system is used to synchronously collect plant height measurement images; when the light intensity is less than or equal to the threshold, the binocular structured light measurement system is used to synchronously collect plant height measurement images;

[0012] Step 3: Perform image processing to obtain image features;

[0013] Step 4: Complete plant height measurement based on sensor parameters and binocular or structured light measurement models under flat and slope terrains.

[0014] The present invention also discloses a plant height measurement method of a multi-terrain all-weather plant height measurement device. When the camera is set for airborne measurement, a calibration plate with a gyroscope is set on the plane of the airborne tool, and the airborne tool is provided with a plant height measurement sensor body located above the plant to be measured through an airborne support rod, comprising the following steps:

[0015] Step 1: Calibrate the sensor parameters in airborne mode;

[0016] Step 2: When the light intensity is greater than the threshold, the binocular measurement system synchronously collects plant height measurement images; when the light intensity is less than or equal to the threshold, the binocular structured light measurement system synchronously collects plant height measurement images;

[0017] Step 3: Perform image processing to obtain image features;

[0018] Step 4: Complete plant height measurement based on the binocular or structured light measurement model in airborne mode.

[0019] Preferably, in step 1, when the ground is a plane, the sensor parameters are: the intrinsic parameter A1 of camera A and the intrinsic parameter A2 of camera B, the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems, the relationship parameter [R t] of the two cameras, and the extrinsic parameters [R1 t1] and [R2 t2] of the two camera coordinate systems relative to the bottom plane of the crop. The specific calibration process is as follows:

[0020] (1) Place the calibration plate in the common field of view of the two cameras, and collect images with the laser turned off and with the laser bar turned on. Collect images of the calibration plate in at least three positions, and one of the positions must coincide with the ground plane.

[0021] (2) Extract the coordinates of the corner points on the chessboard;

[0022] (3) Use Zhang Zhengyou's calibration method to calibrate A1, A2, [R t], [R1 t1] and [R2 t2];

[0023] (4) In each image, a certain number of points on the laser bar are sampled to calculate their three-dimensional coordinates in the camera coordinate system;

[0024] (5) The light plane equations a1 and a2 of the red line laser in the two camera coordinate systems and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems are solved by the least squares method.

[0025] Preferably, when the ground is a slope in step 1, the sensor parameters are: the intrinsic parameter A1 of camera A and the intrinsic parameter A2 of camera B, the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems, the relationship parameter [R t] of the two cameras, and the extrinsic parameters [R1t1] and [R2 t2] of the two camera coordinate systems relative to the slope of the crop bottom, and the angle θ between the slope and the horizontal plane. The specific calibration process is as follows:

[0026] (1) Place the calibration plate in the common field of view of the two cameras, and collect images with the laser turned on and the laser bar turned on, respectively. Collect images of the calibration plate in at least three positions, one of which is required to coincide with the inclined plane, and one of which is required to coincide with the horizontal plane of any point in the field of view on the inclined plane;

[0027] (2) Extract the coordinates of the corner points on the chessboard;

[0028] (3) Using Zhang Zhengyou's calibration method, calibrate A1, A2, [R t], solve the extrinsic parameters [R1 t1] and [R2 t2] of camera A and camera B according to the homography matrix in the pose coincident with the inclined plane, and solve the extrinsic parameters [R3 t3] and [R4 t4] of the two cameras A and B according to the homography matrix in the pose coincident with the horizontal plane;

[0029] (4) In each image, a certain number of points on the laser bar are sampled and their three-dimensional coordinates in the camera coordinate system are calculated;

[0030] (5) Solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems respectively by the least squares method;

[0031] (6) When the calibration plate coincides with the inclined plane, the plane equation of the calibration plate plane in the camera A coordinate system is a5. When the calibration plate coincides with the horizontal plane on the inclined plane, the plane equation of the calibration plate plane in the camera A coordinate system is a6. Calculate the angle cosθ between planes a5 and a6 using the following formula:

[0032]

[0033] Among them, a5 and a6 are calculated as follows:

[0034] α5=R1 -T α0,α6=R3 -T α0 (2)

[0035] Where a0 = [0, 0, 1] T .

[0036] Preferably, in step 1, the airborne sensor parameters are: the intrinsic parameters A1 of camera A and the intrinsic parameters A2 of camera B, the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems, the relationship parameter [R t] of the two cameras, and the extrinsic parameters [R5 t5] and [R6 t6] of the two camera coordinate systems relative to the plane where the gyroscope on the airborne tool is located, and the height h0 of the plane with the gyroscope on the airborne tool and the horizontal plane of the bottom of the airborne tool; the specific calibration process is as follows:

[0037] (1) The airborne tool is located on a horizontal surface. The calibration plate is placed in the common field of view of the two cameras. Images are collected with the laser turned off and with the laser bar turned on. Images of the calibration plate in at least three positions are collected. During calibration, one of the positions of the calibration plate is required to be on the horizontal plane where the airborne gyroscope is installed.

[0038] (2) Extract the coordinates of the corner points on the chessboard;

[0039] (3) Using Zhang Zhengyou’s calibration method, calibrate A1, A2, [R t], and the extrinsic parameters [R5 t5] and [R6 t6] of the two camera coordinate systems relative to the plane where the onboard gyroscope is located;

[0040] (4) In each image, a certain number of points on the laser bar are sampled and their three-dimensional coordinates in the camera coordinate system are calculated;

[0041] (5) Solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems respectively by the least squares method;

[0042] (6) Install the calibration plate with the gyroscope on the horizontal plane of the airborne vehicle and initialize it, and measure the height h0 between the plane where the gyroscope is located on the airborne vehicle and the horizontal plane of the bottom of the airborne vehicle.

[0043] Preferably, when the camera is set on the ground, the specific process of determining the plant height is as follows:

[0044] 1) When the ground is flat:

[0045] When the light intensity is less than or equal to the threshold, according to the set measurement conditions, the laser is turned on, the camera collects images with laser stripes, performs image processing, obtains the feature points of the laser stripes in the image, and calculates the three-dimensional coordinates of the feature points of lasers of different colors in the left and right camera coordinate systems according to the following formula:

[0046]

[0047] Where X clr =[x clr ,yclr ,z clr ],X clb ,X crr ,X crb is the three-dimensional coordinate vector of the laser feature point in the camera coordinate system, X clr The first digit of the subscript c represents the camera coordinate system, the second digit of the subscript l represents camera A, r represents camera B, the third digit of the subscript r represents the red line laser, and the third digit of the subscript b represents the blue line laser. In its homogeneous coordinate form, is the homogeneous form of the pixel coordinates of the laser feature point. The laser feature point in the camera coordinate system is transformed to the coordinate system of the crop bottom plane. The calculation method is as follows:

[0048]

[0049] The three-dimensional coordinates X of the feature point in the coordinate system of the crop bottom plane can be obtained. glr ,X glb ,X grr ,X grr , select the N values ​​with the largest z-direction value and take the average, which is the plant height measurement result h under plane terrain;

[0050] When the light intensity is greater than the threshold, according to the set measurement conditions, the dual cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the dual cameras. The coordinates in the camera coordinate system are transformed to the coordinate system of the crop bottom plane. The clustering method is used to divide the single plant area in the area, and then the plant height of the single plant in the field of view is obtained. The average plant height in the field of view is taken as the plant height measurement result h under the plane terrain;

[0051] 2) When the ground is inclined:

[0052] When the light intensity is less than or equal to the threshold, the laser feature point in the camera coordinate system is transformed to the coordinate system of the crop slope plane, and the N values ​​with the largest laser point depth are selected and averaged, which is recorded as z';

[0053] When the light intensity is greater than the threshold, the coordinates in the camera coordinate system are transformed to the coordinate system of the crop bottom slope, and the clustering method is used to divide the single plant area in the area, and then the plant height of the single plant on the bottom slope in the field of view is obtained, and the average plant height on the bottom slope in the field of view is taken, which is recorded as z';

[0054] The slope plant height is further calculated by the following formula:

[0055] h=z' / cos θ (5)

[0056] Preferably, when the camera is set to be airborne, the specific process of determining the plant height is:

[0057] When the light intensity is less than or equal to the threshold, according to the set measurement conditions, the laser is turned on, the camera collects images with laser stripes, performs image processing, obtains the characteristic points of the laser stripes in the image, calculates the three-dimensional coordinates of the characteristic points of lasers of different colors in the camera A and B coordinate systems, and transforms the three-dimensional coordinates of the laser characteristic points in the camera coordinate system to the three-dimensional coordinates of the coordinate system where the gyroscope is installed horizontally as follows:

[0058]

[0059] Select the N values ​​with the largest z-direction values ​​in the three-dimensional coordinate system of the gyroscope installation horizontal plane and take the average, recorded as h1. The equation of the horizontal plane where the gyroscope is located is transformed to the equation of the plane where the camera A is located:

[0060] α7=R5 -T α0 (7)

[0061] The rotation matrix between the current attitude of the gyroscope (gyroscope real-time plane) and the horizontal plane where the gyroscope is installed is R7. The real-time plane equation of the gyroscope real-time plane in the camera A coordinate system is:

[0062] α8=R5 -T R7α0 (8)

[0063] When measuring the motion of an airborne tool, the cosine value of the angle between the gyroscope real-time plane and the gyroscope installation horizontal plane is:

[0064]

[0065] The airborne plant height h is calculated as follows:

[0066]

[0067] When the light intensity is greater than or equal to the threshold, according to the set measurement conditions, the dual cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the dual cameras. The coordinates in the camera coordinate system are transformed to the coordinate system of the gyroscope calibration plane. The clustering method is used to divide the single plant area in the area, and then the height of the single plant in the field of view on the gyroscope calibration plane is obtained. The average plant height on the gyroscope calibration plane in the field of view is taken, recorded as h1, and the plant height h is solved by formula (10).

[0068] Beneficial effects:

[0069] Aiming at plain and mountainous terrains, the present invention proposes an all-weather plant height measurement device based on binocular multi-line laser vision technology, which is suitable for real-time high-precision measurement under the needs of plant height measurement in ground observation stations, airborne, etc.

[0070] When using sensors to measure plant height, the present invention also has an airborne or ground observation station measurement mode set according to the usage scenario, and for different measurement modes, sensor parameters, calibration methods and measurement methods are set for plane and mountainous terrains respectively. Therefore, the plant height measurement device of the present invention is not limited by the terrain of use.

[0071] The present invention sets the calibration plate to be assembled, which can adapt to different scene requirements. In addition, the assembled calibration plate is easy to install, and the entire calibration plate is split into different models, so that calibration plates of different sizes can be made according to on-site requirements, which is convenient to carry. A gyroscope is set on the calibration plate to facilitate accurate measurement during airborne measurement.

[0072] At the same time, the present invention uses a binocular structure to realize image acquisition under strong light conditions, and uses binocular structured light to realize image acquisition under weak light conditions. Real-time measurement of plant height can be achieved, and it is not affected by factors such as ambient light. In addition, the present invention can use a binocular stereo matching method under strong light conditions to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the two cameras, and use a clustering method to divide the single plant area in the area, and then obtain the average plant height in the field of view. In the case of weak light, the laser bar feature points in the image are used to calculate the three-dimensional coordinates of the laser feature points of different colors in the coordinate system of the two cameras, and then determine the plant height measurement results. In this way, the measurement accuracy will not be affected by light problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is the sensor structure diagram;

[0074] Figure 2 This is the structural diagram of the calibration board;

[0075] Figure 3 It is a work flow chart;

[0076] Figure 4 It is a schematic diagram of slope and flat topographic survey;

[0077] Figure 5 Schematic diagram of airborne plant height measurement.

[0078] Among them, sensor housing-101; camera A-102; power supply module-103; input and output module-104; connecting line-105; processing controller-106; photosensitive module-107; camera B-108; camera B opening-109; laser photosensitive module opening-110; red line laser-111; blue line laser-112; laser-113; camera A opening-114; 201-calibration board module 1; 207-gyroscope; 203-calibration board module 2; 204-plug hole; 205-calibration board module 3; 206-calibration board module 4; 207-chessboard; 208-plug column; plant to be measured-501, plant height measurement sensor-502, airborne support rod-503, calibration board module with gyroscope-504, airborne tool-505. DETAILED DESCRIPTION

[0079] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0080] The invention discloses a multi-terrain all-weather plant height measuring device and a plant height measuring method thereof. The plant height measuring device specifically comprises the following structures: a plant height measuring sensor body and a calibration device with a gyroscope.

[0081] like Figure 1 As shown, the sensor body includes a sensor housing 101; a camera A 102; a power supply module 103; an input-output module 104; a connecting line 105; a processing controller 106; a photosensitive module 107; a camera B 108; a camera B 109 opening; a laser photosensitive module opening 110; a red line laser 111; a blue line laser 112; a laser 113; and a camera A 114 opening. The sensor housing 101 is provided with a camera A opening 114, a camera B opening 109, and a laser photosensitive module opening 110; the power supply module 103 supplies power to each device, and the camera A 102, the camera B 108, the photosensitive module 107, and the laser 113 are all connected to the input-output module 104 through connecting lines, and the input-output module 104 is connected to the processing controller 106.

[0082] After the parameters of the sensor are calibrated, it works according to the set measurement mode in the measurement state. The measurement mode is divided into timed and real-time. In the timed mode, the system clock is used to enter the measurement process at the set time. In the real-time mode, the system hardware capability is used to perform real-time measurement at the system set frequency.

[0083] The calibration device with gyroscope for this sensor is as follows Figure 2As shown, the device includes calibration board module 1-201; gyroscope 202; calibration board module 2-203; plug hole 204; calibration board module 3-205; calibration board module 4-206; chessboard 207; plug column 208. When in use, calibration board module 1, calibration board module 2, calibration board module 3 and calibration board module 4 are connected by plugging the plug column 208 with the plug hole 204 to form a calibration board with a chessboard 207 on the upper plane. The typical chessboard uses an even number of corner points in one direction and an odd number of corner points in one direction for easy distinction. The calibration board is used to calibrate sensor parameters. The method for calibrating plant height sensor parameters is divided into a ground observation station calibration method and an airborne calibration method.

[0084] The sensor plant height measurement process is as follows Figure 3 As shown. The system collects light intensity through the photosensitive module 107, and the collected light intensity is transmitted to the processing controller 106 through the connecting line 105. The processing controller 106 determines whether the light intensity is greater than the threshold according to the software program. If the light intensity is greater than the threshold, the processing controller 106 controls the camera A102 and the camera B108 to form a binocular measurement system, synchronously collects images, and the collected images measure the plant height according to the binocular plant height measurement model. If the light intensity is less than or equal to the threshold, the processing controller 106 controls the laser 113, the camera A102 and the camera B108 to form a binocular structured light measurement system. First, the processing controller 106 controls the laser 113 to turn on. The laser 113 emits red line lasers 111 and blue line lasers 112 of different colors. After the laser is turned on, after a certain delay, the processing controller 106 controls the camera A102 and the camera B108 to synchronously collect images with laser strips. After the image acquisition is completed, the laser is turned off. The collected image measures the plant height according to the structured light plant height measurement model, and the measured plant height value is output for use in the back-end link.

[0085] The above sensor parameter calibration method is divided into ground observation station calibration method and airborne calibration method:

[0086] 1. When used in ground observation stations, there are two calibration methods: plane terrain and slope terrain:

[0087] For flat terrain, the sensor parameters include: the intrinsic parameters A1 of camera A and A2 of camera B, the light plane equations a1 and a2 of the red laser line in the two camera coordinate systems, the light plane equations a3 and a4 of the blue laser line in the two camera coordinate systems, the relationship parameter [R t] of the two cameras, and the extrinsic parameters [R1 t1] and [R2 t2] of the two camera coordinate systems relative to the bottom plane of the crop. The calibration method of the above parameters is as follows:

[0088] (1) Use the assembled calibration plate and place it in the common field of view of the two cameras. Collect images with the laser turned off and with the laser bar turned on. Collect images of the calibration plate in at least three positions, and one of the positions is required to coincide with the ground plane; (2) Extract the coordinates of the corner points on the chessboard; (3) Use Zhang Zhengyou's calibration method to calibrate A1, A2, [R t], [R1 t1] and [R2 t2]; (4) Sample a certain number of points on the laser bar in each image and calculate their three-dimensional coordinates in the camera coordinate system; (5) Use the least squares method to solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems.

[0089] For slope terrain, the sensor parameters include: the intrinsic parameters A1 of camera A and A2 of camera B, the light plane equations a1 and a2 of the red laser line in the two camera coordinate systems, the light plane equations a3 and a4 of the blue laser line in the two camera coordinate systems, the relationship parameter [R t] of the two cameras, and the extrinsic parameters [R1 t1] and [R2 t2] of the two camera coordinate systems relative to the slope of the crop bottom, and the angle θ between the slope and the horizontal plane. The calibration method of the above parameters is as follows:

[0090] (1) Use the assembled calibration plate and place it in the common field of view of the two cameras. Collect images with the laser turned off and with the laser bar turned on. Collect images of the calibration plate in at least three positions, one of which is required to coincide with the inclined plane and one with the horizontal plane on the inclined plane. (2) Extract the coordinates of the corner points on the chessboard. (3) Use Zhang Zhengyou's calibration method to calibrate A1, A2, and [R t]. According to the homography matrix in the position coinciding with the inclined plane, solve [R1 t1] and [R2t2]. According to the homography matrix in the position coinciding with the horizontal plane, solve the relationship parameters of the two cameras [R3 t3] and [R4 t5]. t4]; (4) In each image, a certain number of points on the laser bar are sampled to calculate their three-dimensional coordinates in the camera coordinate system; (5) The light plane equations a1 and a2 of the red line laser in the two camera coordinate systems are solved by the least squares method, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems are solved respectively; (6) When the calibration plate coincides with the inclined plane, the plane equation of the calibration plate plane in the camera A coordinate system is calculated to be a5; when the calibration plate coincides with the horizontal plane on the inclined plane, the plane equation of the calibration plate plane in the camera A coordinate system is calculated to be a6; Calculate the angle cosθ between planes a5 and a6, and the calculation formula is as follows:

[0091]

[0092] Where a5 and a6 are calculated as follows:

[0093] α5=R1 -T α0,α6=R3 -T α0 (2)

[0094] where a0 = [0,0,1] T .

[0095] 2. For airborne measurements, see Figure 5 , a calibration plate 504 with a gyroscope is set on an airborne tool 505, and the airborne tool 505 is provided with a plant height measurement sensor body 502 located above the plant to be measured 501 through an airborne support rod 503. The sensor parameters include: the intrinsic parameter A1 of camera A and the intrinsic parameter A2 of camera B, the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems, the relationship parameter [R t] of the two cameras, and the external parameters [R5 t5] and [R6t6] of the two camera coordinate systems relative to the plane where the airborne gyroscope is located, and the height h0 of the plane with the gyroscope on the airborne tool and the horizontal plane of the bottom of the tool. The calibration method of the above parameters is as follows:

[0096] (1) Use the assembled calibration board and place it in the common field of view of the two cameras. Collect images with the laser turned off and with the laser bar turned on. Collect images of the calibration board in at least three positions, one of which is required to be on the horizontal plane where the gyroscope of the airborne tool is installed; (2) Extract the coordinates of the corner points on the chessboard; (3) Use Zhang Zhengyou's calibration method to calibrate A1, A2, [R t], and the external parameters [R5 t6] and [R6 t7] of the two camera coordinate systems relative to the plane where the gyroscope on the airborne tool is located. t6]; (4) Sample a certain number of points on the laser bar in each image and calculate their three-dimensional coordinates in the camera coordinate system; (5) Solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems respectively by the least squares method; (6) Install the calibration plate with the gyroscope on the horizontal plane of the airborne tool. After initialization, measure the height h0 between the plane with the gyroscope and the bottom horizontal plane of the airborne tool.

[0097] For the above calibrated sensors, the plant height measurement methods are divided into ground observation station measurement method and airborne measurement method:

[0098] 1. In the ground observation station measurement method, in flat terrain, when the light is weak (light intensity is less than or equal to the threshold), the structured light model is used to complete the plant height measurement. According to the set measurement conditions, turn on the laser, the camera collects images with laser stripes, performs image processing, obtains the laser stripe feature points in the image, and calculates the three-dimensional coordinates of the different color laser feature points in the left and right camera coordinate systems according to the following formula:

[0099]

[0100] Where X clr =[x clr ,y clr ,z clr ],X clb ,X crr ,X crb is the three-dimensional coordinate vector (X clr The first subscript c represents the camera coordinate system, the second subscript l represents the left camera 102, r represents the right camera 108, the third subscript r represents red, and the third subscript b represents blue). In its homogeneous coordinate form, is the homogeneous form of the pixel coordinates of the laser feature point. The laser feature point in the camera coordinate system is transformed to the coordinate system of the crop bottom plane. The calculation method is as follows:

[0101]

[0102] The three-dimensional coordinates X of the feature point in the coordinate system of the crop bottom plane can be obtained. glr ,X glb ,X grr ,X grr , select the N values ​​with the largest z-direction values ​​and take the average, which is the structured light plant height measurement result h under plane terrain.

[0103] In the ground observation station measurement method, under flat terrain, when the light intensity is strong (light intensity is greater than the threshold), the binocular model is used to complete the plant height measurement according to the set measurement conditions. The dual cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the dual cameras. Referring to the above formula (4), the coordinates in the camera coordinate system are transformed to the coordinate system of the crop bottom plane, and the clustering method is used to divide the single plant area in the area, and then the plant height of the single plant in the field of view is obtained. The average plant height in the field of view is taken as the binocular height measurement result of the plant.

[0104] In the ground observation station measurement method, under the slope terrain, when the light is weak (light intensity is less than or equal to the threshold), the structured light model is used to complete the plant height measurement. According to the set measurement conditions, the laser is turned on, the camera collects the image with laser stripes, and the image processing is performed to obtain the laser stripe feature points in the image. According to formula (3), the three-dimensional coordinates of the laser feature points of different colors in the left and right camera coordinate systems are calculated. According to formula (4), the laser feature points in the camera coordinate system are transformed to the coordinate system where the crop slope plane is located, and the N values ​​with the largest laser point depth are selected and averaged, recorded as z', as follows: Figure 4 As shown, the slope plant height is further calculated by the following formula:

[0105] h=z' / cosθ (5)

[0106] In the ground observation station measurement method, under the slope terrain, when the light intensity is strong (the light intensity is greater than the threshold), the binocular model is used to complete the plant height measurement according to the set measurement conditions. The dual cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the dual cameras. Similarly, refer to the above formula (4) to transform the coordinates in the camera coordinate system to the coordinate system of the bottom slope of the crop. The clustering method is used to divide the single plant area in the area, and then the plant height of the single plant on the bottom slope in the field of view is obtained. The average plant height on the bottom slope in the field of view is taken, recorded as z', and the plant height h is solved by formula (5).

[0107] 2. In the airborne plant height measurement method, no distinction is made between plane and slope terrain. When the light is weak (light intensity is less than or equal to the threshold), the structured light model is used to complete the plant height measurement. According to the set measurement conditions, the laser is turned on, and the camera collects images with laser stripes. Image processing is performed to obtain the feature points of the laser stripes in the image. The three-dimensional coordinates of the feature points of different color lasers in the camera A and B coordinate systems are calculated according to formula (3). The three-dimensional coordinates of the laser feature points in the camera coordinate system are transformed to the three-dimensional coordinates of the coordinate system where the airborne gyroscope is installed horizontally as follows:

[0108]

[0109] Select the N largest values ​​in the z direction in the horizontal coordinate system where the gyroscope is installed and take the average, which is recorded as h1. The equation of the horizontal plane where the gyroscope is located is transformed to the equation of the plane where the camera is located:

[0110] α7=R5 -T α0 (7)

[0111] The rotation matrix between the current attitude of the gyroscope (gyroscope real-time plane) and the horizontal plane where the gyroscope is installed is R7. The real-time plane equation of the gyroscope real-time plane in the camera A coordinate system is:

[0112] α8=R5 -T R7α0 (8)

[0113] When measuring the motion of an airborne tool, the real-time equivalent plane of the airborne tool (the real-time plane of the gyroscope) may be tilted. The cosine value of the angle between the real-time plane of the gyroscope and the horizontal plane where the gyroscope is installed is:

[0114]

[0115] The airborne plant height h is calculated as follows:

[0116]

[0117] In the airborne plant height measurement method, plane and slope terrain are not distinguished. When the light intensity is strong (light intensity is greater than the threshold), the binocular model is used to complete the plant height measurement according to the set measurement conditions. The dual cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the dual cameras. Similarly, referring to the above formula (4), the coordinates in the camera coordinate system are transformed to the coordinate system where the gyroscope calibration plane is located. The clustering method is used to divide the single plant area in the area, and then the height of the single plant in the field of view on the gyroscope calibration plane is obtained. The average plant height on the gyroscope calibration plane in the field of view is taken, recorded as h1, and the plant height h is solved by formula (10).

[0118] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A plant height measurement method, using a multi-terrain all-weather plant height measurement device, characterized in that: The multi-terrain all-weather plant height measurement device comprises a plant height measurement sensor body and a calibration plate with a gyroscope, wherein the sensor body comprises a sensor housing (101), a camera A (102), a camera B (108), a photosensitive module (107) and a laser (113); the sensor housing (101) is provided with a camera A opening (114), a camera B opening (109) and a laser photosensitive module opening (110); the device also comprises a power supply module (103), an input / output module (104) and a processing controller (106); the power supply module (103) supplies power to each device, the camera A (102), the camera B (108), the photosensitive module (107) and the laser (113) are all connected to the input / output module (104) via a connecting line, and the input / output module (104) is connected to the processing controller (106); When the calibration plate calibrates the parameters of the sensor body, it is placed in the common field of view of the camera A (102) and the camera B (108). The sensor body collects images with and without the laser (113) turned on, and the calibration plate calibrates the parameters of the sensor body. The calibration board with the gyroscope is an assembled calibration board, comprising four calibration board modules and a gyroscope; the four calibration board modules are connected by plugging posts and plugging holes; a chessboard is arranged on the calibration board module, and the chessboard has an even number of corner points in one direction and an odd number of corner points in one direction; the gyroscope is arranged in the calibration board module; The photosensitive module (107) collects light intensity and transmits it to the processing controller (106) through a connecting line. The processing controller (106) determines whether the light intensity is greater than a threshold value. If the light intensity is greater than the threshold value, the processing controller (106) controls the camera A (102) and the camera B (102) to form a binocular measurement system. If the light intensity is less than or equal to the threshold value, the processing controller (106) controls the laser (113), the camera A (102) and the camera B (102) to form a binocular structured light measurement system. When the binocular structured light measurement system is started, the controller (106) controls the laser (113) to turn on the laser (113), and the laser (113) emits a red line laser and a blue line laser. After a certain delay, the processing controller (106) controls the camera A (102) and the camera B (102) to synchronously collect images with laser strips. After the image collection is completed, the laser is turned off. When two cameras are set up on a ground observation station, the following steps are included: Step 1: Calibrate the sensor body parameters under the flat or inclined measurement terrain; When the ground is flat, the sensor body parameters are: the intrinsic parameters A1 of camera A and A2 of camera B, the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems, the relationship parameters [R t] of the two cameras, and the extrinsic parameters [R1 t1] and [R2 t2] of the two camera coordinate systems relative to the bottom plane of the crop. The specific calibration process is as follows: (1) Place the calibration plate in the common field of view of the two cameras, and collect images with the laser turned off and with the laser bar turned on. Collect images of the calibration plate in at least three positions, and one of the positions must coincide with the ground plane. (2) Extract the coordinates of the corner points on the chessboard; (3) Use Zhang Zhengyou's calibration method to calibrate A1, A2, [R t], [R1 t1] and [R2 t2]; (4) In each image, a certain number of points on the laser bar are sampled to calculate their three-dimensional coordinates in the camera coordinate system; (5) Solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems respectively by the least squares method; When the ground is an inclined plane, the sensor body parameters are: the intrinsic parameters A1 of camera A and A2 of camera B, the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems, the relationship parameters [R t] of the two cameras, and the extrinsic parameters [R1 t1] and [R2 t2] of the two camera coordinate systems relative to the slope of the crop bottom, and the angle θ between the slope and the horizontal plane. The specific calibration process is as follows: (1) Place the calibration plate in the common field of view of the two cameras, and collect images with the laser turned on and the laser bar turned on, respectively. Collect images of the calibration plate in at least three positions, one of which is required to coincide with the inclined plane, and one of which is required to coincide with the horizontal plane of any point in the field of view on the inclined plane; (2) Extract the coordinates of the corner points on the chessboard; (3) Using Zhang Zhengyou's calibration method, calibrate A1, A2, [R t], solve the extrinsic parameters [R1 t1] and [R2 t2] of camera A and camera B according to the homography matrix in the pose coincident with the inclined plane, and solve the extrinsic parameters [R3 t3] and [R4 t4] of camera A and camera B according to the homography matrix in the pose coincident with the horizontal plane; (4) In each image, a certain number of points on the laser bar are sampled and their three-dimensional coordinates in the camera coordinate system are calculated; (5) Solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems respectively by the least squares method; (6) When the calibration plate coincides with the inclined plane, the plane equation of the calibration plate plane in the camera A coordinate system is a5. When the calibration plate coincides with the horizontal plane on the inclined plane, the plane equation of the calibration plate plane in the camera A coordinate system is a6. Calculate the angle cosθ between planes a5 and a6 using the following formula: Among them, a5 and a6 are calculated as follows: <h2 style=";text-align:left;direction:ltr">a5=R1<h2 style=";text-align:left;direction:ltr"> -T <h2 style=";text-align:left;direction:ltr"> a0, a6 = R3<h2 style=";text-align:left;direction:ltr"> -T <h2 style=";text-align:left;direction:ltr"> a0 (2) Where a0 = [0, 0, 1] T ; Step 2: When the light intensity is greater than the threshold, the binocular measurement system is used to synchronously collect plant height measurement images; when the light intensity is less than or equal to the threshold, the binocular structured light measurement system is used to synchronously collect plant height measurement images; Step 3: Perform image processing to obtain image features; Step 4: Complete plant height measurement based on the sensor body parameters under flat and inclined terrain and the binocular measurement model or binocular structured light measurement model; When two cameras are set for airborne measurement, a calibration plate with a gyroscope is set on the plane of the airborne tool, and the airborne tool is provided with a plant height measurement sensor body located above the plant to be measured through an airborne support rod, including the following steps: Step 1: Calibrate the sensor body parameters in airborne mode; When the sensor is onboard, the parameters are: the intrinsic parameters A1 of camera A and A2 of camera B, the light plane equations a1 and a2 of the red laser line in the two camera coordinate systems, the light plane equations a3 and a4 of the blue laser line in the two camera coordinate systems, the relationship parameters [R t] of the two cameras, and the extrinsic parameters [R5 t5] and [R6 t6] of the two camera coordinate systems relative to the calibration plate plane on the plane where the gyroscope on the airborne tool is located, and the height h0 of the gyroscope plane on the airborne tool and the horizontal plane of the bottom of the airborne tool; the specific calibration process is as follows: (1) The airborne tool is located on a horizontal surface. The calibration plate is placed in the common field of view of the two cameras. Images are collected with the laser turned off and with the laser bar turned on. Images of the calibration plate in at least three positions are collected. During calibration, one of the positions of the calibration plate is required to be on the horizontal plane where the airborne gyroscope is installed. (2) Extract the coordinates of the corner points on the chessboard; (3) Using Zhang Zhengyou's calibration method, calibrate A1, A2, [R t], and the extrinsic parameters [R5 t5] and [R6 t6] of the two camera coordinate systems relative to the calibration plate on the plane where the onboard gyroscope is located; (4) In each image, a certain number of points on the laser bar are sampled and their three-dimensional coordinates in the camera coordinate system are calculated; (5) Solve the light plane equations a1 and a2 of the red line laser in the two camera coordinate systems, and the light plane equations a3 and a4 of the blue line laser in the two camera coordinate systems respectively by the least squares method; (6) Install the calibration plate with the gyroscope on the horizontal plane of the airborne tool and initialize it, and measure the height h0 between the plane where the gyroscope is located on the airborne tool and the horizontal plane of the bottom of the airborne tool; Step 2: When the light intensity is greater than the threshold, the binocular measurement system synchronously collects plant height measurement images; when the light intensity is less than or equal to the threshold, the binocular structured light measurement system synchronously collects plant height measurement images; Step 3: Perform image processing to obtain image features; Step 4: Complete plant height measurement based on the binocular measurement model or binocular structured light measurement model in airborne mode.

2. The plant height measurement method according to claim 1, characterized in that: When two cameras are set on the ground, the specific process of determining plant height is as follows: 1) When the ground is flat: When the light intensity is less than or equal to the threshold, according to the set measurement conditions, the laser is turned on, and the two cameras collect images with laser stripes. Image processing is performed to obtain the feature points of the laser stripes in the image, and the three-dimensional coordinates of the feature points of lasers of different colors in the camera coordinate system are calculated according to the following formula: Where X clr =[x clr ,y clr ,z clr ],X clb ,X crr ,X crb is the three-dimensional coordinate vector of the laser feature point in the camera coordinate system, X clr The first digit of the subscript c represents the camera coordinate system, the second digit of the subscript l represents camera A, r represents camera B, the third digit of the subscript r represents the red line laser, and the third digit of the subscript b represents the blue line laser. In its homogeneous coordinate form, is the homogeneous form of the pixel coordinates of the laser feature point. The laser feature point in the camera coordinate system is transformed to the coordinate system of the crop bottom plane. The calculation method is as follows: The three-dimensional coordinates X of the feature point in the coordinate system of the crop bottom plane can be obtained glr ,X glb ,X grr ,X grr , X glr The first subscript g represents the coordinate system of the crop bottom plane. The N values ​​with the largest z-direction values ​​are selected and averaged to obtain the plant height measurement result h under flat terrain. When the light intensity is greater than the threshold, according to the set measurement conditions, the two cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the two cameras. The coordinates in the camera coordinate system are transformed to the coordinate system of the crop bottom plane. The clustering method is used to divide the single plant area in the area, and then the plant height of the single plant in the field of view is obtained. The average plant height in the field of view is taken as the plant height measurement result h under the plane terrain; 2) When the ground is inclined: When the light intensity is less than or equal to the threshold, the laser feature point in the camera coordinate system is transformed to the coordinate system of the bottom slope of the crop, and the N values ​​with the largest laser point depth are selected and averaged, which is recorded as z'; When the light intensity is greater than the threshold, the coordinates in the camera coordinate system are transformed to the coordinate system of the crop bottom slope, and the clustering method is used to divide the single plant area in the area, and then the plant height of the single plant on the crop bottom slope in the field of view is obtained, and the average plant height on the crop bottom slope in the field of view is taken, which is recorded as z'; The slope plant height is further calculated by the following formula: h=z' / cosθ (5).

3. The plant height measurement method according to claim 1, characterized in that: The specific process of determining plant height when two cameras are set to be airborne is as follows: When the light intensity is less than or equal to the threshold, according to the set measurement conditions, the laser is turned on, the camera collects images with laser stripes, performs image processing, obtains the characteristic points of the laser stripes in the image, calculates the three-dimensional coordinates of the characteristic points of lasers of different colors in the camera coordinate system, and transforms the three-dimensional coordinates of the laser characteristic points in the camera coordinate system to the three-dimensional coordinates of the coordinate system of the calibration plate on the horizontal plane where the gyroscope is installed as follows: Select the N values ​​with the largest z-direction values ​​in the three-dimensional coordinate system of the calibration plate on the horizontal plane where the gyroscope is installed and take the average, recorded as h1. The equation of the horizontal plane where the gyroscope is located is transformed to the plane equation where the camera A is located: <h2 style=";text-align:left;direction:ltr">a7=R5<h2 style=";text-align:left;direction:ltr"> -T <h2 style=";text-align:left;direction:ltr"> a0 (7) Where a0 = [0, 0, 1] T ; The rotation matrix between the current plane fed back by the gyroscope in real time and the horizontal plane on which the gyroscope is installed is R7. The current plane equation of the current plane fed back by the gyroscope in real time in the camera A coordinate system is: a8=R5 -T R7a0 (8) When measuring the motion of an airborne tool, the cosine value of the angle between the real-time current plane of the gyroscope and the horizontal plane on which the gyroscope is installed is: The airborne plant height h is calculated as follows: When the light intensity is greater than the threshold, according to the set measurement conditions, the two cameras collect images of the area to be measured, and the stereo matching method is used to obtain the three-dimensional coordinate information of the camera coordinate system in the overlapping area of ​​the two cameras. The coordinates in the camera coordinate system are transformed to the coordinate system of the calibration plate on the horizontal plane where the gyroscope is installed. The clustering method is used to divide the single plant area in the area, and then the height of the single plant in the field of view on the gyroscope calibration plane is obtained. The average plant height on the gyroscope calibration plane in the field of view is taken, recorded as h1, and the plant height h is solved by formula (10).

Images