Fruit tree vibration harvesting trunk parameter automatic measurement method based on depth camera

Abstract

The invention provides a fruit tree vibration harvesting trunk parameter automatic measurement method based on a depth camera. The method comprises a coordinate system establishment step, a coordinate system transformation step, a trunk depth image acquisition step, a trunk point cloud extraction step and a trunk parameter measurement step. According to the method, the mapping relation between the depth image and the three-dimensional point cloud is established through coordinate system transformation, the tree trunk point cloud with the given height is extracted through depth image segmentation, the tree trunk model is constructed through cylinder fitting, key parameters such as the position, the dip angle and the diameter at breast height of the tree trunk are measured, low-cost and high-precision automatic measurement of the tree trunk parameters is achieved, and the method is suitable for large-scale popularization and application. Powerful data support is provided for working parameter setting of fruit tree vibration harvesting.

Description

Technical Field

[0001] The invention relates to the technical field of fruit tree harvesting, in particular to fruit tree trunk vibration harvesting, and specifically to an automatic measurement method for fruit tree vibration harvesting trunk parameters based on a depth camera. Background Art

[0002] Trunk vibration harvesting is an important method of mechanized harvesting of fruit trees. Parameters such as the position, inclination and breast diameter of the trunk determine the working parameters of the clamping device, such as displacement, rotation angle and vibration time. Therefore, rapid and accurate measurement of trunk parameters is of great significance to improving the efficiency of mechanized harvesting of fruit trees.

[0003] At present, there are two main methods for automatic measurement of tree trunk parameters:

[0004] (1) LiDAR measurement method. LiDAR collects three-dimensional point cloud data on the surface of tree trunks by actively emitting laser beams, and then uses circle fitting or cylinder fitting technology to build a trunk model and extract trunk parameters. However, the structure of LiDAR is relatively complex, which makes its manufacturing cost high. In addition, the field of view and resolution of LiDAR in the vertical direction are relatively low, resulting in an insufficient number of collection points in the vertical direction of the trunk, which affects the measurement accuracy of the trunk parameters and makes the measurement results inaccurate.

[0005] (2) RGB camera measurement method. The RGB camera captures the visible light information reflected by the tree trunk to generate an RGB image, and then uses image processing technology to segment the tree trunk to extract the trunk parameters. However, RGB images are easily affected by the complex lighting environment of the orchard, and changes in the surface texture of the tree trunk will also interfere with it, resulting in inaccurate trunk segmentation. In addition, RGB images lack depth information, which makes it difficult to achieve accurate results when measuring tree trunk parameters.

[0006] Depth cameras use advanced technologies such as binocular, structured light and ToF to obtain distance information of objects, and then accurately calculate three-dimensional coordinates. Depth cameras have many advantages. They have low manufacturing costs, are relatively affordable, are less affected by light, and can provide high-resolution depth images, which can accurately capture the detailed features of the trunk surface. The present invention uses a depth camera to measure trunk parameters, which has the dual advantages of low cost and high precision. It can effectively overcome the shortcomings of traditional measurement methods and provide a more reliable and efficient solution for the automatic measurement of trunk parameters. Summary of the invention

[0007] The purpose of the present invention is to provide a method for automatically measuring trunk parameters based on a depth camera for the problem of automatically measuring trunk parameters of fruit trees during vibration harvesting. The method first establishes a mapping relationship between a depth image and a three-dimensional point cloud through coordinate system transformation, then segments a trunk point cloud of a given height from the trunk depth image, and finally establishes a cylindrical model of the trunk point cloud, thereby measuring the position, inclination and breast diameter parameters of the trunk.

[0008] The technical solution of the present invention is:

[0009] The present invention provides a method for automatically measuring trunk parameters of fruit trees during vibration harvesting based on a depth camera, the method comprising the following steps:

[0010] S1. Coordinate system establishment step: Establishing the depth image coordinate system O i -uv, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -xyz. Depth image coordinate system O i -uv, origin O i Located in the upper left corner of the image, u is the pixel column number, and v is the pixel row number. Camera coordinate system O s -x′y′z′, the x′ axis points to the right, the y′ axis points vertically downward, and the z′ axis points to the trunk to be measured. w -xyz, origin O w Yes i The ground projection of the x-axis points to the right, that is, the horizontal direction, the y-axis points to the trunk to be measured, that is, the depth direction, and the z-axis points vertically upward, that is, the height direction;

[0011] S2, coordinate system transformation step: obtain the depth image coordinate system O i -uv and world coordinate system O w -The transformation relationship between xyz;

[0012] S3, tree trunk depth image acquisition step: using a depth camera to acquire a depth image of the tree trunk to be tested;

[0013] S4, tree trunk point cloud extraction step: extract a given height z from the depth image t Point cloud of tree trunks below;

[0014] S5. Trunk parameter measurement steps: measure the given height z t The trunk position coordinates (x t ,y t ), trunk inclination angle θ t and trunk diameter at breast height d t .

[0015] Furthermore, the S2 is specifically:

[0016] S21, depth image coordinate system O i -uv and depth camera coordinate system O s -x′y′z′ transformation relationship is:

[0017]

[0018] Among them, c x′ and c y′ are the principal point coordinates of the camera in the x′ and y′ directions, f x′ and f y′ is the focal length of the camera in the x′ and y′ directions, Δd is the camera depth scale factor, and I(u, v) is the grayscale value of the pixel with coordinates (u, v);

[0019] S22, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -The transformation relationship of xyz is:

[0020]

[0021] Where H is the height of the camera from the ground;

[0022] S23, depth image coordinate system O i -uv and world coordinate system O w -The transformation relationship of xyz is:

[0023]

[0024] The inverse transform is:

[0025]

[0026] Furthermore, the S4 is specifically:

[0027] S41, according to the set minimum trunk depth y min and the maximum value y max , find the minimum gray value I on the depth image min and the maximum value I max , mark the pixels falling into this gray value range as foreground (white), and the rest of the pixels as background (black);

[0028] S42, dividing the foreground pixels into a number of connected domains using a connected domain analysis algorithm, retaining the connected domain with the largest area including the tree trunk, and removing the remaining connected domains;

[0029] S43, tree trunk extraction: calculate the world coordinates of the remaining white pixels, extract the height in z t The tree trunk point cloud within the range of ±Δz, where Δz is the height margin, usually 8-12mm.

[0030] Furthermore, the S5 is specifically:

[0031] S51. Use the random sampling and consensus (RANSAC) algorithm to fit the cylinder according to the tree trunk point cloud and obtain the center coordinates of the cylinder (x c ,y c , z c ), direction vector (x d ,y d ,z d ) and radius r;

[0032] S52. According to the cylinder parameters, the trunk parameters are obtained as follows:

[0033]

[0034] Beneficial effects of the present invention:

[0035] The method for automatic measurement of trunk parameters of fruit trees vibrating harvesting based on a depth camera of the present invention establishes a mapping relationship between a depth image and a three-dimensional point cloud by coordinate system transformation, extracts a trunk point cloud of a given height by depth image segmentation, constructs a trunk model by cylindrical fitting, and measures key parameters such as the position, inclination and breast diameter of the trunk; it provides a low-cost and high-precision solution for automatic measurement of trunk parameters in complex orchard environments, thereby providing strong data support for the setting of working parameters for mechanized harvesting of fruit trees.

[0036] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0038] Figure 1 The flowchart of automatic measurement of trunk parameters of the present invention is shown.

[0039] Figure 2 A schematic diagram of a depth image coordinate system, a depth camera coordinate system and a world coordinate system in an embodiment of the present invention is shown.

[0040] Figure 3 A schematic diagram of the tree trunk extraction result in an embodiment of the present invention is shown.

[0041] Figure 4 A schematic diagram of tree trunk cylinder fitting in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0043] like Figure 1 As shown, a method for automatically measuring trunk parameters of fruit trees for vibration harvesting based on a depth camera, the method comprising the following steps:

[0044] S1. Coordinate system establishment step: Establishing the depth image coordinate system O i -uv, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -xyz. Depth image coordinate system O i -uv, origin O i Located in the upper left corner of the image, u is the pixel column number, and v is the pixel row number. Camera coordinate system O s -x′y′z′, the x′ axis points to the right, the y′ axis points vertically downward, and the z′ axis points to the trunk to be measured. w -xyz, origin O w Yes i The ground projection of the x-axis points to the right, that is, the horizontal direction, the y-axis points to the trunk to be measured, that is, the depth direction, and the z-axis points vertically upward, that is, the height direction;

[0045] S2, coordinate system transformation step: obtain the depth image coordinate system O i -uv and world coordinate system O w- The transformation relationship between xyz is as follows:

[0046] S21, depth image coordinate system O i -uv and depth camera coordinate system O s -x′y′z′ transformation relationship is:

[0047]

[0048] Among them, c x′ and c y′ are the principal point coordinates of the camera in the x′ and y′ directions, f x′ and f y′ is the focal length of the camera in the x′ and y′ directions, Δd is the camera depth scale factor, and I(u, vv) is the grayscale value of the pixel with coordinates (u, v);

[0049] S22, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -The transformation relationship of xyz is:

[0050]

[0051] Where H is the height of the camera from the ground;

[0052] S23, depth image coordinate system O i -uv and world coordinate system O w -The transformation relationship of xyz is:

[0053]

[0054] The inverse transform is:

[0055]

[0056] S3, tree trunk depth image acquisition step: using a depth camera to acquire a depth image of the tree trunk to be tested;

[0057] S4, tree trunk point cloud extraction step: extract a given height z from the depth image t The tree trunk point cloud below is as follows:

[0058] S41, according to the set minimum trunk depth y min and the maximum value y max , find the minimum gray value I on the depth image min and the maximum value I max , mark the pixels falling into this gray value range as foreground (white), and the rest of the pixels as background (black);

[0059] S42, dividing the foreground pixels into a number of connected domains using a connected domain analysis algorithm, retaining the connected domain with the largest area including the tree trunk, and removing the remaining connected domains;

[0060] S43, tree trunk extraction: calculate the world coordinates of the remaining white pixels, extract the height in z t The tree trunk point cloud within the range of ±Δz, where Δz is the height margin, usually 8-12mm.

[0061] S5. Trunk parameter measurement steps: measure the given height z t The trunk position coordinates (x t ,y t ), trunk inclination angle θ t and trunk diameter at breast height d t , specifically:

[0062] S51. Use the random sampling and consensus (RANSAC) algorithm to fit the cylinder according to the tree trunk point cloud and obtain the center coordinates of the cylinder (x c ,yc , z c ), direction vector (x d ,y d ,z d ) and radius r;

[0063] S52. According to the cylinder parameters, the trunk parameters are obtained as follows:

[0064]

[0065] When implementing:

[0066] The present invention uses a depth camera model RealSense D435 to measure tree trunk parameters. The horizontal field of view of the camera is 87°, the vertical field of view is 58°, the depth image resolution is 1280×720, the optimal measurement range is 0.3-3m, and the depth error is less than 2% at 2m. First, establish the depth image coordinate system O i -uv, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -xyz, such as Figure 2 As shown in Figure 2, the height H of the camera from the ground is 1m. Then, the given height z is extracted from the trunk depth image. t Point cloud of tree trunk, z t is set to 0.8m, the height margin Δz is set to 10mm, and the minimum trunk depth y min Set to 1.5m, the maximum trunk depth y max Set to 2.5m, Figure 3 The tree trunk pixels extracted from the foreground object are given. Finally, the tree trunk point cloud is fitted with a cylinder, and the fitting result is shown in Figure 4 As shown, the given height z is measured t The position coordinates of the lower trunk (x t ,y t ) is (-0.22m, 2.21m), the trunk inclination angle θ t is 89.5°, trunk diameter at breast height d t It is 0.137m.

[0067] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for automatically measuring trunk parameters of fruit trees for vibration harvesting based on a depth camera, characterized in that: The method comprises the following steps: S1. Coordinate system establishment step: Establishing the depth image coordinate system O i -uv, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -xyz. Depth image coordinate system O i- In uv, the origin O i Located in the upper left corner of the image, u is the pixel column number, and v is the pixel row number. Camera coordinate system O s -x′y′z′, the x′ axis points to the right, the y′ axis points vertically downward, and the z′ axis points to the trunk to be measured. w -xyz, origin O w Yes i The ground projection of the x-axis points to the right, that is, the horizontal direction, the y-axis points to the trunk to be measured, that is, the depth direction, and the z-axis points vertically upward, that is, the height direction; S2, coordinate system transformation step: obtain the depth image coordinate system O i -uv and world coordinate system O w -The transformation relationship between xyz; S3, tree trunk depth image acquisition step: using a depth camera to acquire a depth image of the tree trunk to be tested; S4, tree trunk point cloud extraction step: extract a given height z from the depth image t Point cloud of tree trunks below; S5. Trunk parameter measurement steps: measure the given height z t The trunk position coordinates (x t ,y t ), trunk inclination angle θ t and trunk diameter at breast height d t .

2. The automatic measurement method for fruit tree vibration harvesting trunk parameters based on a depth camera according to claim 1 is characterized in that: The S2 is specifically: S21, depth image coordinate system O i -uv and depth camera coordinate system O s -x′y′z′ transformation relationship is: Among them, c x′ and c y′ are the principal point coordinates of the camera in the x′ and y′ directions, f x′ and f y′ is the focal length of the camera in the x′ and y′ directions, Δd is the camera depth scale factor, and I(u, v) is the grayscale value of the pixel with coordinates (u, v); S22, depth camera coordinate system O s -x′y′z′ and world coordinate system O w -The transformation relationship of xyz is: Where H is the height of the camera from the ground; S23, depth image coordinate system O i -uv and world coordinate system O w -The transformation relationship of xyz is: The inverse transform is:

3. The automatic measurement method for fruit tree vibration harvesting trunk parameters based on depth camera according to claim 1 is characterized in that: The S4 is specifically: S41, according to the set minimum trunk depth y min and the maximum value y max , find the minimum gray value I on the depth image min and the maximum value I max , mark the pixels falling into this gray value range as foreground (white), and the rest of the pixels as background (black); S42, dividing the foreground pixels into a number of connected domains using a connected domain analysis algorithm, retaining the connected domain with the largest area including the tree trunk, and removing the remaining connected domains; S43, tree trunk extraction: calculate the world coordinates of the remaining white pixels, extract the height in z t The tree trunk point cloud within the range of ±Δz, where Δz is the height margin, usually 8-12mm.

4. The automatic measurement method for fruit tree vibration harvesting trunk parameters based on a depth camera according to claim 1, characterized in that: The S5 is specifically: S51. Use the random sampling and consensus (RANSAC) algorithm to fit the cylinder according to the tree trunk point cloud and obtain the center coordinates of the cylinder (x c ,y c , z c ), direction vector (x d ,y d , z d ) and radius r; S52. According to the cylinder parameters, the trunk parameters are obtained as follows:

Images