A method for measuring distance between stubble-covered ground and ground based on multi-source image threshold segmentation

Abstract

A method for measuring distance on stubble-covered ground based on multi-source image threshold segmentation relates to the technical field of agricultural machinery surface measurement. A depth image and a plane image of the stubble-covered surface are obtained, and the plane image and the depth image are aligned through registration. The plane image is threshold segmented to obtain a binary image of the stubble-covered area. Morphological operations are performed on the binary image of the stubble area to improve robustness. The depth information of the area affected by the cover in the depth image is removed using the binary image of the stubble area, thereby obtaining a depth image without the cover. The average pixel value of the area without the cover is calculated and converted into distance to obtain the true distance on the ground. This method solves the problem that traditional fixed-depth distance measurement methods cannot accurately measure distance on stubble-covered surfaces, and can more accurately obtain ground distance information without the interference of cover. The threshold segmentation operation is fast and has the advantages of strong adaptability and high precision.

Description

Technical Field

[0001] The invention relates to the technical field of agricultural machinery surface measurement, in particular to a method for measuring distance of stubble-covered ground based on multi-source image threshold segmentation. Background Art

[0002] Traditional depth control methods primarily focus on tilled seedbeds, using mechanical limiters, pressure control, and laser / ultrasonic sensors to measure or control the depth of the implement. However, when the ground is covered with stubble, these methods are unable to accurately measure ground distance.

[0003] In the depth control of operations such as fixed-depth sowing, the pressure control type targets the prepared seedbed and achieves a consistent distance by maintaining a constant pressure. However, under surface cover conditions, the pressure feedback and depth are not linearly related, and the distance cannot be accurately obtained to maintain the fixed depth operation. The single-point ranging type uses ultrasonic / laser sensors. Under surface cover conditions, the measured distance is the distance to the cover, not the distance to the ground. Therefore, it is also unable to obtain the ground distance information under the cover and cannot maintain a constant depth.

[0004] In view of the above reasons, the present invention obtains ground information over a large range through the planar image obtained by the 2D camera and the depth image obtained by the 3D camera, and obtains ground information by performing threshold segmentation on the covered area to remove the influence of the covered area, thereby achieving the purpose of accurately obtaining the ground distance. Summary of the Invention

[0005] In order to address the shortcomings of the background technology, the present invention provides a stubble covered ground distance measurement method based on multi-source image threshold segmentation, which can more accurately obtain ground distance information without the interference of covering materials. The threshold segmentation operation speed is fast and has the advantages of strong adaptability and high precision.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for measuring the distance of stubble covered ground based on multi-source image threshold segmentation, comprising the following steps:

[0007] Step 1: Obtain a depth image and a planar image of the stubble-covered surface. Align the planar image with the depth image through registration so that pixels at the same pixel coordinate (x, y) correspond to the same object. The distance to the object corresponding to the pixel at the depth image (x, y) is z(x, y).

[0008] Step 2: Perform threshold segmentation on the plane image processed in step 1 to obtain the binary image mask1 of the stubble area, which represents the stubble coverage area S′ C and the area G′ without stubble coverage C, the pixel values m′(x,y) at different (x,y) coordinates in the binary image mask1 of the stubble area are:

[0009]

[0010] Step 3: Perform morphological operations on the binary image mask1 of the stubble area obtained in step 2 to improve robustness and obtain the binary image mask2 of the stubble area, which represents the stubble coverage area S. C and no stubble covered area G C , the pixel values m(x,y) at different (x,y) coordinates in the binary image mask2 of the stubble area are:

[0011]

[0012] Step 4: Let D be the set of all pixel coordinates (x, y) in the depth image, S D is the set of pixel coordinates in the stubble covered area, G D is the set of pixel coordinates in the area without stubble coverage. From step 1, we can know that:

[0013] S D =S C

[0014] According to the binary image mask2 of the stubble area, the stubble coverage area S in the depth image is C The pixel values of the corresponding points are set to 0, and the pixel values of the other points remain unchanged, thereby obtaining the depth image dst after removing the stubble coverage. The pixel value at (x, y) in the depth image is d(x, y). The pixel value dst(x, y) at (x, y) in the depth image dst after removing the stubble coverage is as follows:

[0015]

[0016] Find the average value avr of non-zero dst(x,y), and convert avr to distance z using the unit conversion factor g, where z is the actual ground distance.

[0017] Compared with the prior art, the present invention has the following beneficial effects: for ground distance measurement under complex surface conditions with stubble cover, the present invention obtains a plane image of the ground in a large range through a 2D camera and a depth image obtained by a 3D camera, uses a threshold to segment the plane image, and uses the segmentation result after morphological processing to remove the cover area of the depth image, so that the ground distance information after removing the interference of the cover can be obtained more accurately, solving the problem that the traditional fixed-depth ranging method cannot accurately measure the distance on the surface covered with stubble, and can be used for obtaining information for fixed-depth control in operations such as tillage, sowing, and transplanting. Compared with the single-point ranging method, the present invention can eliminate the interference of the cover, use the plane image aligned with the depth image as the basis for removing the stubble area of the depth image, have richer feature information, can more accurately determine the stubble area, and the threshold segmentation operation speed is fast, which can meet the needs of real-time control, and has the advantages of strong adaptability and high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a system block diagram of the present invention;

[0019] Figure 2 is the depth image collected in Example 1;

[0020] Figure 3 is the planar image collected in Example 1;

[0021] Figure 4 is the planar image aligned with the depth image in Example 1;

[0022] Figure 5 is the binary image of the stubble area obtained by threshold segmentation in Example 1;

[0023] Figure 6 Yes Figure 5 A more robust binary image is obtained after the dilation operation;

[0024] Figure 7 The depth image obtained by removing the area affected by the covering in the depth image in Example 1;

[0025] Figure 8 is the color plane image aligned with the depth image in Example 2;

[0026] Figure 9 is the binary image of the stubble area obtained by threshold segmentation in Example 2;

[0027] Figure 10 is the color plane image aligned with the depth image in Example 3;

[0028] Figure 11 is the binary image of the stubble area obtained by threshold segmentation in Example 3;

[0029] Figure 12 is the light intensity perception image acquired by the TOF camera in Example 4;

[0030] Figure 13 This is the binary image of the stubble area obtained by threshold segmentation in Example 4. DETAILED DESCRIPTION

[0031] The technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0032] like Figure 1 As shown, a method for measuring the distance between stubble-covered ground and the ground based on multi-source image threshold segmentation includes the following steps:

[0033] Step 1: Use a camera to obtain a depth image and a plane image of the stubble-covered surface. The depth image can be obtained using a time-of-flight (TOF) camera, and the plane image can be a light intensity-sensing image obtained by a TOF camera, or a black-and-white or color image obtained by a 2D camera. Align the plane image and the depth image through registration so that pixels at the same pixel coordinate (x, y) correspond to the same object. The pixel at the depth image (x, y) corresponds to the object at distance z(x, y).

[0034] Step 2: Perform threshold segmentation on the plane image processed in step 1 to obtain the binary image mask1 of the stubble area, which represents the stubble coverage area S′ C and the area G′ without stubble coverage C , the pixel values m′(x,y) at different (x,y) coordinates in the binary image mask1 of the stubble area are:

[0035]

[0036] Step 3: Perform morphological operations on the binary image mask1 of the stubble area obtained in step 2 to improve robustness and obtain the binary image mask2 of the stubble area, which represents the stubble coverage area S. C and no stubble covered area G C , the pixel values m(x,y) at different (x,y) coordinates in the binary image mask2 of the stubble area are:

[0037]

[0038] Step 4: Let D be the set of all pixel coordinates (x, y) in the depth image, S D is the set of pixel coordinates in the stubble covered area, G D is the set of pixel coordinates in the area without stubble coverage. From step 1, we can know that:

[0039] S D =S C

[0040] Therefore, the depth information of the area affected by the covering in the depth image can be removed by using the binary image mask2 of the stubble area in step 3, that is, the stubble covering area S in the depth image C The pixel values of the corresponding points are set to 0, and the pixel values of the other points remain unchanged, thereby obtaining the depth image dst after removing the stubble coverage. The pixel value at (x, y) in the depth image is d(x, y). The pixel value dst(x, y) at (x, y) in the depth image dst after removing the stubble coverage is as follows:

[0041]

[0042] Find the average value avr of non-zero dst(x,y), and convert avr to distance z using the unit conversion factor g, where z is the actual ground distance.

[0043] Example 1

[0044] This embodiment uses the method of the present invention to achieve distance measurement on a stubble-covered surface without interference from the covering. The information sources used are a TOF camera and a color 2D camera. The color plane image is segmented using a threshold to obtain the stubble cover range, thereby removing the stubble cover influence area in the depth image and obtaining the true ground distance. The details are as follows:

[0045] Step 1: Use a TOF camera and a 2D camera to obtain a depth image and a plane image of the stubble-covered surface, respectively. Align the plane image with the depth image through registration so that pixels at the same pixel coordinates (x, y) correspond to the same object. The distance to the object corresponding to the pixel at the depth image (x, y) is z(x, y).

[0046] Among them, the depth image is combined Figure 2 The image is a 16-bit single-channel grayscale image with a resolution of 640×480. Figure 3The figure shows a 3-channel 8-bit RGB color image with a resolution of 1280×1024. The planar image has radial distortion. The radial distortion of the planar image is corrected, and then the intrinsic parameter matrix of the color 2D camera and the TOF camera is used. The extrinsic parameter matrix of the TOF camera with the color 2D camera coordinate system as the world coordinate system is used to align the planar image and the depth image by reverse mapping. The obtained planar image aligned with the depth image is combined Figure 4 As shown in Figure 2, the aligned planar image and the depth image have the same resolution of 640×480. Pixels with the same coordinates (x, y) correspond to the corresponding positions of the target object. The coordinates (x, y) in the aligned color planar image can be directly applied to operations on the depth image.

[0047] Step 2: Perform threshold segmentation on the plane image processed in step 1 to obtain the binary image mask1 of the stubble area, which represents the stubble coverage area S′ C and the area G′ without stubble coverage C , where the color threshold segmentation method is used to convert the color space of the plane image from RGB to HSV, and the plane image is segmented using the color threshold. The lower limit of the color threshold is (0,0,100) and the upper limit is (255,255,255). The segmentation result is a single-channel 8-bit binary image mask1 combined with Figure 5 As shown in the figure, the pixel values of each point in the binary image mask1 of the stubble area are as follows:

[0048]

[0049] Step 3: Dilate the binary image mask1 of the stubble area obtained in step 2 to obtain a more robust binary image mask2 of the stubble area. Figure 6 As shown, the stubble coverage area S C and no stubble covered area G C , the pixel values of each point in the binary image mask2 of the stubble area are as follows:

[0050]

[0051] The binary image mask2 after the dilation operation is an 8-bit image and cannot be directly operated with the depth image obtained by the TOF camera. It needs to be converted to a 16-bit image. The pixel values of each point after conversion are as follows:

[0052]

[0053] Step 4: Let D be the set of all pixel coordinates (x, y) in the depth image, S D is the set of pixel coordinates in the stubble covered area, G Dis the set of pixel coordinates in the area without stubble coverage. From step 1, we can know that:

[0054] S D =S C

[0055] Therefore, the depth information of the area affected by the covering in the depth image can be removed by using the binary image mask2 of the stubble area in step 3, that is, the stubble covering area S in the depth image C The pixel values of the corresponding points are set to 0, and the pixel values of the other points remain unchanged, thereby obtaining the depth image dst after removing the stubble coverage. In this embodiment, the pixel value d(x, y) of the 16-bit image of mask2 at the coordinate (x, y) is subtracted from the pixel value d(x, y) of the depth image at the coordinate (x, y) to obtain the pixel value dst(x, y) of the depth image after removing the stubble coverage at the coordinate (x, y):

[0056]

[0057] Depth image combination after removing the influence of stubble coverage Figure 7 As shown, the average value avr of non-zero dst(x, y) is calculated and multiplied by the pixel-to-millimeter unit conversion factor g (g = 6.5536 in this example) to obtain the distance z in millimeters. z is the ground distance after the stubble is removed. In this example, the z value obtained is 531 mm. The average distance calculated using the depth image without stubble removal is 525 mm, and the average distance calculated using the depth image after the stubble is completely cleared is 532 mm. The results show that this method is effective in removing stubble interference.

[0058] Example 2

[0059] The difference between this embodiment and embodiment 1 is that: in step 2, Figure 8 After the aligned color plane image is converted into a single-channel grayscale image, the threshold is selected according to the Otsu algorithm for threshold segmentation, and the binary image of the stubble area is combined with Figure 9 shown.

[0060] Example 3

[0061] The difference between this embodiment and embodiment 1 is that: in step 1, a black and white 2D camera is used to obtain a plane image, and the aligned black and white plane images are combined Figure 10 As shown in the figure, in step 2, Gaussian adaptive threshold is used for threshold segmentation, and the binary image of the stubble area is combined with Figure 11 shown.

[0062] Example 4

[0063] The difference between this embodiment and embodiment 1 is that the plane image in step 1 is a combination of the light intensity perception image obtained by the TOF camera Figure 12 As shown, it has been aligned with the depth image obtained by the TOF camera when it was acquired, so no additional registration and alignment is required. In step 2, the plane image is segmented by selecting a threshold according to the Otsu algorithm, and the binary image of the stubble area is combined with Figure 13 shown.

[0064] The above embodiments verify that the method of the present invention can solve the problem of traditional distance measurement being interfered by covered objects by obtaining distance information over a large range. The feature segmentation of the stubble area using the planar image is effective and has high precision, and can effectively eliminate the interference of surface stubble and accurately obtain the ground distance.

[0065] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other configurations without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations coming within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0066] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for measuring distance to stubble-covered ground based on multi-source image threshold segmentation, characterized by: The following steps are involved: Step 1: Obtain a depth image and a planar image of the stubble-covered surface. Align the planar image with the depth image through registration so that pixels at the same pixel coordinate (x, y) correspond to the same object. The distance to the object corresponding to the pixel at the depth image (x, y) is z(x, y). Step 2: Perform threshold segmentation on the plane image processed in step 1 to obtain the binary image mask1 of the stubble area, which represents the stubble coverage area S′ C and the area G′ without stubble coverage C , the pixel values m′(x,y) at different (x,y) coordinates in the binary image mask1 of the stubble area are: Step 3: Perform morphological operations on the binary image mask1 of the stubble area obtained in step 2 to improve robustness and obtain the binary image mask2 of the stubble area, which represents the stubble coverage area S. C and no stubble covered area G C , the pixel values m(x,y) at different (x,y) coordinates in the binary image mask2 of the stubble area are: Step 4: Let D be the set of all pixel coordinates (x, y) in the depth image, S D is the set of pixel coordinates in the stubble covered area, G D is the set of pixel coordinates in the area without stubble coverage. From step 1, we can know that: S D =S C According to the binary image mask2 of the stubble area, the stubble coverage area S in the depth image is C The pixel values of the corresponding points are set to 0, and the pixel values of the other points remain unchanged, thereby obtaining the depth image dst after removing the stubble coverage. The pixel value at (x, y) in the depth image is d(x, y). The pixel value dst(x, y) at (x, y) in the depth image dst after removing the stubble coverage is as follows: Find the average value avr of non-zero dst(x,y), and convert avr to distance z using the unit conversion factor g, where z is the actual ground distance.

2. The method for measuring distance to stubble-covered ground based on multi-source image threshold segmentation according to claim 1, characterized in that: The depth image in the step 1 is acquired using a TOF camera, and the plane image is a light intensity perception image acquired using a TOF camera, or a black and white or color image acquired using a 2D camera.

Images