Ratoon rice combine harvester visual navigation system and method based on binocular vision

Abstract

The invention discloses a ratooning rice combine harvester visual navigation system based on binocular vision. The ratooning rice combine harvester visual navigation system comprises an equipment position adjusting unit, an image collecting unit, an image processing unit and a hydraulic control unit. The equipment position adjusting unit adjusts the orientation of the image acquisition unit in real time according to the posture of the harvester; the image acquisition unit is used for acquiring ratooning rice images and transmitting the images to the image processing unit; the image processing unit processes the ratooning rice image, identifies a crop boundary to obtain boundary line information, judges the steering and posture position of the harvester according to the boundary line information, and sends a generated speed execution signal and a steering execution signal to the hydraulic control unit; the hydraulic control unit controls the advancing speed of the harvester according to the speed execution signal and controls steering of the harvester according to the steering execution signal. According to the method for obtaining the boundary navigation line by utilizing the probability Hough transformation, the problem of low image imaging precision caused by overlarge vibration of a traditional intelligent harvester is solved.

Description

Technical Field

[0001] This invention relates to a system for a combine harvester of ratooned rice, and more specifically, to a visual navigation system for a combine harvester of ratooned rice based on binocular vision. Background Technology

[0002] Wide-narrow row planting of ratooning rice simplifies field management, increases light penetration, and reduces humidity, thus minimizing pests and diseases. Ratooning rice cultivation requires combine harvesters to travel within predetermined row spacing during harvesting to minimize mechanical damage to the rice. However, the complex and varied terrain of the fields causes significant machine movement, resulting in low image accuracy from cameras. Therefore, identifying ratooning rice boundaries, acquiring navigation information, controlling the harvester's start, stop, and steering, and obtaining high-precision images become crucial. Summary of the Invention

[0003] Therefore, it is necessary to provide a vision navigation system for a combine harvester of regenerated rice based on binocular vision to address the aforementioned technical problems.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0005] A binocular vision-based visual navigation system for a combine harvester of ratooning rice, characterized in that the binocular vision-based visual navigation system for a combine harvester of ratooning rice includes an equipment position adjustment unit, an image acquisition unit, an image processing unit, and a hydraulic control unit.

[0006] The device position adjustment unit adjusts the orientation of the image acquisition unit in real time according to the posture of the harvester;

[0007] The image acquisition unit is used to acquire images of ratooned rice and transmit the ratooned rice images to the image processing unit;

[0008] The image processing unit processes the image of the ratooning rice, identifies the crop boundary to obtain boundary line information, and determines the turning and attitude position of the harvester through the boundary line information, and sends the generated speed execution signal and turning execution signal to the hydraulic control unit.

[0009] The hydraulic control unit controls the forward speed of the harvester according to the speed execution signal and controls the direction of the harvester according to the steering execution signal.

[0010] In a preferred embodiment of the present invention, the device position adjustment unit includes a main support, on which a pair of rotary motors are mounted. Each rotary motor has a rotating head at its output end, and a guide rail is fixedly connected to the rotating head. A rack is provided on one side of the guide rail, and the rack is fixedly connected to the guide rail.

[0011] The guide rail has a sliding guide rail seat, and the guide rail seat has a rotatable guide rail shaft. The guide rail shaft passes through the guide rail seat. The front part of the guide rail shaft has a mounting base, and the mounting base has an image acquisition unit. The rear part of the guide rail shaft has a drive gear and a stepper motor. The drive gear cooperates with a rack, and the stepper motor drives the drive gear.

[0012] In a preferred embodiment of the present invention, the device position adjustment unit acquires the real-time attitude of the harvester through a gravity sensor and transmits the real-time attitude of the harvester to a stepper motor and a rotary motor, wherein the stepper motor and the rotary motor control the orientation of the image acquisition unit.

[0013] In a preferred embodiment of the present invention, the image acquisition unit is an industrial high-definition camera.

[0014] In a preferred embodiment of the present invention, the image processing unit is an industrial control computer.

[0015] In a preferred embodiment of the present invention, the hydraulic control unit controls the forward speed of the harvester through an electromagnetic proportional valve and controls the steering of the harvester through an electromagnetic directional valve.

[0016] A visual navigation method for a combine harvester of ratooning rice based on binocular vision, comprising the following steps:

[0017] Step S1: Install the image acquisition unit;

[0018] Step S2: Acquire images of ratooned rice: The image acquisition unit acquires images of ratooned rice and transmits the images of ratooned rice to the image processing unit.

[0019] Step S3: The image processing unit enhances the image of the ratooned rice in step S2, segments the harvested area and the unharvested area, obtains boundary line information, and determines the turning and attitude position of the harvester through the boundary line information, and sends the generated speed execution signal and turning execution signal to the hydraulic control unit.

[0020] Step S4: The starter motor of the harvester receives the signal from the image processing unit via the CAN bus and starts the harvester; the engine controller of the harvester receives the signal from the image processing unit via the CAN bus and stops the harvester.

[0021] Step S5: The hydraulic control unit controls the forward speed of the harvester according to the speed execution signal sent by the image processing unit, and controls the steering of the harvester according to the steering execution signal.

[0022] In a preferred embodiment of the present invention, the boundary line information is a continuous boundary line segment.

[0023] In a preferred embodiment of the present invention, in step S3, the image processing unit converts the acquired regenerated rice color image from the RGB color space to the HSV color space, selects the saturation S channel as the image for subsequent processing, and uses a two-dimensional zero-mean discrete Gaussian filter to smooth the selected channel image.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] This invention provides a visual navigation system for a combine harvester of ratooning rice based on binocular vision. The invention proposes a method for acquiring images of ratooning rice based on binocular vision technology and obtaining boundary navigation lines using probabilistic Hough transform, which provides effective technical support for precise and intelligent navigation of the combine harvester and solves the problem of low image imaging accuracy caused by excessive vibration in traditional intelligent harvesters. Attached Figure Description

[0026] To more clearly illustrate the solutions in this invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a system structure diagram of the present invention;

[0028] Figure 2 This is a three-dimensional structural diagram of the device position adjustment unit of the present invention;

[0029] Figure 3 for Figure 2 A three-dimensional structural diagram of the device position adjustment unit, providing another perspective;

[0030] Figure 4 This is a schematic diagram illustrating the transformation from the camera coordinate system to the world coordinate system according to the present invention;

[0031] Figure 5 This is a binary diagram of the present invention;

[0032] Figure 6 A schematic diagram of the closed operation of the invention;

[0033] Figure 7 This is a schematic diagram of the navigation line fitting of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0035] like Figure 1 As shown, the binocular vision-based visual navigation system for the combine harvester of regenerated rice includes an equipment position adjustment unit A, an image acquisition unit B, an image processing unit C, and a hydraulic control unit D.

[0036] The device's position adjustment unit A adjusts the orientation of the image acquisition unit B in real time according to the harvester's posture. Image acquisition unit B acquires images of the ratooned rice and transmits them to the image processing unit C. Image processing unit C processes the ratooned rice images, identifies crop boundaries to obtain boundary line information, and uses this information to determine the harvester's steering and posture position. It then sends the generated speed execution signal and steering execution signal to the hydraulic control unit D. The hydraulic control unit D controls the harvester's forward speed based on the speed execution signal and controls its steering based on the steering execution signal.

[0037] In addition, the hydraulic control unit D controls the forward speed of the harvester through the electromagnetic proportional valve D1 and controls the steering of the harvester through the electromagnetic directional valve D2.

[0038] like Figure 2 and Figure 3 As shown, the device position adjustment unit A includes a main support 3, on which a pair of rotary motors 8 are provided. A rotary head 10 is provided on the output end of the rotary motor 8. A guide rail 2 is fixedly connected to the rotary head 10. A rack 7 is provided on one side of the guide rail 2. The rack 7 is fixedly connected to the guide rail 2.

[0039] The guide rail 2 is provided with a sliding guide rail seat 4, and a rotatable guide rail shaft 9 is provided on the guide rail seat 4. The guide rail shaft 9 passes through the guide rail seat 4. The front part of the guide rail shaft 9 is provided with a mounting base 11, and the mounting base 11 is provided with an image acquisition unit B. The rear part of the guide rail shaft 9 is provided with a drive gear 5 and a stepper motor 6. The drive gear 5 cooperates with a rack 7, and the stepper motor 6 drives the drive gear 5.

[0040] It should be noted that the position adjustment unit A of the device obtains the real-time attitude of the harvester through the gravity sensor and transmits the real-time attitude of the harvester to the stepper motor 6 and the rotary motor 8. The stepper motor 6 and the rotary motor 8 control the orientation of the image acquisition unit B.

[0041] In addition, image acquisition unit B is an industrial high-definition camera. Image processing unit C is an industrial control computer.

[0042] The visual navigation method for a combine harvester of ratooning rice based on binocular vision includes the following steps:

[0043] Step S1: Install image acquisition unit B;

[0044] Step S2: Acquire images of ratooned rice: Image acquisition unit B acquires images of ratooned rice, and then transmits the images of ratooned rice to image processing unit C.

[0045] Step S3: Image processing unit C enhances the image of the ratooned rice from step S2, segments the harvested and unharvested areas, obtains boundary line information, and uses this boundary line information to determine the turning and attitude position of the harvester. The generated speed execution signal and turning execution signal are then sent to the hydraulic control unit D. It should be noted that this boundary line information consists of continuous boundary line segments.

[0046] Step S4: The starter motor E of the harvester receives the signal from the image processing unit C via the CAN bus and starts the harvester; the engine controller F of the harvester receives the signal from the image processing unit C via the CAN bus and stops the harvester.

[0047] Step S5: The hydraulic control unit D controls the forward speed of the harvester according to the speed execution signal sent by the image processing unit C, and controls the direction of the harvester according to the steering execution signal.

[0048] The steps described above will be explained in further detail below.

[0049] In step S3, the image processing unit C performs image processing on the image of the regenerated rice.

[0050] The acquired color images of regenerated rice were converted from the RGB color space to the HSV color space, and the saturation (S) channel was selected as the image for subsequent processing. Then, a two-dimensional zero-mean discrete Gaussian filter was used to smooth the selected channel image to suppress noise. The Gaussian filter function is expressed as:

[0051]

[0052] Where (u,v) are the coordinates of the filter kernel relative to the center, and σ is the standard deviation.

[0053] To remove noise from the color image and enhance the difference between harvested and unharvested areas of the ratooning rice, the gray values ​​of the filtered image were normalized to within [0,1]. The normalized image was then subjected to the following enhancement processes. First, histogram equalization was performed. Let the input normalized gray value be f, and the output gray value be h. The transformation function h = T(f) is constructed based on the cumulative gray distribution function of the original image. Next, a Gamma transformation was performed, i.e., Gamma correction was applied to the histogram equalized image. The transformation relationship is O = K·I. γ Where I and O are the normalized gray values ​​before and after the transformation, respectively, K is a constant with a value of 1, and γ is the Gamma value with a value of 2.

[0054] The image undergoing image enhancement is binarized using a local adaptive thresholding method, converting all pixels to either 0 or 255 to obtain a binary image, such as... Figure 5 As shown in the figure. This method mainly uses each pixel as the center and calculates the average value and standard deviation of the neighborhood of each pixel as the threshold for the current point. If the value exceeds the threshold, the pixel value of that point is 255; otherwise, it is 0, thus solving the problem of uneven illumination. The specific formula is shown below:

[0055] T uv =k1z uv +k2σ uv (2)

[0056] Where k1 and k2 represent nonnegative constants, z uv and σ uv Let represent the mean and standard deviation of the set of pixel grayscale values ​​contained in the region centered at (u,v), respectively.

[0057] The specific implementation steps of the improved Hough transform are as follows:

[0058] (1) Select a point from the set of edge points D every 5 edge points:

[0059] D'={d i ∈D|imod5=0} (3)

[0060] Where D' represents the sampled point set, d i This represents the i-th point in the set of edge points.

[0061] (2) Randomly select d i ∈D', and take the next 4 consecutive points (add points from the front if necessary) to form:

[0062] D s ={d i-2 ,d i-1 ,d i,d i+1 ,d i+2} (4)

[0063] If it is impossible to form 5 points, return to step 1.

[0064] (3) Regarding D s Calculate the parameters (θ, ρ) of adjacent points in D (where θ represents the angle in polar coordinates and ρ represents the distance from the point to the origin in polar coordinates), resolution Δθ, Δρ, and construct the accumulator A(θ, ρ): A(θ, ρ) = A(θ, ρ) + 1 for each point in D s The points in the array are accumulated. If there exists an accumulated value: maxA(θ,ρ)≥H (where H represents the voting threshold of the accumulator, i.e., the minimum number of points required to form a straight line), then the corresponding point d and the corresponding line parameters (θ,ρ) are retained; otherwise, D is cleared. s Then return to step 2.

[0065] (4) Starting from d, search for other points d in D'. k If distance(d) k ,line(θ,ρ))≤H dist If the linear distance between a point and a line is less than a threshold, then that point d is considered to be a true point. k If a point belongs to this line, continue searching until no more points meet the conditions.

[0066] (5) Starting from the current endpoint, search along the direction of the straight line, d k ∈D', if: distance(d k ,line(θ,ρ))≤H dist Then update the endpoint coordinates to d. k The coordinates are used until the condition is no longer met.

[0067] (6) Calculate the length L of the line. If: L≥H len (where H) len If the line length threshold is not met, the line segment is retained; otherwise, return to step 1.

[0068] (7) Among all the drawn line segments, select the longest straight line as the navigation line: Set its starting point as: (u m ,v m ), the endpoint is (u n ,v n ).

[0069] (8) The equation for calculating the navigation line is:

[0070]

[0071] Furthermore, such as Figure 6 and 7 As shown, by combining the closing operation in morphology to filter out scattered points and burrs from the obtained line segments and connecting the broken edges, a clearer and more continuous boundary line segment S is obtained, that is, the boundary line information is obtained.

[0072] Furthermore, the origin of the pixel coordinate system in the images captured by the two industrial high-definition cameras is located at the upper left corner of the image, with coordinates u and v. The formula for converting image coordinates to image plane coordinates is:

[0073]

[0074] In the formula, and The transformation factor between the image plane and the image pixel coordinate system is 1028.86 and 635.28, respectively.

[0075] The transformation formula between the image plane and the camera coordinate system is:

[0076]

[0077] In the formula, the camera focal length is f = 2mm.

[0078] like Figure 4 As shown, the transformation relationship between the camera coordinate system and the world coordinate system is as follows:

[0079]

[0080] The transformation relationship between the world coordinate system and the image pixel coordinate system is as follows:

[0081]

[0082] In the formula Let f be the intrinsic parameter matrix of the camera. x and f y For the effective focal length, u0 and v0 are the offsets between the camera's optical axis and the imaging plane. Let R be the extrinsic parameter matrix of the camera, and t be the rotation matrix. The specific parameter values ​​of the matrix are obtained from the camera calibration.

[0083] In step S5, the hydraulic control unit D receives signals from the image processing system via the CAN bus. By changing the current, it alters the opening size of the electromagnetic proportional valve D1, thereby changing the oil flow and thus the forward speed. Similarly, it changes the direction of the hydraulic oil flow by changing the electromagnetic directional valve D2, thus changing the steering. The harvester reduces its forward speed by steering, thereby changing the feed rate.

[0084] Image processing unit C sends commands to hydraulic control unit D via CAN bus to ensure a constant speed and stable feed rate of the harvester. Based on navigation lines, image processing unit C sends commands to control the harvester's automatic steering, ensuring it travels within the predetermined row spacing.

[0085] This invention proposes a method for obtaining boundary navigation lines based on binocular vision technology to acquire images of ratooned rice and using probabilistic Hough transform, which provides effective technical support for precise and intelligent navigation of combine harvesters and solves the problem of low image imaging accuracy caused by excessive vibration in traditional intelligent harvesters.

[0086] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application.

Claims

1. A visual navigation system for a combine harvester of ratooning rice based on binocular vision, characterized in that, The binocular vision-based visual navigation system for the combined harvester of regenerated rice includes an equipment position adjustment unit (A), an image acquisition unit (B), an image processing unit (C), and a hydraulic control unit (D). The device position adjustment unit (A) adjusts the orientation of the image acquisition unit (B) in real time according to the posture of the harvester; The image acquisition unit (B) is used to acquire images of the regenerated rice and transmit the images of the regenerated rice to the image processing unit (C). The image processing unit (C) processes the image of the ratooned rice, identifies the crop boundary to obtain boundary line information, and determines the turning and attitude position of the harvester through the boundary line information, and sends the generated speed execution signal and turning execution signal to the hydraulic control unit (D). The hydraulic control unit (D) controls the forward speed of the harvester according to the speed execution signal and controls the direction of the harvester according to the steering execution signal.

2. The visual navigation system for a combine harvester of ratooning rice based on binocular vision according to claim 1, characterized in that, The device position adjustment unit (A) includes a main support (3), on which a pair of rotary motors (8) are provided. A rotating head (10) is provided on the output end of the rotary motors (8). A guide rail (2) is fixedly connected to the rotating head (10). A rack (7) is provided on one side of the guide rail (2). The rack (7) is fixedly connected to the guide rail (2). The guide rail (2) is provided with a sliding guide rail seat (4), and the guide rail seat (4) is provided with a rotatable guide rail shaft (9). The guide rail shaft (9) passes through the guide rail seat (4). The front part of the guide rail shaft (9) is provided with a mounting base (11), and the mounting base (11) is provided with an image acquisition unit (B). The rear part of the guide rail shaft (9) is provided with a drive gear (5) and a stepper motor (6). The drive gear (5) cooperates with the rack (7), and the stepper motor (6) drives the drive gear (5).

3. The visual navigation system for a combine harvester of ratooning rice based on binocular vision according to claim 2, characterized in that, The device position adjustment unit (A) acquires the real-time attitude of the harvester through a gravity sensor and transmits the real-time attitude of the harvester to the stepper motor (6) and the rotary motor (8). The stepper motor (6) and the rotary motor (8) control the orientation of the image acquisition unit (B).

4. The visual navigation system for a combine harvester of ratooning rice based on binocular vision according to claim 1, characterized in that, The image acquisition unit (B) is an industrial high-definition camera.

5. The visual navigation system for a combine harvester of ratooning rice based on binocular vision according to claim 1, characterized in that, The image processing unit (C) is an industrial control computer.

6. The visual navigation system for a combine harvester of ratooning rice based on binocular vision according to claim 1, characterized in that, The hydraulic control unit (D) controls the forward speed of the harvester through the electromagnetic proportional valve (D1) and controls the steering of the harvester through the electromagnetic directional valve (D2).

7. A visual navigation method for a combine harvester of ratooning rice based on binocular vision, comprising the following steps: Step S1: Install the image acquisition unit (B); Step S2: Acquire images of ratooned rice: The image acquisition unit (B) acquires images of ratooned rice and transmits the images of ratooned rice to the image processing unit (C). Step S3: The image processing unit (C) performs image enhancement on the ratooning rice image in step S2, segments the harvested area and the unharvested area, obtains boundary line information, and determines the turning and attitude position of the harvester through the boundary line information, and sends the generated speed execution signal and turning execution signal to the hydraulic control unit (D). Step S4: The starter motor (E) of the harvester receives the signal sent by the image processing unit (C) through the CAN bus and starts the harvester. The engine controller (F) of the harvester receives the signal sent by the image processing unit (C) through the CAN bus and stops the harvester. Step S5: The hydraulic control unit (D) controls the forward speed of the harvester according to the speed execution signal sent by the image processing unit (C), and controls the steering of the harvester according to the steering execution signal.

8. The visual navigation method for a combine harvester of ratooning rice based on binocular vision according to claim 6, characterized in that, The boundary line information is a continuous boundary line segment.

9. The visual navigation method for a combine harvester of ratooning rice based on binocular vision according to claim 6, characterized in that, In step S3, the image processing unit (C) converts the acquired regenerated rice color image from the RGB color space to the HSV color space, selects the saturation S channel as the image for subsequent processing, and uses a two-dimensional zero-mean discrete Gaussian filter to smooth the selected channel image.

Images