Vision-based laser scanning garlic stalk cutting machine and method

Abstract

The invention discloses a vision-based laser scanning garlic stalk cutting machine and method, and relates to the technical field of agricultural machinery, the vision-based laser scanning garlic stalk cutting machine comprises a multi-rotor unmanned aerial vehicle with a GPS module, and the bottom of the multi-rotor unmanned aerial vehicle is provided with a laser emitter and a binocular camera through a holder; the GPS module is used for guiding the multi-rotor unmanned aerial vehicle to advance in the garlic planting direction, the binocular camera is used for obtaining the three-dimensional space position of a target garlic bulb, and the laser transmitter is used for scanning and cutting off the garlic bulb in a spiral path according to the three-dimensional space position of the target garlic bulb. The garlic bulb cutting device can cut garlic bulbs at a time, and the garlic shoot cutting efficiency and effect are improved.

Description

Technical Field

[0001] This invention relates to the field of agricultural machinery technology, and in particular to a vision-based laser scanning garlic scape cutting machine and method. Background Technology

[0002] Garlic scape growth relies on the continuous division of the apical meristem. The garlic scape is an aggregate of the apical meristem and undeveloped flowers. Once the entire garlic scape is laser-cut, the source of cell division is completely removed, and the scape stops growing. Removing the garlic scape at the tip inhibits scape growth. Existing garlic scape cutting machines can be mainly divided into two categories: First, there are reciprocating cutting mechanisms based on mechanical contact. These machines typically use tracked or wheeled walking devices that travel along the garlic planting rows. A front-mounted rake guides the garlic scapes to the designated cutting area, and then a rotating blade or reciprocating cutter performs batch cutting. Second, there are semi-automatic positioning cutting devices. These devices add a simple photoelectric sensor module to the mechanical cutting mechanism. The photoelectric sensor detects the approximate position of the garlic scapes, triggering the cutting mechanism. Some machines are also equipped with a simple height adjustment device to accommodate garlic of different growth heights.

[0003] Existing garlic scape cutting machines still produce the following problems during the scape cutting process: (1) The garlic scapes are bent and mixed with the garlic leaves, and the bending direction is messy and disorderly. It is difficult to accurately identify and grab the garlic scapes when mechanically pulling out the scapes, resulting in the scapes being pinched in the wrong direction and causing damage to the plant. (2) Mechanical clamping method: Because the process of clamping and releasing the garlic scape is repeated, it has a weak ability to adapt to changes in the posture, position and flexibility of the garlic scape, which eventually leads to the leakage of the scape; (3) The diameter of the garlic blister is about 4~8mm, while the diameter of the laser beam is about 0.2mm, so it cannot remove the garlic blister in one go. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a vision-based laser scanning garlic scape cutting machine and method, which can remove garlic scapes in one go, thereby improving the efficiency and effectiveness of garlic scape cutting.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, embodiments of the present invention provide a vision-based garlic laser scanning and cutting machine, comprising a multi-rotor drone with a GPS module, wherein a laser emitter and a binocular camera are mounted on the bottom of the multi-rotor drone via a gimbal; the GPS module is used to guide the multi-rotor drone to move along the garlic planting direction, the binocular camera is used to acquire the three-dimensional spatial position of the target garlic scape, and the laser emitter is used to scan and cut the garlic scape using a spiral path based on the three-dimensional spatial position of the target garlic scape.

[0006] As a further implementation, the gimbal includes a rotatable navigation mount; The binocular camera is fixed to the bottom of the navigation frame, and the laser emitter is located in the middle of the binocular camera.

[0007] As a further implementation, the gimbal includes a rotatable gimbal body, with the Z-axis as the rotation center; The navigation frame is connected to the bottom of the gimbal body via a rotating mechanism, and the rotation axis of the navigation frame is perpendicular to the rotation center of the gimbal body.

[0008] As a further implementation, the gimbal body is connected to a rotation mechanism, which is used for coarse alignment of the laser emitter and the binocular camera in the axial direction; the rotation mechanism is used for coarse alignment of the laser emitter and the binocular camera in the height.

[0009] As a further implementation, the laser emitter is equipped with a high-speed resonant mirror module, which is used to control the scanning path of the laser emitter.

[0010] Secondly, embodiments of the present invention also provide a vision-based garlic laser scanning cutting method, employing the aforementioned garlic laser scanning cutting machine, comprising: Establish the absolute coordinate system for the multi-rotor UAV; Garlic images are acquired, and the bounding boxes of garlic pods are obtained based on object detection algorithms. The central region of the bounding box is then thinned using stereo matching with binoculars to obtain sub-pixel coordinates in the camera coordinate system. The position of the garlic pods is obtained based on a trajectory prediction algorithm, and the predicted coordinates are output. Calculate the gimbal angle deviation, use the angle offset as the position loop input, and adjust the center of one camera to align with the center of the bulb; calculate the error compensation angle to align the center of the laser emitter with the center of the bulb. The laser is controlled by a high-speed resonant mirror module to scan and remove the target garlic bulb in a spiral pattern.

[0011] As a further implementation, the gimbal angle deviation includes the gimbal yaw angle and pitch angle.

[0012] As a further implementation, the gimbal angle deviation calculation includes: First, calculate the pixel offset of the target center relative to the image center. Then, convert the pixel offset into an angular offset using the camera parameter FOV, so that the camera optical axis points to the target center in real time. The target center coordinates in the camera coordinate system are obtained by using a binocular camera. The offset distance from the laser emitter optical axis to the camera coordinate system is obtained through calibration. Angle error compensation is calculated to make the laser head optical axis point to the target center in real time.

[0013] As a further implementation, the image is preprocessed, and the preprocessing node completes distortion correction and exposure normalization within a set time; the preprocessed image is then input into the target detection algorithm.

[0014] As a further implementation method, using the GPS module's directional baseline as a reference, feedforward waypoints are output in real time according to the set speed to establish the UAV's absolute coordinate system.

[0015] The beneficial effects of this invention are as follows: (1) The garlic scape cutting machine of the present invention includes a multi-rotor drone with a GPS module. A laser emitter and a binocular camera are installed on the bottom of the multi-rotor drone via a gimbal. The binocular camera is used to obtain the three-dimensional spatial position of the target garlic scape, and the laser emitter is used to scan and cut the garlic scape according to the three-dimensional spatial position of the target garlic scape in a spiral path, thereby achieving one-time cutting of the garlic scape.

[0016] (2) The present invention uses a GPS module and a binocular camera to form a stereo perception unit, which generates the three-dimensional coordinates of the UAV-garlic bubble in real time. Combined with deep learning algorithms, it completes identification-positioning-attitude estimation in a short time, guides the gimbal to actively track and compensate for the shaking of the UAV, and ensures that the laser focus always moves along the optimal cutting surface. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is a schematic diagram of a quadcopter drone equipped with a gimbal according to one or more embodiments of the present invention; Figure 2 This is a schematic diagram of the structure of a quadcopter drone according to one or more embodiments of the present invention; Figure 3 This is a schematic diagram of the gimbal structure according to one or more embodiments of the present invention. Figure 1 ; Figure 4 This is a schematic diagram of the gimbal structure according to one or more embodiments of the present invention. Figure 2 ; Figure 5 This is a flowchart of the laser scanning method for cutting stalks according to one or more embodiments of the present invention; Figure 6 This is a coordinate system schematic diagram of the present invention according to one or more embodiments; Figure 7 This is a schematic diagram of the camera coordinate system according to one or more embodiments of the present invention.

[0019] Among them, 1. Quadcopter drone, 2. Gimbal; 101. GPS module; 102. Base; 201. Outer clamp; 202. Bearing; 203. First drive gear; 204. Mounting plate; 205. Rotary motor; 206. Rotating motor; 207. Second drive gear; 208. Transmission gear; 209. Navigation frame; 210. Support component; 211. Gear ring; 212. Inner clamp. 2091. Laser emitter; 2092. Binocular camera. Detailed Implementation

[0020] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] For ease of description, the words "up," "down," "left," and "right" appearing in this invention only indicate that they are consistent with the up, down, left, and right directions of the accompanying drawings themselves, and do not limit the structure. They are merely for the purpose of facilitating the description of this invention and simplifying the description, and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0022] Example 1: This embodiment provides a vision-based laser scanning garlic scape cutting machine, such as... Figures 1-4 As shown, the device includes a multi-rotor drone with a GPS module 101, which guides the drone along the garlic planting direction. A gimbal 2 is mounted on the underside of the base 102 of the multi-rotor drone, and a laser emitter 2091 and a binocular camera 2092 are mounted on the gimbal 2. In this embodiment, the multi-rotor drone is a quadcopter drone 1.

[0023] like Figure 3 and Figure 4 As shown, the gimbal 2 mainly includes a gimbal body, a navigation frame 209, a slewing mechanism, and a rotating mechanism. The gimbal body is connected to the slewing mechanism, and the slewing center of the gimbal body is the Z-axis. The navigation frame 209 is connected to the rotating mechanism, and its rotation axis is perpendicular to the Z-axis direction.

[0024] Specifically, the gimbal body includes an outer clamping plate 201 and an inner clamping plate 212 coaxially arranged. In this embodiment, the outer clamping plate 201 is a ring structure, and the inner clamping plate 212 is a disk structure. The rotation mechanism includes a rotary motor 205, a first drive gear 203, and a gear ring 211. The bottom of the outer clamping plate 201 is fixedly connected to the gear ring 211. The gear ring 211 is connected to the inner clamping plate 212 through a bearing 202. The outer circumference of the gear ring 211 is provided with teeth, which mesh with the first drive gear 203. The first drive gear 203 is connected to the rotary motor 205, and the rotary motor 205 is fixed to the lower side of the mounting plate 204. Under the driving action of the rotary motor 205, the navigation frame 209 rotates around the Z-axis, realizing the coarse alignment of the binocular camera 2092 and the laser emitter 2091 in the Z-axis direction.

[0025] The rotating mechanism includes a rotating motor 206, a second drive gear 207, and a transmission gear 208. The rotating motor 206 is also fixed to the lower side of the mounting plate 204, and the axial direction of the rotating motor 206 is perpendicular to the axial direction of the rotary motor 205. The second drive gear 207 is installed at the output end of the rotating motor 206, and the transmission gear 208 meshes with the second drive gear 207. The transmission gear 208 is connected to the navigation frame 209, and the navigation frame 209 is rotatably connected to the support member 210 at the bottom of the mounting plate 204. Under the meshing action of the gears, the navigation frame 209 is driven to rotate relative to the support member 210 to achieve coarse alignment in the height direction of the binocular camera 2092 and the laser emitter 2091.

[0026] In this embodiment, the output shaft of the rotary motor 205 is the yaw axis of the gimbal, and the output shaft of the rotating motor 206 is the pitch axis of the gimbal. In this embodiment, all the gears are helical gears to improve transmission smoothness and load-bearing capacity; both the rotary motor 205 and the rotating motor 206 are stepper motors.

[0027] A binocular camera 2092 and a laser emitter 2091 are mounted on the bottom of the navigation frame 209, with the laser emitter 2091 positioned between the two cameras. The binocular camera 2092 is used to detect the target garlic bulb and acquire its three-dimensional spatial position; the laser emitter 2091 is used to scan and cut the garlic bulb using a spiral path based on its three-dimensional spatial position.

[0028] The laser emitter 2091 is equipped with a high-speed resonant mirror module, which adjusts the laser path emitted by the laser emitter 2091; by emitting a spiral laser path, the garlic blisters can be removed in one go.

[0029] In this embodiment, a high-speed resonant mirror module with a bandwidth of 100 Hz and a resonant mirror that always operates within an optical angle range of ±10° is selected.

[0030] This embodiment uses a multi-rotor UAV as a transport platform, which can complete the entire garlic scape cutting task in the air; the laser emitter 2091 mounted on the gimbal 2 scans with a focused beam to instantly cut the garlic scape, eliminating the damage to the garlic leaves and pseudostem; a stereo perception unit composed of a GPS module and a binocular camera generates the three-dimensional coordinates of the UAV and garlic scape in real time, and combined with a deep learning algorithm, completes identification, positioning and attitude estimation in a short time, guiding the gimbal 2 to actively track and compensate for the sway of the UAV, ensuring that the laser focus always scans and moves along the optimal cutting surface.

[0031] Example 2: This embodiment provides a vision-based method for garlic scape cutting using laser scanning, employing the garlic laser scanning scape cutting machine described in Embodiment 1. Figure 5 As shown, it includes the following steps: Step 1: Using the GPS module's directional baseline as a reference, output feedforward waypoints at a certain speed (e.g., 2 km / h) in real time to establish the UAV's absolute coordinate system.

[0032] Step 2: The stereo camera acquires images of garlic, preprocesses the images, and obtains the garlic puff bounding boxes based on the object detection algorithm. Specifically, the image preprocessing node completes distortion correction and exposure normalization within 5 ms, allowing the object detection algorithm to detect the garlic puff bounding boxes within 30 ms.

[0033] In this embodiment, the target detection algorithm uses the lightweight YOLOv8n, which has the fewest parameters and the fastest inference speed, thus improving detection efficiency.

[0034] Step 3: Perform SGM refinement on the center region of the box using stereo matching to obtain the sub-pixel coordinates (Xc, Yc, Zc) in the camera coordinate system. Then, use a 1D-LSTM trajectory prediction network to predict the position of the bubble 35 ms later using historical data from the last 5 frames, and output the predicted coordinates (X, Y, Z).

[0035] Step 4: The angle offset difference calculation unit calculates the gimbal angle offset based on pixel deviation; using the angle offset as the position loop input, it drives the gimbal's yaw and pitch axes to rotate, achieving real-time alignment between the left camera center and the bulb center; then, based on the offset between the spatial installation positions of the left camera and the laser emitter, and the distance relationship between the left camera and the bulb center, it calculates the angle error compensation amount of the laser emitter's optical axis, achieving precise alignment between the laser emitter center and the bulb center. This embodiment improves accuracy through a two-stage alignment logic of mechanical alignment and computational compensation.

[0036] Furthermore, such as Figure 6 and Figure 7As shown, the gimbal yaw angle θyaw and pitch angle θpitch are calculated. The image coordinate system O has its origin at the top left corner of the image, in pixels. The camera coordinate system Oc has its origin at the optical center of the left camera, with the Zc axis pointing towards the scene along the optical axis, the Xc axis parallel to the u axis, and the Yc axis parallel to the v axis. The gimbal coordinate system Og has its origin at the intersection of the two axes of the gimbal, with the x-axis parallel to the pitch axis and the y-axis parallel to the yaw axis. Initially, it coincides with the camera coordinate system. (1) Calculate the pixel offset of the target center relative to the image center: (1) Where W represents the width of the left camera image, H represents the height of the left camera image, in pixels; u represents the horizontal pixel coordinate of the center of the garlic bubble in the image coordinate system, v represents the vertical pixel coordinate of the center of the garlic bubble in the image coordinate system, the image coordinate system takes the upper left corner of the image as the origin, u-axis is to the right and v-axis is to the down; Δu and Δv represent the pixel offset of the target center relative to the image center.

[0037] (2) Convert the pixel offset into angular offset using the camera parameter FOV, so that the camera optical axis points to the center of the target in real time; (2) Among them, FOV w The field of view (FOV) represents the horizontal angle of view of the left camera. h This indicates the field of view of the left camera in the vertical direction, obtained from the camera's intrinsic parameters. This indicates the yaw angle of the gimbal when the optical axis of the left camera is pointed towards the center of the target. This indicates the pitch angle of the gimbal when the optical axis of the left camera is pointing towards the center of the target.

[0038] (3) Obtain the three-dimensional coordinates (X, X) of the target center in the camera coordinate system using a stereo camera. oj Y oj Z oj ), where Z oj This represents the depth distance of the target center along the optical axis of the left camera; the offset distance from the laser emitter's optical axis to the left camera's optical axis is obtained through calibration. l The angle error compensation Δθ is calculated to make the laser head optical axis point to the target center in real time. The angle error compensation is applied to the corresponding gimbal yaw axis or pitch axis according to the actual installation direction of the laser emitter relative to the camera, and its sign is determined by the direction of the calibration coordinate system.

[0039] (3) If there is a horizontal offset between the laser emitter's optical axis and the left camera's optical axis, the angle error compensation Δθ is applied to the yaw axis. ;in, This indicates the yaw angle of the gimbal when the optical axis of the laser emitter is pointed towards the center of the target.

[0040] (4) If there is a vertical offset between the laser emitter's optical axis and the left camera's optical axis, the angle error compensation Δθ is applied to the yaw axis θpitch. Wherein, This indicates the pitch angle of the gimbal when the optical axis of the laser emitter is pointed towards the center of the target.

[0041] (5) Step 5: The laser is controlled by the high-speed resonant mirror module to complete the scanning and cutting in a spiral pattern.

[0042] This embodiment can complete the identification-prediction-scanning process of garlic scapes within 50 ms, achieving rapid scape removal.

[0043] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A vision-based laser scanning garlic scape cutting machine, characterized in that, The device includes a multi-rotor drone equipped with a GPS module. The multi-rotor drone has a laser emitter and a binocular camera mounted on its bottom via a gimbal. The GPS module is used to guide the multi-rotor drone to move along the direction of garlic planting. The binocular camera is used to acquire the three-dimensional spatial position of the target garlic bulb. The laser emitter is used to scan and cut the garlic bulb using a spiral path based on the three-dimensional spatial position of the target garlic bulb.

2. The vision-based laser scanning garlic scape cutting machine according to claim 1, characterized in that, The gimbal includes a rotatable navigation mount; The binocular camera is fixed to the bottom of the navigation frame, and the laser emitter is located in the middle of the binocular camera.

3. The vision-based laser scanning garlic scape cutting machine according to claim 2, characterized in that, The gimbal includes a rotatable gimbal body, with the Z-axis as the rotation center; The navigation frame is connected to the bottom of the gimbal body via a rotating mechanism, and the rotation axis of the navigation frame is perpendicular to the rotation center of the gimbal body.

4. A vision-based laser scanning garlic scape cutting machine according to claim 3, characterized in that, The gimbal body is connected to a rotation mechanism, which is used for coarse alignment of the laser emitter and the binocular camera in the axial direction; the rotation mechanism is used for coarse alignment of the laser emitter and the binocular camera in the height.

5. A vision-based laser scanning garlic scape cutter according to claim 1, characterized in that, The laser emitter is equipped with a high-speed resonant mirror module, which is used to control the scanning path of the laser emitter.

6. A vision-based laser scanning method for cutting garlic scapes, characterized in that, The garlic laser scanning scape cutting machine as described in any one of claims 1-5 includes: Establish the absolute coordinate system for the multi-rotor UAV; Garlic images are acquired, and the bounding boxes of garlic pods are obtained based on object detection algorithms. The central region of the bounding box is then thinned using stereo matching with binoculars to obtain sub-pixel coordinates in the camera coordinate system. The location of the garlic pods is obtained based on a trajectory prediction algorithm, and the predicted coordinates are output. Calculate the gimbal angle deviation, use the angle offset as the position loop input, and adjust the center of one camera to align with the center of the bulb; calculate the error compensation angle to align the center of the laser emitter with the center of the bulb. The laser is controlled by a high-speed resonant mirror module to scan and remove the target garlic bulb in a spiral pattern.

7. A vision-based laser scanning method for cutting garlic scapes according to claim 6, characterized in that, The gimbal angle deviation includes the gimbal yaw angle and pitch angle.

8. A vision-based laser scanning method for cutting garlic scapes according to claim 7, characterized in that, The calculation of the gimbal angle deviation includes: First, calculate the pixel offset of the target center relative to the image center. Then, convert the pixel offset into an angular offset using the camera parameter FOV, so that the camera optical axis points to the target center in real time. The target center coordinates in the camera coordinate system are obtained by using a binocular camera. The offset distance from the laser emitter optical axis to the camera coordinate system is obtained through calibration. Angle error compensation is calculated to make the laser head optical axis point to the target center in real time.

9. A vision-based laser scanning method for cutting garlic scapes according to claim 6, characterized in that, The image is preprocessed, and the preprocessing node completes distortion correction and exposure normalization within a set time. The preprocessed image is then input into the target detection algorithm.

10. A vision-based laser scanning method for cutting garlic scapes according to claim 6, characterized in that, Using the GPS module's directional baseline as a reference, feedforward waypoints are output in real time according to the set speed to establish the UAV's absolute coordinate system.

Images