A deep learning-based visual farmland weed laser weeding method

Abstract

The application relates to the technical field of agricultural machinery, in particular to a laser weeding device and method based on deep learning vision in farmland, which comprises a vehicle frame, a Mecanum wheel is arranged at each corner of the vehicle frame, a lifting component is arranged on the vehicle frame, an XY-direction movable component is arranged at the movable end of the lifting component, a laser emitter is arranged at the movable end of the XY-direction movable component, a binocular image acquisition device is arranged at the fixed part of the XY-direction movable component, and the binocular image acquisition device, the XY-direction movable component and the laser emitter are electrically connected with a data processing module. The application combines binocular stereo vision technology, a YOLO11 deep learning model and laser weeding technology, and can accurately identify and position weeds.

Description

Technical Field

[0001] This invention relates to the field of agricultural machinery technology, specifically to a laser weeding method for farmland based on deep learning vision. Background Technology

[0002] Traditional farmland weeding methods mainly rely on manual and chemical weeding. Manual weeding is inefficient and labor-intensive; while chemical weeding is efficient, it easily causes environmental pollution and pesticide residues, harming human health and the ecological environment. In recent years, with the rapid development of machine vision and artificial intelligence technologies, machine vision-based intelligent weeding technology has become a research hotspot. However, most existing machine vision weeding technologies use monocular cameras, which struggle to accurately acquire three-dimensional information about weeds, resulting in low weeding precision and a risk of accidentally damaging crops. Summary of the Invention

[0003] The purpose of this invention is to provide a deep learning-based visual laser weeding method for farmland to solve the above-mentioned problems. This invention combines binocular stereo vision technology, the YOLO11 deep learning model, and laser weeding technology, which can accurately identify and locate weeds and use lasers for precise weeding, improving weeding efficiency and accuracy, reducing harm to the environment and crops. At the same time, by combining deep learning and binocular stereo matching technology, it identifies and distinguishes different types and forms of weeds, and finally uses lasers of different powers to remove weeds, thus protecting crop seedlings and preventing them from being accidentally damaged.

[0004] To achieve the above objectives, the present invention provides the following solution: A deep learning-based visual laser weeding device for farmland includes a frame with Mecanum wheels at its four corners. A lifting component is mounted on the frame, and an XY-direction movable component is mounted on the movable end of the lifting component. A laser emitter is mounted on the movable end of the XY-direction movable component, and a binocular image acquisition device is mounted on the fixed part of the XY-direction movable component. The binocular image acquisition device, the XY-direction movable component, and the laser emitter are electrically connected to a data processing module.

[0005] Preferably, the lifting component includes four electric cylinders, which are vertically arranged and mounted on the vehicle frame. The XY-direction movable component is mounted on the movable end of the electric cylinder, and the electric cylinder is electrically connected to the data processing module.

[0006] Preferably, the XY-direction movable component includes an aluminum frame, a slide rail slider is slidably mounted on the aluminum frame, a lead screw is threadedly connected to the slide rail slider, a Y-direction stepper motor is shaft-connected to the lead screw, the Y-direction stepper motor is fixedly connected to the aluminum frame, a slide rail is fixedly connected to the slide rail slider, and an X-direction stepper motor for controlling the movement of the laser emitter slider is mounted on the slide rail.

[0007] Preferably, the binocular image acquisition device is a ZED binocular camera.

[0008] A deep learning-based visual laser weed control method for farmland, implemented using the aforementioned device, specifically includes the following steps: The binocular image acquisition device acquires 2D images, depth maps, and 3D point cloud images of farmland. A YOLO11 model is set up within the data processing module; The data processing module identifies the weeds and their locations in the farmland after analyzing the 2D image, depth map, and 3D point cloud image of the farmland using the YOLO11 model. The data processing module controls the operation of the lifting component and the XY-direction movable component, driving the laser emitter to a suitable position, and clearing weeds through the laser emitter.

[0009] Preferably, the information acquired by the binocular image acquisition device includes a left-eye image and a right-eye image, wherein the image with the largest display target area in the left-eye image and the right-eye image is used as the 2D image output for the YOLO11 model analysis, which includes two-dimensional bounding box coordinate information and weed category information. Preferably, the standard convolutional structure in the YOLO11 model is replaced with a Ghost convolutional module, and a channel spatial attention mechanism is added thereto, with the channel spatial attention mechanism added to both the backbone network and the neck network.

[0010] Preferably, the YOLO11 model is as follows: Input image size: 512×512 Backbone network: The first three layers use GhostConv output channels of 32, 64, and 128 respectively; A CBAM module is inserted in the middle of the first two and third layers, and the CBAM module is inserted in the middle of the third and fourth layers. Downsampling uses the GhostConv module with a stride of 2 to replace the traditional max pooling (MaxPool). The input is the output feature map of each layer in the backbone network. Convolution is used to reduce the spatial size and enhance the feature learning ability. Neck structure: Information is extracted from feature maps at different levels, and upper and lower FPNs and lower and upper PANs are performed. Information is fused in two directions, and a lightweight CBAM module is introduced into each connection node to improve the feature fusion effect, making the model more accurate in detecting targets of different sizes; Detection head: The P2 / P3 / P4 feature maps are used to detect small objects, and the output of each layer includes classification, confidence and bounding box regression.

[0011] The present invention has the following technical effects: This invention uses a ZED binocular camera combined with a YOLO11 deep learning model, which not only improves image recognition accuracy but also acquires the three-dimensional spatial location information of weeds in real time, solving the problem of accurately locating weeds in existing systems.

[0012] By using depth maps and 3D point cloud data, the coordinates of weeds can be accurately obtained, enabling the laser system to move and irradiate precisely in the X, Y, and Z directions, thereby effectively avoiding damage to crops and improving the accuracy and safety of weeding. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a schematic diagram of the XY direction moving platform structure of the present invention.

[0015] The components include: 1. Wheel connecting axle; 2. Mecanum wheel; 3. Frame; 4. Triangular block; 5. Aluminum material; 6. ZED binocular camera; 7. Laser emitter; 8. Laser emitter slider; 9. X-axis stepper motor; 10. Electric cylinder; 11. Electric cylinder telescopic rod; 12. Lead screw; 13. Y-axis stepper motor; 14. Aluminum connecting block; 15. Slide rail; 16. Slide rail slider; 17. Wheel connecting beam. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Reference Figures 1 to 2 As shown, this embodiment provides a deep learning-based vision-based laser weeding device for farmland, including a frame 3, Mecanum wheels 2 installed at the four corners of the frame 3, a lifting component installed on the frame 3, an XY direction movable component installed at the movable end of the lifting component, a laser emitter 7 installed at the movable end of the XY direction movable component, a binocular image acquisition device installed on the fixed part of the XY direction movable component, and a data processing module electrically connected to the binocular image acquisition device, the XY direction movable component, and the laser emitter 7.

[0019] The scheme is further optimized. The lifting component includes four electric cylinders 10. The electric cylinders 10 are set vertically and are mounted on the frame 3. The XY direction movable component is installed on the movable end of the electric cylinder 10. The electric cylinder 10 is electrically connected to the data processing module.

[0020] The scheme is further optimized. The movable parts in the XY direction include an aluminum frame 5, a slide rail slider 16 is slidably mounted on the aluminum frame 5, a lead screw 12 is threadedly connected to the slide rail slider 16, a Y-direction stepper motor 13 is shaft-connected to the lead screw 12, the Y-direction stepper motor 13 is fixedly connected to the aluminum frame 5, a slide rail 15 is fixedly connected to the slide rail slider 16, and an X-direction stepper motor 9 is mounted on the slide rail 15 to control the movement of the laser emitter slider 8.

[0021] The frame 3 and the electric cylinder 10 are reinforced and fixed by the triangular block 4. The top surface of the electric cylinder telescopic rod 11 of the electric cylinder 10 is fixedly connected to the aluminum connecting block 14. The aluminum connecting block 14 is fixedly connected to the aluminum frame 5. The Mecanum wheel 2 is connected to the wheel connecting beam 17 for steering control.

[0022] The solution was further optimized, and the binocular image acquisition device was the ZED binocular camera 6.

[0023] A deep learning-based visual laser weed control method for farmland, implemented using the aforementioned device, specifically includes the following steps: 2D images, depth maps, and 3D point cloud images of farmland are acquired using a binocular image acquisition device; Set up a YOLO11 model within the data processing module; The data processing module identifies weeds and their locations in the farmland after analyzing the 2D images, depth maps, and 3D point cloud images using the YOLO11 model. The data processing module controls the operation of the lifting components and the XY-direction moving components, which drives the laser emitter 7 to move to a suitable position and clear the weeds through the laser emitter 7.

[0024] Further optimization of the scheme: the information collected by the binocular image acquisition device includes the left eye image and the right eye image. The image with the largest display target area in the left eye image and the right eye image is used as the 2D image for YOLO11 model analysis. The output includes the coordinate information of the two-dimensional bounding box and the weed category information. Further optimizations were made by replacing the standard convolutional structure in the YOLO11 model with a Ghost convolutional module and adding a channel spatial attention mechanism to both the backbone and neck networks.

[0025] Further optimization of the scheme, the YOLO11 model is as follows: Input image size: 512×512 Backbone network: The first three layers use GhostConv output channels of 32, 64, and 128 respectively; A CBAM module is inserted in the middle of the first two and third layers, and the CBAM module is inserted in the middle of the third and fourth layers. Downsampling uses the GhostConv module with a stride of 2 to replace the traditional max pooling (MaxPool). The input is the output feature map of each layer in the backbone network. Convolution is used to reduce the spatial size and enhance the feature learning ability. Neck structure: Information is extracted from feature maps at different levels, and upper and lower FPNs and lower and upper PANs are performed. Information is fused in two directions, and a lightweight CBAM module is introduced into each connection node to improve the feature fusion effect, making the model more accurate in detecting targets of different sizes; Detection head: The P2 / P3 / P4 feature maps are detected to perform small object detection, and the output of each layer includes classification, confidence and bounding box regression.

[0026] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0027] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A laser weeding method for farmland based on deep learning vision, characterized in that, The device includes a laser weeding device for farmland, the device includes a frame (3), Mecanum wheels (2) are installed at the four corners of the frame (3), a lifting component is installed on the frame (3), an XY direction moving component is installed on the movable end of the lifting component, a laser emitter (7) is installed on the movable end of the XY direction moving component, a binocular image acquisition device is installed on the fixed part of the XY direction moving component, and the binocular image acquisition device, the XY direction moving component, and the laser emitter (7) are electrically connected to a data processing module; The lifting component includes four electric cylinders (10), the electric cylinders (10) are vertically arranged, the electric cylinders (10) are mounted on the frame (3), the XY direction movable component is mounted on the movable end of the electric cylinder (10), and the electric cylinder (10) is electrically connected to the data processing module; The XY-direction movable component includes an aluminum frame (5), on which a slide rail slider (16) is slidably mounted. The slide rail slider (16) is threadedly connected to a lead screw (12), and the lead screw (12) is axially connected to a Y-direction stepper motor (13). The Y-direction stepper motor (13) is fixedly connected to the aluminum frame (5). A slide rail (15) is fixedly connected to the slide rail slider (16), and an X-direction stepper motor (9) for controlling the movement of the laser emitter slider (8) is mounted on the slide rail (15). The laser weeding method for farmland includes the following steps: The binocular image acquisition device acquires 2D images, depth maps, and 3D point cloud images of farmland. A YOLO11 model is set up within the data processing module; The data processing module identifies the weeds and their locations in the farmland after analyzing the 2D image, depth map, and 3D point cloud image of the farmland using the YOLO11 model. The data processing module controls the operation of the lifting component and the XY direction movable component, driving the laser emitter (7) to move to a suitable position, and clearing weeds through the laser emitter (7); The information acquired by the binocular image acquisition device includes a left-eye image and a right-eye image. The image with the largest display target area in the left-eye image and the right-eye image is used as the 2D image output for the YOLO11 model analysis, which includes two-dimensional bounding box coordinate information and weed category information. In the YOLO11 model, the standard convolutional structure is replaced with a Ghost convolutional module, and a channel spatial attention mechanism is added to it. The channel spatial attention mechanism is added to both the backbone network and the neck network. The YOLO11 model is as follows: Input image size: 512×512 Backbone network: The first three layers use GhostConv output channels of 32, 64, and 128 respectively; A CBAM module is inserted in the middle of the first two and third layers, and the CBAM module is inserted in the middle of the third and fourth layers. Downsampling uses the GhostConv module with a stride of 2 to replace the traditional max pooling. The input is the output feature map of each layer in the backbone network. Convolution is used to reduce the spatial size and enhance the feature learning ability. Neck structure: Information is extracted from feature maps at different levels, and upper and lower FPNs and lower and upper PANs are performed. Information is fused in two directions, and a lightweight CBAM module is introduced into each connection node to improve the feature fusion effect, making the model more accurate in detecting targets of different sizes; Detection head: The P2 / P3 / P4 feature maps are used to detect small objects, and the output of each layer includes classification, confidence and bounding box regression.

2. The laser weeding method for farmland based on deep learning vision according to claim 1, characterized in that, The binocular image acquisition device is a ZED binocular camera (6).

Images