Farmland weed laser weeding device and weeding method based on deep learning vision

Abstract

The invention relates to the technical field of agricultural machinery, in particular to a farmland weed laser weeding device based on deep learning vision and a weeding method.The farmland weed laser weeding device comprises a vehicle frame, Mecanum wheels are installed at the four corners of the vehicle frame, a lifting component is installed on the vehicle frame, and an XY-direction movable component is installed at the movable end of the lifting component; a laser transmitter is installed at the movable end of the XY-direction movable component, a binocular image collecting device is installed on the fixed part of the XY-direction movable component, and the binocular image collecting device, the XY-direction movable component and the laser transmitter are electrically connected with a data processing module. According to the method, the binocular stereoscopic vision technology, the YOLO11 deep learning model and the laser weeding technology are combined, and weeds can be accurately recognized and positioned.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of agricultural machinery, in particular to a laser weeding device and method for visual farmland weeds based on deep learning. BACKGROUND

[0002] Traditional farmland weeding methods mainly rely on manual weeding and chemical weeding, but manual weeding is low in efficiency and high in labor intensity; secondly, although chemical weeding is high in efficiency, it is easy to cause environmental pollution and pesticide residues, and harm human health and the ecological environment. In recent years, with the rapid development of machine vision and artificial intelligence technology, intelligent weeding technology based on machine vision has become a research hotspot. However, most of the existing machine vision weeding technologies adopt a monocular camera, which is difficult to accurately obtain the three-dimensional information of weeds, resulting in low weeding precision and easy damage to crops. SUMMARY

[0003] The application aims to provide a laser weeding device and method for visual farmland weeds based on deep learning, which solves the above problems. The application combines binocular stereo vision technology, YOLO11 deep learning model and laser weeding technology, can accurately identify and locate weeds, and use laser for accurate weeding, improve weeding efficiency and precision, reduce harm to the environment and crops, and at the same time, through the combination of deep learning and binocular stereo matching technology, different types and forms of weeds are identified and distinguished, and finally different power lasers are used to remove weeds, protect crop seedlings and prevent them from being damaged.

[0004] To achieve the above purpose, the application provides the following scheme: A laser weeding device for visual farmland weeds based on deep learning, comprising a vehicle frame, a Mecanum wheel is installed at each corner of the vehicle frame, a lifting component is installed on the vehicle frame, an XY direction movable component is installed at the movable end of the lifting component, a laser emitter is installed at the movable end of the XY direction movable component, a binocular image acquisition device is installed at the fixed part of the XY direction movable component, and the binocular image acquisition device, the XY direction movable component and the laser emitter are electrically connected with a data processing module.

[0005] Preferably, the lifting component comprises four electric cylinders, the electric cylinders are vertically arranged, the electric cylinders are installed on the vehicle frame, the XY direction movable component is installed at the movable end of the electric cylinders, and the electric cylinders are electrically connected with the data processing module.

[0006] Preferably, the XY direction movable component comprises the aluminum material frame, a sliding rail slider is slidingly installed on the aluminum material frame, a lead screw is threadedly connected to the sliding rail slider, a Y direction stepper motor is connected to the lead screw, the Y direction stepper motor is fixedly connected to the aluminum material frame, a sliding rail is fixedly connected to the sliding rail slider, and an X direction stepper motor for controlling the movement of the laser emitter slider is installed on the sliding rail.

[0007] Preferably, the binocular image acquisition device is a ZED binocular camera.

[0008] A deep learning-based visual farmland weed laser weeding method, which is implemented based on the device of the foregoing scheme, specifically comprises the following steps: A 2D image, a depth map and a three-dimensional point cloud image of the farmland are acquired by the binocular image acquisition device; A YOLO11 model is arranged in the data processing module; The data processing module identifies weeds and position information in the farmland after analyzing the 2D image, the depth map and the three-dimensional point cloud image of the farmland by the YOLO11 model; The data processing module controls the lifting component and the XY direction movable component to move the laser emitter to a suitable position, and the laser emitter removes weeds.

[0009] Preferably, the information acquired by the binocular image acquisition device includes a left eye image and a right eye image, one of which with the largest display target area is used as the 2D image output two-dimensional convenient frame coordinate information and weed category information for the YOLO11 model analysis; Preferably, the standard convolution structure in the YOLO11 model is replaced by a Ghost convolution module, and a channel spatial attention mechanism is added to the Ghost convolution module, and the channel spatial attention mechanism is added to the backbone network and the neck network, respectively.

[0010] Preferably, the improved YOLO11 model is as follows: Input image size: 512x512 Backbone network: The first three layers adopt GhostConv, and the output channels are 32, 64 and 128, respectively; The CBAM module is inserted into the middle part of the first two layers and the third layer, and the CBAM module is inserted into the middle part of the third layer and the fourth layer; Downsampling adopts a GhostConv module with a step of 2 to replace the traditional MaxPool, and the input is the output feature map of each layer in the backbone network, and the spatial size is reduced and the feature learning ability is enhanced through convolution; Neck structure: Extract information from different levels of feature maps, up and down, and down up; Two-way information fusion, each connection node introduces a lightweight CBAM module, improves feature fusion effect, makes the model more accurate when detecting different size targets; Detection head: Detect P2 / P3 / P4 three-layer feature maps, detect small targets, and each layer output includes classification, confidence and boundary box regression.

[0011] The present application has the following technical effects: The present application adopts ZED binocular camera, combined with YOLO11 deep learning model, not only improves the image recognition accuracy, but also can obtain the three-dimensional spatial position information of weeds in real time, solves the problem of difficult accurate positioning of weeds in the existing system.

[0012] Through the depth map and 3D point cloud data, the coordinates of the weeds are accurately obtained, the laser system is accurately moved and irradiated in X, Y and Z directions, so as to effectively avoid injuring crops and improve the precision and safety of weeding. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Fig. 1 The present application is a structural schematic diagram; Fig. 2 The present application is a structural schematic diagram of the XY direction moving platform.

[0015] 1, wheel connecting shaft; 2, Mecanum wheel; 3, frame; 4, triangular block; 5, aluminum material; 6, ZED binocular camera; 7, laser emitter; 8, laser emitter slider; 9, X direction stepping motor; 10, electric cylinder; 11, electric cylinder telescopic rod; 12, lead screw; 13, Y direction stepping motor; 14, aluminum material connecting block; 15, slide rail; 16, slide rail slider; 17, wheel connecting beam. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0017] In order to make the above objectives, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0018] Referring to Figs. 1-2 The embodiment provides a laser weeding device for visual farmland weeds based on deep learning, which comprises a vehicle frame 3, a Mecanum wheel 2 installed at the four corners of the vehicle frame 3, a lifting component installed on the vehicle frame 3, an XY-direction moving component installed at the movable end of the lifting component, a laser emitter 7 installed at the movable end of the XY-direction moving component, a binocular image acquisition device installed at the fixed part of the XY-direction moving component, and a data processing module electrically connected with the binocular image acquisition device, the XY-direction moving component and the laser emitter 7.

[0019] Further optimization scheme, the lifting component comprises four electric cylinders 10, the electric cylinders 10 are vertically arranged, the electric cylinders 10 are installed on the vehicle frame 3, the XY-direction moving component is installed at the movable end of the electric cylinders 10, and the electric cylinders 10 are electrically connected with the data processing module.

[0020] Further optimization scheme, the XY-direction moving component comprises an aluminum frame 5, a sliding rail slider 16 slidably installed on the aluminum frame 5, a lead screw 12 threadedly connected with the sliding rail slider 16, a Y-direction stepping motor 13 shaft-connected with the lead screw 12, the Y-direction stepping motor 13 fixedly connected with the aluminum frame 5, a sliding rail 15 fixedly connected with the sliding rail slider 16, and an X-direction stepping motor 9 installed on the sliding rail 15 and used for controlling the movement of a laser emitter slider 8.

[0021] The vehicle frame 3 and the electric cylinders 10 are reinforced and fixed through a triangular block 4, an electric cylinder telescopic rod 11 of the electric cylinder 10 is fixedly connected with an aluminum connecting block 14, the aluminum connecting block 14 is fixedly connected with the aluminum frame 5, and a wheel connecting beam 17 is connected with the Mecanum wheel 2 and used for controlling steering.

[0022] Further optimization scheme, the binocular image acquisition device is a ZED binocular camera 6.

[0023] A laser weeding method for visual farmland weeds based on deep learning, which is implemented based on the device described in the foregoing scheme and specifically comprises the following steps: Obtaining a 2D image, a depth map and a three-dimensional point cloud image of a farmland through a binocular image acquisition device; Setting a YOLO11 model in a data processing module; The data processing module identifies weeds and position information in the farmland after analyzing the 2D image, the depth map and the three-dimensional point cloud image of the farmland through the YOLO11 model; The data processing module controls the operation of the lifting component and the XY direction moving component, drives the laser emitter 7 to move to a suitable position, and removes weeds through the laser emitter 7.

[0024] In a further optimization scheme, the information collected by the binocular image acquisition device includes a left eye image and a right eye image, wherein one image with the largest display target area in the left eye image and the right eye image is used as a 2D image for YOLO11 model analysis to output two-dimensional convenient frame coordinate information and weed category information. In a further optimization scheme, the standard convolution structure in the YOLO11 model is replaced by a Ghost convolution module, and a channel spatial attention mechanism is added therein.

[0025] In a further optimization scheme, the improved YOLO11 model is as follows: Input image size: 512*512 Backbone of the main network: The first three layers adopt GhostConv, and the output channels are 32, 64 and 128 respectively; The CBAM module is inserted in the middle part of the first two layers and the third layer, and the CBAM module is inserted in the middle part of the third layer and the fourth layer; Downsampling adopts a GhostConv module with a step of 2 to replace the traditional MaxPool, and the input is the output feature map of each layer in the main network, and the spatial size is reduced and the feature learning ability is enhanced through convolution; Neck structure: Information is extracted from feature maps at different levels, and up-down FPN and down-up PAN are performed; Information fusion in two directions, a lightweight CBAM module is introduced in each connection node to improve the feature fusion effect, so that the model is more accurate when detecting different size targets; Head: Detect P2 / P3 / P4 three-layer feature maps to detect small targets, and each layer output includes classification, confidence and boundary box regression.

[0026] In the description of the present application, it should be understood that the orientations or positional relationships indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore cannot be understood as a limitation on the present application.

[0027] The above described embodiments are only to illustrate the preferred modes of the present application, and are not intended to limit the scope of the present application. Any modification and improvement made by those skilled in the art to the technical solutions of the present application without departing from the design spirit of the present application shall fall within the protection scope of the present application as defined by the claims.

Claims

1. A laser weeding device for farmland based on deep learning vision, characterized in that, The vehicle includes a frame (3), with Mecanum wheels (2) installed at the four corners of the frame (3). A lifting component is installed on the frame (3), and an XY direction moving component is installed on the movable end of the lifting component. A laser emitter (7) is installed on the movable end of the XY direction moving component. A binocular image acquisition device is installed on the fixed part of the XY direction moving component. The binocular image acquisition device, the XY direction moving component, and the laser emitter (7) are electrically connected to a data processing module.

2. The laser weeding device for farmland based on deep learning vision according to claim 1, characterized in that, The lifting component includes four electric cylinders (10), which are vertically arranged and mounted on the frame (3). The XY direction movable component is mounted on the movable end of the electric cylinder (10), and the electric cylinder (10) is electrically connected to the data processing module.

3. The laser weeding device for farmland based on deep learning vision according to claim 1, characterized in that, The XY-direction movable component includes the aluminum frame (5), a slide rail slider (16) slidably mounted on the aluminum frame (5), a lead screw (12) threadedly connected to the slide rail slider (16), a Y-direction stepper motor (13) axially connected to the lead screw (12), the Y-direction stepper motor (13) fixedly connected to the aluminum frame (5), a slide rail (15) fixedly connected to the slide rail slider (16), and an X-direction stepper motor (9) for controlling the movement of the laser emitter slider (8) mounted on the slide rail (15).

4. The laser weeding device for farmland based on deep learning vision according to claim 1, characterized in that, The binocular image acquisition device is a ZED binocular camera (6).

5. A laser weeding method for farmland based on deep learning vision, characterized in that, The method is implemented based on the apparatus according to any one of claims 1-4, and specifically includes the following steps: The binocular image acquisition device acquires 2D images, depth maps, and 3D point cloud images of farmland. A YOLO11 model is set up within the data processing module; The data processing module identifies the weeds and their locations in the farmland after analyzing the 2D image, depth map, and 3D point cloud image of the farmland using the YOLO11 model. The data processing module controls the operation of the lifting component and the XY direction moving component, driving the laser emitter (7) to move to a suitable position, and clearing weeds through the laser emitter (7).

6. The laser weeding device for farmland based on deep learning vision according to claim 5, characterized in that, The information acquired by the binocular image acquisition device includes a left-eye image and a right-eye image. The image with the largest display target area in the left-eye image and the right-eye image is used as the 2D image output for the YOLO11 model analysis, which includes two-dimensional bounding box coordinate information and weed category information.

7. The laser weeding device for farmland based on deep learning vision according to claim 5, characterized in that, In the YOLO11 model, the standard convolutional structure is replaced with a Ghost convolutional module, and a channel spatial attention mechanism is added to it. The channel spatial attention mechanism is added to both the backbone network and the neck network.

8. The laser weeding device for farmland based on deep learning vision according to claim 7, characterized in that, The improved YOLO11 model is as follows: Input image size: 512×512 Backbone network: The first three layers use GhostConv output channels of 32, 64, and 128 respectively; A CBAM module is inserted in the middle of the first two and third layers, and the CBAM module is inserted in the middle of the third and fourth layers. Downsampling uses the GhostConv module with a stride of 2 to replace the traditional max pooling (MaxPool). The input is the output feature map of each layer in the backbone network. Convolution is used to reduce the spatial size and enhance the feature learning ability. Neck structure: Information is extracted from feature maps at different levels, and up-down (FPN) and down-up (PAN) operations are performed. Information is fused in two directions, and a lightweight CBAM module is introduced into each connection node to improve the feature fusion effect, making the model more accurate in detecting targets of different sizes; Detection head: The P2 / P3 / P4 feature maps are detected to perform small object detection, and the output of each layer includes classification, confidence and bounding box regression.

Images