A smart, real-time, dynamic, and precise laser weeding method and device

Abstract

This invention discloses an intelligent, real-time, dynamic, and precise laser weeding method and device, relating to the field of laser weeding. The weeding device includes an image acquisition module, a processor, a laser reflector, a microcontroller control module, an IMU sensor, and a high-precision encoder. The image acquisition module acquires images of crops and weeds in the field. The processor processes the acquired crop and weed images based on the LW-YOLO model, extracting the coordinate information of the weed positions and simultaneously optimizing the laser scanning path. The laser reflector controls the laser beam to irradiate the target location by rotating a laser mirror. The microcontroller control module controls the phase voltage of the brushless motor and transmits commands from the processor to the sensors and actuators, improving motor positioning accuracy through an FOC algorithm. This invention significantly improves weeding accuracy and efficiency, reduces waiting time during static weeding, and possesses advantages of high precision and high efficiency.

Description

Technical Field

[0001] This invention relates to the field of laser weeding technology, specifically to an intelligent, real-time, dynamic, and precise laser weeding method and device. Background Technology

[0002] Laser weeding is a non-contact weeding technology that uses a high-energy laser beam to irradiate weeds, causing thermal damage to the weed cells until they die. Compared with traditional mechanical and chemical weeding methods, laser weeding can achieve precise weed control, which not only reduces disturbance to the soil and crop roots, but also avoids the impact of chemical residues on the environment and food safety, and reduces the risk of herbicide resistance, meeting the needs of modern green and sustainable agricultural development.

[0003] However, most current laser weeding technologies operate in a static manner, requiring crops and weeds to be in a relatively static state for treatment. As a result, the efficiency of a single operation is low, making it difficult to meet the needs of large-scale field operations. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides an intelligent, real-time, dynamic, and precise laser weeding method and device. This invention employs dynamic laser weeding technology, combined with a self-designed LW-YOLO model, to achieve precise and rapid dynamic laser weeding of weeds detected during field movement, thereby improving operational efficiency and reducing energy consumption.

[0005] The technical solution adopted by this invention to solve its technical problem is as follows:

[0006] Intelligent, real-time, dynamic, and precise laser weed control methods, including

[0007] Step 1, Image Acquisition and Processing

[0008] Collect raw crop and weed images, perform image preprocessing and target labeling on the collected raw images, and randomly divide the labeled images and labels into training set and validation set according to the proportion;

[0009] The divided image data is enhanced separately for the training set and the validation set. The enhancement methods include random rotation, random horizontal and vertical mirror flipping, random brightness change, random contrast change, and adding Gaussian noise. All images and labels are enhanced in sequence using these five methods.

[0010] Step 2: Establish an LW-YOLO model for weed target identification;

[0011] The LW-YOLO model is based on the YOLO11n model and is an improved version of the YOLO11n model.

[0012] A small target detection head is added to the output layer of the YOLO11n model undersampling, and an efficient multi-scale attention mechanism EMA is added to the layer before each detection head;

[0013] The ADown convolution module is used to replace the Backbone downsampling module;

[0014] The convolutional layer of the residual block Bottleneck in the feature extraction module C3K2 is replaced with a partial convolution (PConv).

[0015] Step 3, obtain the coordinates of the weed location.

[0016] Upon receiving the command to start weeding, the processor sends a power-on instruction to the image acquisition module and the microcontroller.

[0017] The microcontroller will initialize the IMU sensor and high-precision encoder, and control the weeding device to move towards the work area;

[0018] The image acquisition module acquires raw images of crops and weeds and sends them to the processor. The processor then uses the LW-YOLO model to recognize the received raw images.

[0019] The LW-YOLO model dynamically extracts keyframes from the video stream and outputs the pixel-level position coordinates of the centroid of weeds in the x_mirror coordinate system by recalibrating the feature space.

[0020] The method for obtaining the pixel-level position coordinates of the centroid of the weeds is as follows:

[0021] Weed detection is performed using the LW-YOLO model, and the output weed detection boxes are cropped. Image processing is then applied to the resulting weed images to create a connected component for each weed detection box. The centroid pixel coordinates of each weed are then calculated using the following formula:

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] Where M and N represent the rows and columns of the mask, respectively. The mask image is at coordinate points The gray value at that location, for a binary image It is 0 or 1. Represents the zeroth moment of the image. Represents the first 1st moment of the image. This represents the second first-order moment of the image. The centroid pixel coordinates of the connected domain;

[0029] The pixel-level position coordinates of the centroid of the weeds are sequentially transformed, and the nine-point calibration method is used to complete the coordinate transformation from the camera coordinate system to the laser reflector x_mirror coordinate system;

[0030] The processor adds identifier keywords and timestamps to the x and y coordinates of the weeds, packages the data, and sends it to the microcontroller via serial port 1.

[0031] Step 4: Microcontroller and Processor Data Processing

[0032] The microcontroller synchronously receives the weed position coordinates and pose data from the processor and IMU sensor, performs average filtering on each data set in groups of five, and stores the results; the microcontroller also acquires the absolute angle data of the brushless motor provided by the high-precision encoder and performs similar processing.

[0033] Among them, the pose data from the IMU sensor needs to be converted into actual motion speed through integration calculation, and then averaged and filtered.

[0034] The microcontroller extracts the processed position and velocity data at a preset frequency, performs data fusion using the Kalman filter algorithm, and after calculating the high-precision weed coordinates, the microcontroller sends the weed coordinates back to the processor via serial port 2. The processor converts the weed coordinates into three-dimensional spatial coordinates in the camera coordinate system through inverse coordinate transformation, and generates reference data by fusing confidence parameters and timestamp information. Based on this data, the RRT-MPC fusion algorithm is started, and dynamic path decision is made by combining real-time pose information and historical trajectory features to generate a laser scanning path that meets the crop obstacle avoidance requirements and includes displacement prediction compensation. Finally, the motion command is transmitted to the microcontroller via serial port 3.

[0035] Step 5, Precision Dynamic Laser Weed Control

[0036] After receiving the planned coordinates, the microcontroller uses the FOC control algorithm to fuse the corrected target angle with the motor angle data fed back by the encoder, calculates the digital phase voltage of the brushless motor, and achieves precise control of the brushless motor rotation, thereby adjusting the laser reflector to the target angle.

[0037] The microcontroller sends an activation command to the laser, causing the laser beam to enter the laser reflector. After being deflected by the laser reflector, it is precisely projected onto the center of the weeds. At the same time, a dynamic compensation mechanism is embedded in the laser projection process. By fusing attitude data and visual coordinate feedback, the projection deviation caused by mechanical vibration and visual processing delay is eliminated in real time, maintaining the millimeter-level dynamic matching accuracy between the laser irradiation center and the center of the weeds.

[0038] Repeat the above steps until the entire weeding operation is completed.

[0039] Furthermore, throughout the entire operation, the microcontroller controls the weeding device to move forward at a constant speed of 5cm / s, and the speed can be adjusted to the range of 1-10cm / s according to the operation requirements.

[0040] Furthermore, the cropping of the weed detection boxes output by the model and the image processing operation on the resulting weed image include:

[0041] Separate the three channels r, g, b of the weed image and calculate the supergreen index. And normalize it to the interval [0, 255];

[0042] The weed image was converted to grayscale using the 2g-rb color feature factor, and the Gaussian weighted average method in the automatic thresholding segmentation method was used for automatic thresholding segmentation of the grayscale image.

[0043] Set a minimum area threshold for the segmented binary mask to filter out smaller contours;

[0044] The contour obtained after area filtering is conditionally judged. If there is only one connected region, the area of ​​the connected region is directly calculated, and its centroid is calculated using a second-order matrix to obtain the centroid pixel coordinates of the weeds, which are used for subsequent coordinate transformation.

[0045] For cases where two or more connected components remain after area filtering, calculate the centroid of each connected component and the coordinates of the center of the weed detection box, so that each connected component extends along the direction from its centroid to the center of the weed detection box.

[0046] After extension processing, multiple connected components are merged into one connected component, and its centroid coordinates are calculated.

[0047] Furthermore, the method of using Gaussian weighted average in the automatic thresholding segmentation method is used to perform automatic thresholding segmentation on the grayscale image; the calculation steps are as follows:

[0048] Define a Gaussian kernel, which is a k×k matrix G whose elements... Calculated from a two-dimensional Gaussian function;

[0049] Calculate the weighted average for each pixel. Calculate the weighted average of gray values ​​within its domain. ;

[0050] Calculate the final threshold, which can be adjusted by subtracting a constant from the weighted average;

[0051] Apply threshold segmentation for each pixel ,if If the value is true, the pixel is set as the target foreground (white); otherwise, it is set as the background (black).

[0052] The obtained binary mask is subjected to morphological processing. Using an elliptical kernel, closing and opening operations are performed sequentially to fill the holes in the connected components and eliminate noise, thus obtaining the final binary mask.

[0053] Furthermore, the extended processing method includes:

[0054] First, draw the line connecting the centroid of each connected component to the center of the detection box, obtain the intersection point of the line and the contour of the connected component, and extend the line from the intersection point until the center of the detection box.

[0055] The extension width is 1 / 6 of the maximum diameter of each connected region along the direction vector;

[0056] The method for calculating the maximum diameter of each connected component is further explained by rotating the image so that the direction vector is aligned with the horizontal axis. In the rotated image coordinate system, the connected components are scanned along the direction perpendicular to the direction vector to find the maximum width. Based on the scanning results, the maximum width in that direction is calculated.

[0057] After extension, the case of multiple connected domains will be transformed into a single connected domain, and the centroid coordinates of this connected domain will be calculated.

[0058] In order to solve its technical problem, the present invention also adopts the following technical solutions:

[0059] Intelligent, real-time, dynamic, precision laser weed control device, including

[0060] The image acquisition module is used to acquire images of crops and weeds in the field;

[0061] The processor processes the acquired crop and weed images based on the LW-YOLO model, extracts the weed location coordinates, so as to realize laser weeding in the future, and optimizes the laser scanning path.

[0062] A laser reflector is used to control the laser beam to the target position by rotating a laser reflector, thereby achieving laser weed removal on a plane 500-600 mm away from the laser source.

[0063] IMU sensor, used to measure the pose data of the weeding device;

[0064] A high-precision encoder is used to provide feedback on the angular position information of the brushless motor of the laser reflector.

[0065] The microcontroller control module controls the phase voltage of the brushless motor and transmits commands from the processor to the sensors and actuators, while also receiving feedback data from the sensors. It also uses a Kalman filter algorithm to calculate the weed position coordinates and the weeding device pose data, compensates for the time delay error of the vision system and the attitude deviation caused by mechanical vibration, and constructs a displacement compensation closed loop through the FOC algorithm, thereby improving the motor positioning accuracy and ensuring the realization of dynamic laser weeding.

[0066] Furthermore, the image acquisition module includes a high-resolution camera and an image processing unit for real-time acquisition and processing of field images.

[0067] Furthermore, a laser head mounting bracket is provided on the right side of the outer shell of the laser reflector, and a laser reflecting device is provided on the left side and the upper side inside the reflector.

[0068] The two laser reflection devices include:

[0069] Brushless motor, laser reflector mounted thereon, and reflector mounting bracket;

[0070] The brushless motor is used to control the rotation angle of the laser reflector in order to regulate the optical path of the laser beam.

[0071] The laser reflector includes a laser reflector x_mirror and a laser reflector y_mirror. The laser reflector x_mirror is responsible for deflecting the laser beam along the x-axis, while the laser reflector y_mirror deflects the laser beam along the y-axis. The reflecting surfaces of the laser reflector and their respective connected brushless motor rotating shafts are kept coplanar.

[0072] The housing is used to mount the brushless motor and laser head mounting bracket, and is installed on the weeding device.

[0073] Furthermore, the right side of the laser reflector housing is provided with a T-shaped guide groove for adjusting the horizontal position of the laser head fixing bracket, so as to facilitate the alignment of the laser beam with the center of the laser reflector x_mirror.

[0074] Furthermore, the right side of the laser reflector housing is provided with a set of through bolt holes arranged in a ring, and a bearing seat hole is provided on the opposite side at the same position, for fixing the relative position of the laser head mounting bracket and the laser reflector housing, and for adjusting the axial position of the bearing and bearing seat on the other side.

[0075] Furthermore, a quartz glass plate is disposed on the right side inside the housing of the laser reflector. The center of the glass plate coincides with the axis of the laser head fixing frame. The glass plate is 10 mm long, 10 mm wide, and 2 mm thick. It is used to isolate external dust and debris and to allow the laser beam to pass through.

[0076] Furthermore, a cylindrical strong magnet is installed at the tail of the brushless motor. The magnet can rotate with the motor shaft. A high-precision encoder is installed 2 mm away from the surface of the magnet. The encoder position is fixed and it uses the SPI communication protocol to interact with the microcontroller. The brushless motor, high-precision encoder and magnet are encased in a shell.

[0077] Furthermore, the IMU sensor is used to measure the motion state of the weeding device, including velocity and acceleration information in the x, y, and z axes.

[0078] Compared with the prior art, the present invention can bring the following beneficial effects:

[0079] This invention integrates image acquisition, target recognition, path planning, and precision targeting technologies. It utilizes a high-resolution camera to capture images of crops and weeds in a field and employs an improved LW-YOLO model for real-time detection. Lightweight optimization enhances the model's recognition accuracy and speed. A 3D model of a laser reflector is designed and optimized. The laser reflector is driven by dual brushless motors, enabling precise positioning in two dimensions. Simultaneously, an IMU sensor and a high-precision encoder improve targeting accuracy. A Kalman filter algorithm and a composite control system compensate for motion vibration and displacement errors. A microcontroller controls the laser reflector, combining RRT and MPC algorithms to optimize the laser path, and an FOC algorithm to improve motor positioning accuracy. This method significantly improves weeding accuracy and efficiency, reduces waiting time during static weeding, and offers advantages of high precision and high efficiency. Attached Figure Description

[0080] To more clearly illustrate the embodiments of this invention or the technical solutions in the prior art, the following is a brief introduction to the drawings used in the embodiments. The drawings described below are only some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0081] Figure 1 This is a schematic diagram of the laser dynamic weeding device in Example 1;

[0082] Figure 2 This is a schematic diagram of the internal structure of the laser reflector in Example 1 (right side view).

[0083] Figure 3 This is a schematic diagram of the internal structure of the laser reflector in Example 1 (left side view).

[0084] Figure 4 This is a schematic diagram of the dynamic weeding process in Example 5;

[0085] Figure 5 This is a schematic diagram of the weed identification and location process in Example 2;

[0086] Figure 6 This is the grayscale image of the original image processed using the Super Green Index as described in Example 2;

[0087] Figure 7 The grayscale image is a binary image that has undergone morphological processing.

[0088] Figure 8 This is a schematic diagram of the calculated centroid of the binary image;

[0089] Figure 9 This is a flowchart of the centroid processing in the case of multiple connected domains in Example 2;

[0090] Figure 10 This is a flowchart illustrating a complete identification and positioning process in Example 2. Detailed Implementation

[0091] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings.

[0092] It should be noted that the embodiments described are only some embodiments of the present invention, and not all embodiments. All other embodiments based on the present invention, obtained by those skilled in the art without inventive effort, are within the scope of protection of the present invention.

[0093] To make the above-mentioned objectives, features and advantages of the present invention clearer and easier to understand, the present invention will be further explained and described in detail below with reference to the accompanying drawings and specific embodiments, using lettuce as a representative crop.

[0094] Example 1

[0095] An intelligent real-time dynamic precision laser weeding machine includes an image acquisition module 1, a processor 2, a laser reflector 3, a microcontroller 4, an IMU sensor and a high-precision encoder, a weeding machine 7, and a laser 8.

[0096] Reference Figures 1 to 3 The image acquisition module 1 consists of a high-resolution camera and an image processing unit, used to acquire real-time images of lettuce and weeds in the field and transmit them back to the processor 2;

[0097] Processor 2 uses the trained LW-YOLO model to process the acquired images of lettuce and weeds, obtains the coordinates of the weeds, and transforms them from the camera coordinate system to the laser reflector coordinate system. Simultaneously, it transforms the weed coordinates corrected by microcontroller 4 back from the laser reflector coordinate system to the camera coordinate system, and performs laser scanning shortest path planning in this coordinate system. The path calculation uses a hybrid approach combining the RRT algorithm and the MPC algorithm. Laser reflector 3 controls the laser emission to strike the target location by deflecting the laser reflector at an angle of 500-600 mm. mm; IMU sensor, used to detect the pose data of the weeder 7 and transmit it to the microcontroller 4, including velocity and acceleration information in the x, y, and z axes; high-precision encoder, used to detect the angular position information of brushless motors 32 and 33 in the laser reflector 3; microcontroller control module, used to control the phase voltage of brushless motors 32 and 33, and transmit the commands issued by the processor 2 to the sensor and actuator, while receiving sensor feedback data; at the same time, Kalman filter algorithm is used to calculate the weed position data and the pose data of the weeder 7, to compensate for the time delay error of the vision system and the posture deviation caused by mechanical vibration, and to build a displacement compensation closed loop through the FOC algorithm to improve the motor positioning accuracy and ensure the realization of dynamic laser weeding;

[0098] Weeder 7 is used to carry the above-mentioned components and perform dynamic weeding operations in the field.

[0099] A T-shaped guide groove is provided on the right side of the laser reflector 3 housing for adjusting the horizontal position of the laser head mounting bracket 5, facilitating the alignment of the laser beam from the laser head 6 with the center of the laser reflector x_mirror 31. A set of circularly arranged through bolt holes is provided on the right side of the laser reflector 3 housing, with bearing seat holes designed on the opposite side at the same location. These holes are used to fix the relative position of the laser head mounting bracket 5 and the laser reflector 3 housing, and to adjust the axial position of the bearing 37 on the other side and its bearing seat. Inside the laser reflector 3 housing, on the right side, a quartz glass plate 38 is placed. The center of the glass plate 38 coincides with the axis of the laser head mounting bracket 5. The glass plate 38 is 10 mm long, 10 mm wide, and 2 mm thick, used to isolate external dust and debris and to allow the laser beam to pass through. Inside the laser reflector 3 housing, on the top and left sides, are two brushless motors 32 and 33, respectively, with a vertical distance of 20 mm between their axes. Laser reflectors are mounted on two brushless motors 32 and 33 respectively. The laser reflector x_mirror 31, which is responsible for deflecting the laser beam in the x-axis direction, is 13 mm long and 10 mm wide, and is mounted by the x_mirror mounting bracket 36. The laser reflector y_mirror 35, which is responsible for deflecting the laser beam in the y-axis direction, is 60 mm long and 20 mm wide, and is mounted and fixed by the y_mirror mounting bracket 34.

[0100] The laser reflector 3 housing comprises four parts. The right side is used to fix the laser head mounting bracket 5 and bearing 37. The left side is used to fix the brushless motor 33 that controls the deflection of the laser beam in the y-direction. The bottom is designed with a hollow structure to allow the laser beam to pass through. The top and front and rear are integrally designed with an inverted "U" shape to fix the brushless motor 32 that controls the deflection of the beam in the x-direction. After the four parts are assembled, the whole structure is rectangular, with effective internal dimensions of 105 mm in length, 100 mm in width, and 78 mm in height. A cylindrical strong magnet is installed at the tail of each of the brushless motors 32 and 33. The magnets can rotate with the motor shaft. A high-precision encoder is installed 2 mm away from the surface of the magnet. The encoder position is fixed and it uses the SPI communication protocol to interact with the microcontroller 4. The housing is designed to enclose the motors, high-precision encoders, and magnets.

[0101] Example 2

[0102] The weed identification and location process of an intelligent, real-time, dynamic, and precise laser weeding method, referring to... Figure 5 This method specifically includes:

[0103] Collect raw images containing lettuce and weeds, preprocess and label the collected images, and randomly divide the labeled images and their labels into training and validation sets in an 8:2 ratio.

[0104] The image data in the training and validation sets are enhanced respectively. The enhancement methods include random rotation, random horizontal and vertical mirror flipping, random brightness change, random contrast change, and adding Gaussian noise. The above five enhancement methods are applied to all images and their labels in turn.

[0105] Construct an LW-YOLO model for target recognition of lettuce and weeds;

[0106] Upon receiving the command to start weeding, processor 2 sends a power-on command to image acquisition module 1 and microcontroller 4.

[0107] The microcontroller 4 will wake up the IMU sensor, the high-precision encoder and the weeder 7, and then the weeder 7 will start moving towards the work area;

[0108] Image acquisition module 1 acquires raw images of lettuce and weeds and sends them to processor 2. Processor 2 calls the LW-YOLO model to recognize the received images.

[0109] The LW-YOLO model is used for target detection. Each weed detection box output by the model is cropped, and the cropped weed image is processed to form a connected component within each detection box, thereby calculating the centroid pixel coordinates of each weed.

[0110] Specifically, the LW-YOLO model dynamically extracts keyframes from the video stream and outputs the pixel-level position coordinates of the centroid of the weeds in the x_mirror coordinate system through feature space recalibration;

[0111] The method for obtaining the pixel-level position coordinates of the centroid of the weeds is as follows:

[0112] Weed detection is performed using the LW-YOLO model, and the output weed detection boxes are cropped. Image processing is then applied to the resulting weed images to create a connected component for each weed detection box. The centroid pixel coordinates of each weed are then calculated using the following formula:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] Where M and N represent the rows and columns of the mask, respectively. The mask image is at coordinate points The gray value at that location, for a binary image It is 0 or 1. Represents the zeroth moment of the image. Represents the first 1st moment of the image. This represents the second first-order moment of the image. The centroid pixel coordinates of the connected domain;

[0120] The pixel-level position coordinates of the centroid of the weeds are sequentially transformed, and the coordinate transformation from the camera coordinate system to the laser reflector x_mirror31 coordinate system is completed using the nine-point calibration method.

[0121] Processor 2 adds identifier keywords and timestamps to the x and y coordinates of the weeds, packages the data, and sends it to microcontroller 4 via serial port 1.

[0122] Example 3

[0123] As described in Example 2, the construction of the LW-YOLO model for target recognition of lettuce and weeds includes:

[0124] The model framework was designed and improved based on YOLO11n;

[0125] A small target detection head Xsmall is added to the output layer of the original model with a 4x downsampling, and an efficient multi-scale attention mechanism EMA is added to the layer before each detection head;

[0126] The ADown convolution module is used to replace the Backbone downsampling module;

[0127] Partial convolution (PConv) is used to replace the convolutional layers in the residual block Bottleneck of the feature extraction module C3K2.

[0128] Example 4

[0129] The image processing operation for each cropped weed image as described in Example 2 includes:

[0130] Gaussian filtering and median filtering are applied to the weed images to achieve image smoothing and noise removal.

[0131] The three channels r, g, and b of the weed image are separated, and the three channels are converted to 32-bit floating-point numbers to avoid overflow issues in integer operations; the supergreen index is calculated. Its calculation formula is

[0132] ;

[0133] And The array is normalized to the interval [0, 255] for subsequent processing; the normalization formula is:

[0134] 55;

[0135] in, yes The minimum value in the array. The maximum value, It is the normalized value;

[0136] The weed image was converted to grayscale using the aforementioned 2g-rb color feature factor, such as... Figure 6 As shown, the grayscale image is segmented using the Gaussian weighted average method in the automatic thresholding segmentation method. The calculation steps are as follows:

[0137] ;

[0138] Define a Gaussian kernel, which is a k×k matrix G whose elements are... It is calculated from a two-dimensional Gaussian function; where, These are the coordinates in the Gaussian kernel. It is the standard deviation, used to control the width of the Gaussian function;

[0139] Calculate the weighted average: for each pixel Calculate the weighted average of gray values ​​within its domain. ;

[0140] ;

[0141] Calculate the final threshold: subtract a constant from the weighted average. To adjust and determine the final threshold ;

[0142] ;

[0143] Apply threshold segmentation for each pixel ,if If the value is true, the pixel is set as the target foreground (white); otherwise, it is set as the background (black).

[0144] The obtained binary mask undergoes morphological processing. Using an elliptical kernel, closing and opening operations are performed sequentially to fill holes in connected components and eliminate noise, resulting in the final binary mask, as shown below. Figure 7 As shown.

[0145] The Canny edge detection algorithm is applied to the binary image to refine the boundaries of the weeds and extract their contours;

[0146] Set a minimum area threshold for the contours to filter out smaller contours.

[0147] The contour obtained after area filtering is conditionally evaluated. If there is only one connected component, the area of ​​that connected component is calculated, and its centroid is calculated using a second-order matrix to obtain the centroid pixel coordinates of the weeds. Figure 8 As shown.

[0148] If two or more connected components still exist after area filtering, calculate the centroid of each connected component and the center coordinates of the trimmed weed detection frame. Then, extend each connected component along the direction from its centroid towards the center of the detection frame. Calculate the center coordinates of the detection frame using the following formula.

[0149] ;

[0150] ;

[0151] in,( () represents the coordinates of the top-left corner of the weed detection box output by the model. () represents the coordinates of the lower right corner of the weed detection box.

[0152] To further elaborate on the extension method: draw the line connecting the centroid of each connected component to the center of the detection box, and obtain the intersection point of the line with the contour of the connected component. Extend from the intersection point until the extension reaches the center of the detection box and then stops.

[0153] The extension width is 1 / 6 of the maximum diameter of each connected region along the direction vector.

[0154] To further explain the calculation method of the maximum diameter of each connected region: First, rotate the image so that the direction vector is aligned with the horizontal axis. Then, in the rotated image coordinate system, scan the connected regions along the direction perpendicular to the direction vector to find their maximum width, and calculate the maximum diameter in that direction accordingly.

[0155] like Figure 9 As shown, for cases where there are still two or more connected components after processing, the above extension method will merge multiple connected components into one connected component, and the centroid coordinates of the connected component will be calculated.

[0156] The centroid pixel coordinates of the weeds are transformed sequentially from pixel coordinates to image coordinates, then to camera coordinates. The camera coordinates are then converted to the x_mirror coordinates using a nine-point calibration method.

[0157] Finally, in this embodiment, Figure 10This demonstrates the entire process of processing an image captured by a camera, from the original image to the final output of the centroid of the weeds.

[0158] Example 5

[0159] After the microcontroller 4 receives the weed location information and the speed data of the weeder 7 from the multiple sensors, it performs preprocessing through mean filtering and uses timestamps to synchronize and calibrate the data.

[0160] Specifically, after receiving the weed coordinate data sent by the processor 2, the microcontroller 4 stores it in the FIFO1 buffer. When the buffer is full, it performs an average calculation on the data and stores the average value in array 1. Simultaneously, the microcontroller 4 receives the movement speed and acceleration information of the weeder 7 provided by the IMU sensor at the same frequency as the coordinate data sent by the processor 2, and converts it into the actual movement speed of the weeder 7 through integration. This data is stored in the FIFO2 buffer. When the buffer is full, it performs an average calculation and stores the result in array 2. The microcontroller 4 also obtains the absolute angle data of the high-precision encoder about the brushless motors 32 and 33, and stores the data in FIFO3 and array 3 respectively according to the same processing method. Each of the above buffers can only hold 5 data of the corresponding type, while each array can store 2048 data of the corresponding type.

[0161] The microcontroller 4 retrieves data from the position and velocity data arrays at a preset frequency and performs a fusion operation using a simplified Kalman filter formula to obtain accurate weed position coordinates. Simultaneously, the IMU acquires the three-axis motion velocity and angular acceleration data of the weeder 7 in real time and performs time-stamp synchronization calibration with the camera's visual positioning information to construct a Kalman filter state vector containing planar position deviation and its velocity drift components. By fusing the dual observation data of IMU velocity integral displacement and camera spatial coordinates, the system state can be estimated and predicted in real time, generating dynamic compensation quantities. These compensation quantities are mapped and converted into x and y axis deflection angle commands, providing accurate angle references for subsequent control.

[0162] Based on this, after calculating the accurate coordinates of the weeds, the microcontroller 4 transmits the data to the processor 2 via serial port. The processor 2 performs inverse coordinate calculation on the received weed coordinates to obtain the three-dimensional spatial coordinates in the camera coordinate system. It then integrates the timestamp and confidence parameters to perform hierarchical path planning and generate path planning data. Based on this data, the RRT-MPC collaborative algorithm is triggered. First, the RRT algorithm generates a path connecting the current pose and the target point in three-dimensional space. Obstacle avoidance detection and path optimization are performed in conjunction with the crop protection area. Then, the MPC algorithm constructs a kinematic model of the weeder 7. Based on the current motor speed, the pose state of the weeder 7, and historical trajectory data, the laser irradiation deviation at the next moment is predicted. The error caused by vibration and system delay is compensated in real time by solving for the optimal control quantity. Finally, a high-precision motor control command is generated and sent to the microcontroller 4 via serial port 3.

[0163] That is, the global planning layer of the hierarchical path planning uses the RRT algorithm to construct a global probabilistic route map and combines it with dynamic obstacle topology pruning technology to generate an initial obstacle avoidance path; the local planning layer uses the MPC rolling time-domain optimization method to incorporate the weeder 7 dynamic model and the laser reflector angle constraint into the objective function, smooth the local trajectory in real time, compensate for disturbances, and calculate the optimal path for laser scanning. After path planning, the weed coordinates are mapped to the x_mirror31 coordinate system of the laser reflector.

[0164] After receiving the discrete angle control command obtained through path planning, the microcontroller 4 uses the real-time moving speed of the weeder 7 to perform speed compensation of the weeder 7 according to the same Kalman operation to obtain the compensated accurate target angle.

[0165] The microcontroller 4 combines the compensated target angle with the angle data of the brushless motors 32 and 33 to establish a feedforward-feedback composite control loop. It uses the IMU speed feedforward to predict the compensation angle of the laser reflector, and combines the Kalman filter feedback to correct the cumulative error, mapping the displacement deviation into x and y axis deflection angle commands.

[0166] Subsequently, an FOC architecture is adopted to integrate the target angle with the encoder feedback angle. Through PID regulation, Clarke-Park transformation and SVPWM, the angle error is decoupled to the synchronous coordinate system of the brushless motors 32 and 33 rotors. Three-phase drive signals are generated using space vector pulse width modulation to output the digital phase voltages of brushless motors 32 and 33. The motor phase voltages are adjusted by the DRV8303 chip mounted on the microcontroller control module, and the brushless motors 32 and 33 are driven to dynamically adjust the deflection angle of the laser reflector, thereby achieving precise adjustment of the laser reflector deflection angle.

[0167] The microcontroller 4 sends an activation command to the laser 8, which emits a laser beam that enters the laser reflector 3 and is then deflected by the laser reflector mirror to be precisely emitted to the center of the weeds; (Refer to...) Figure 4 During the movement of the weeder 7, the microcontroller 4 continuously updates the deflection angle control command of the laser reflector and simultaneously incorporates a dynamic compensation mechanism. By fusing attitude data and visual coordinate feedback, it eliminates deviations caused by vibration and visual delay in real time, maintains millimeter-level dynamic matching accuracy between the laser irradiation center and the weed center, and eliminates weeds sequentially according to the planned path.

[0168] Repeat the above steps until the weeding operation is complete. During the execution of the above steps, the microcontroller 4 controls the weeder 7 to move at a constant speed of 5 cm / s. Depending on different work efficiency requirements, this speed can be adjusted within the range of 1-10 cm / s.

[0169] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent, real-time, dynamic, and precise laser weeding method, characterized by: include Step 1, Image Acquisition and Processing Collect raw crop and weed images, perform image preprocessing and target labeling on the collected raw images, randomly divide the labeled images and labels into training and validation sets according to the proportion, and perform image enhancement. Step 2: Establish an LW-YOLO model for weed target identification; The LW-YOLO model is based on the YOLO11n model and is an improved version of the YOLO11n model. A small target detection head is added to the output layer of the YOLO11n model undersampling, and an efficient multi-scale attention mechanism EMA is added to the layer before each detection head; The ADown convolution module is used to replace the Backbone downsampling module; Partial convolution is used to replace the convolutional layer of the residual block Bottleneck in the feature extraction module; Step 3, obtain the coordinates of the weed location. Upon receiving the command to start weeding, the processor sends a power-on instruction to the image acquisition module and the microcontroller. The microcontroller will initialize the IMU sensor and high-precision encoder, and control the weeding device to move towards the work area; The image acquisition module acquires raw images of crops and weeds and sends them to the processor. The processor then uses the LW-YOLO model to recognize the received raw images. The LW-YOLO model dynamically extracts keyframes from the video stream and outputs the pixel-level position coordinates of the centroid of weeds in the x_mirror coordinate system by recalibrating the feature space. The processor adds identifier keywords and timestamps to the x and y coordinates of the weeds, packages the data, and sends it to the microcontroller via serial port 1. Step 4: Microcontroller and Processor Data Processing The microcontroller receives the weed location coordinates and pose data from the processor and IMU sensor, performs average filtering on each group, and stores the results. The microcontroller synchronously acquires the absolute angle data of the brushless motor provided by the high-precision encoder and performs similar processing. The microcontroller extracts the processed position and velocity data at a preset frequency, performs data fusion using the Kalman filter algorithm, and after calculating the high-precision weed coordinates, the microcontroller sends the weed coordinates back to the processor via serial port 2. The processor converts the weed coordinates into three-dimensional spatial coordinates in the camera coordinate system through inverse coordinate transformation, and generates reference data by fusing confidence parameters and timestamp information. Based on this data, the RRT-MPC fusion algorithm is started, and dynamic path planning is performed by combining real-time pose information and historical trajectory features to generate a laser scanning path that meets the crop obstacle avoidance requirements and includes displacement prediction compensation. Finally, the motion command is transmitted to the microcontroller via serial port 3. Step 5, Precision Dynamic Laser Weed Control After receiving the planned coordinates, the microcontroller uses the FOC control algorithm to fuse the corrected target angle with the motor angle data fed back by the encoder, calculates the digital phase voltage of the brushless motor, and controls the rotation of the brushless motor, thereby adjusting the laser reflector to the target angle. The microcontroller sends an activation command to the laser, causing the laser beam to enter the laser reflector and be precisely projected onto the center of the weeds after being deflected by the laser reflector. Repeat the above steps until the entire weeding operation is completed.

2. The intelligent real-time dynamic precision laser weeding method according to claim 1, characterized in that: The method for obtaining the pixel-level position coordinates of the centroid of the weeds is as follows: Weed detection is performed using the LW-YOLO model, and the output weed detection boxes are cropped. Image processing is then applied to the resulting weed images to create a connected component for each weed detection box. The centroid pixel coordinates of each weed are then calculated using the following formula: ； ； ； ； ； ； Where M and N represent the rows and columns of the mask, respectively. The mask image is at coordinate points The gray value at that location, for a binary image It is 0 or 1. Represents the zeroth moment of the image. Represents the first 1st moment of the image. This represents the second first-order moment of the image. The coordinates of the centroid pixel of the connected region.

3. The intelligent real-time dynamic precision laser weeding method according to claim 1, characterized in that: The weed detection bounding boxes output by the model are cropped, and image processing operations are performed on the resulting weed images, including: Separate the three channels r, g, b of the weed image and calculate the supergreen index. And normalize it to the interval [0, 255]; The weed image was converted to grayscale using the 2g-rb color feature factor, and the Gaussian weighted average method in the automatic thresholding segmentation method was used for automatic thresholding segmentation of the grayscale image. Set a minimum area threshold for the segmented binary mask to filter out smaller contours; The contour obtained after area filtering is conditionally judged. If there is only one connected region, the area of ​​the connected region is directly calculated, and its centroid is calculated using a second-order matrix to obtain the centroid pixel coordinates of the weeds, which are used for subsequent coordinate transformation. For cases where two or more connected components remain after area filtering, calculate the centroid of each connected component and the coordinates of the center of the weed detection box, so that each connected component extends along the direction from its centroid to the center of the weed detection box. After extension processing, multiple connected components are merged into one connected component, and its centroid coordinates are calculated.

4. The intelligent real-time dynamic precision laser weeding method according to claim 3, characterized in that: The method of using Gaussian weighted average in automatic thresholding segmentation to perform automatic thresholding segmentation on grayscale images involves the following calculation steps: Define a Gaussian kernel, which is a k×k matrix G whose elements... Calculated from a two-dimensional Gaussian function; Calculate the weighted average for each pixel. Calculate the weighted average of the gray values ​​within its domain. ; Calculate the final threshold, which can be adjusted by subtracting a constant from the weighted average; Apply threshold segmentation for each pixel ,if If the value is true, the pixel is set as the target foreground; otherwise, it is set as the background. The obtained binary mask is subjected to morphological processing. Using an elliptical kernel, closing and opening operations are performed sequentially to fill the holes in the connected components and eliminate noise, thus obtaining the final binary mask.

5. The intelligent real-time dynamic precision laser weeding method according to claim 3, characterized in that: The extended processing method includes: First, draw the line connecting the centroid of each connected component to the center of the detection box, obtain the intersection point of the line and the contour of the connected component, and extend the line from the intersection point until the center of the detection box. The extension width is 1 / 6 of the maximum diameter of each connected domain along the direction vector; The method for calculating the maximum diameter of each connected component is further explained by rotating the image so that the direction vector is aligned with the horizontal axis. In the rotated image coordinate system, the connected components are scanned along the direction perpendicular to the direction vector to find the maximum width. Based on the scanning results, the maximum width in that direction is calculated. After extension, the case of multiple connected domains will be transformed into a single connected domain, and the centroid coordinates of this connected domain will be calculated.

6. An intelligent, real-time, dynamic, precise laser weeding device employing the laser weeding method as described in any one of claims 1 to 5, characterized in that: include The image acquisition module is used to acquire images of crops and weeds in the field; The processor processes the acquired crop and weed images based on the LW-YOLO model described in claim 1, extracts the weed location coordinates, and simultaneously performs optimal planning of the laser scanning path. A laser reflector is used to control the laser beam to illuminate a target location by rotating a laser mirror. IMU sensor, used to measure the pose data of the weeding device; A high-precision encoder is used to provide feedback on the angular position information of the brushless motor of the laser reflector. The microcontroller control module controls the phase voltage of the brushless motor and transmits commands from the processor to the sensors and actuators, while also receiving feedback data from the sensors. It also uses a Kalman filter algorithm to calculate the weed position coordinates and the weeding device pose data, compensates for the time delay error of the vision system and the attitude deviation caused by mechanical vibration, and constructs a displacement compensation closed loop through the FOC algorithm to improve the motor positioning accuracy and ensure the realization of dynamic laser weeding.

7. The intelligent real-time dynamic precision laser weeding device according to claim 6, characterized in that: The laser reflector housing is provided with a laser head fixing frame on the right side, and a laser reflecting device is provided on the left side and the top side inside. The two laser reflectors include: Brushless motors are used to drive the laser reflectors and reflector mounting brackets mounted on them; The brushless motor is used to control the rotation angle of the laser reflector in order to regulate the optical path of the laser beam. The laser reflector includes a laser reflector x_mirror and a laser reflector y_mirror. The laser reflector x_mirror is responsible for deflecting the laser beam along the x-axis, while the laser reflector y_mirror deflects the laser beam along the y-axis. The reflecting surfaces of the laser reflector and their respective connected brushless motor rotating shafts are kept coplanar. The housing is used to mount the brushless motor and laser head mounting bracket, and is installed on the weeding device.

8. The intelligent real-time dynamic precision laser weeding device according to claim 7, characterized in that: The laser reflector housing has a T-shaped guide groove on the right side for adjusting the horizontal position of the laser head mounting bracket, which facilitates the alignment of the laser beam with the center of the laser reflector x_mirror. The right side of the laser reflector housing has a set of through bolt holes arranged in a ring, and a bearing seat hole is provided on the opposite side at the same position. This is used to fix the relative position of the laser head mounting bracket and the laser reflector housing, and to adjust the axial position of the bearing and bearing seat on the other side.

9. The intelligent real-time dynamic precision laser weeding device according to claim 7, characterized in that: A quartz glass plate is installed on the right side inside the housing of the laser reflector. The center of the glass plate coincides with the axis of the laser head mounting bracket. This is used to isolate external dust and debris and to allow the laser beam to pass through.

10. The intelligent real-time dynamic precision laser weeding device according to claim 6, characterized in that: The IMU sensor is used to measure the motion state of the weeding device, including velocity and angular acceleration information in the x, y, and z axes.

Images