Intelligent real-time dynamic precise laser weeding method and device

Abstract

The invention discloses an intelligent real-time dynamic precise laser weeding method and device, and relates to the field of laser weeding, and the weeding device comprises an image acquisition module, a processor, a laser reflector, a single-chip microcomputer control module, an IMU sensor, a high-precision encoder and the like. Wherein the image acquisition module is used for acquiring field crop and weed images, the processor processes the acquired crop and weed images based on an LW-YOLO model, extracts weed position coordinate information, and performs optimal planning on a laser scanning path; the laser reflector is used for controlling the laser to irradiate a target position through the rotation of the laser reflector; the single-chip microcomputer control module is used for controlling the phase voltage of the brushless motor, transmitting a command issued by the processor to the sensor and the execution mechanism, and improving the positioning precision of the motor through an FOC algorithm. The weeding precision and efficiency are remarkably improved, the waiting time during static weeding is shortened, and the advantages of high precision and high efficiency are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of laser weeding, and in particular to an intelligent real-time dynamic and precise laser weeding method and device. Background Art

[0002] Laser weed control is a non-contact weed control technology that uses high-energy laser beams to irradiate weeds, damaging their cells with heat until they die. Compared to traditional mechanical and chemical weed control methods, laser weed control achieves precise weed control, minimizing disturbance to the soil and crop roots, while also avoiding the environmental and food safety impacts of chemical residues and reducing the risk of herbicide resistance. This approach meets the needs of green and sustainable modern agriculture.

[0003] However, most current laser weed control technologies use a static operation method, requiring crops and weeds to be in a relatively static state for treatment. Therefore, the efficiency of a single operation is low and it is difficult to meet the needs of large-scale field operations. Summary of the Invention

[0004] To address these issues, the present invention provides an intelligent, real-time, dynamic, and precise laser weed control method and device. This method utilizes dynamic laser weed control technology, combined with a proprietary LW-YOLO model, to achieve precise and rapid dynamic laser weed control of weeds detected while moving across the field, thereby improving operational efficiency and reducing energy consumption.

[0005] The technical solution adopted by the present invention to solve its technical problems is: Intelligent real-time dynamic precision laser weed control method, including Step 1: Image acquisition and processing Collect original crop and weed images, perform image preprocessing and object labeling on the collected original images, and randomly divide the labeled images and labels into training sets and validation sets according to proportion; The divided image data are enhanced according to the training set and the validation set respectively. The enhancement methods include random rotation, random horizontal and vertical mirror flipping, random brightness change, random contrast change, and adding Gaussian noise. All images and labels are subjected to these five enhancement methods in turn. Step 2: Establish the LW-YOLO model for weed target recognition; The LW-YOLO model is designed and improved based on the YOLO11n model. A small object detection head is added to the output layer of the downsampled YOLO11n model, and an efficient multi-scale attention mechanism (EMA) is added to the previous layer of each detection head. Use ADown convolution module to replace Backbone's downsampling module; Partial convolution (PConv) is used to replace the convolution layer of the residual block Bottleneck in the feature extraction module C3K2; Step 3: Get the weed location coordinates After receiving the command to start weeding, the processor sends a power-on command to the image acquisition module and the single-chip microcomputer; The MCU will initialize the IMU sensor and high-precision encoder and control the weeding device to move to the working area; The image acquisition module collects the original crop and weed images and sends them to the processor. The processor uses the LW-YOLO model to identify the received original images. The LW-YOLO model dynamically intercepts key frames of the video stream and outputs the pixel-level position coordinates of the weed mass center in the x_mirror coordinate system through feature space recalibration; The method for obtaining the pixel-level position coordinates of the weed mass center is: Detection is performed according to the LW-YOLO model, and the weed detection frame output by the model is cropped. The obtained weed image is processed so that each weed detection frame is processed to obtain a connected domain, thereby calculating the pixel coordinates of the centroid of each weed. The calculation formula is: ; ; ; ; ; ; Wherein, M and N represent the rows and columns of the mask respectively, is the mask image at coordinate point Gray value at, for binary image is 0 or 1, represents the 0th-order moment of the image, represents the first order moment of the image, represents the second first-order moment of the image, is the pixel coordinate of the centroid of the connected domain; The pixel-level position coordinates of the weed mass center are sequentially transformed, and the nine-point calibration method is used to complete the coordinate transformation from the camera coordinate system to the laser reflector x_mirror coordinate system; The processor adds identification keywords and timestamps to the x and y coordinates of the weeds, packages the data, and sends it to the microcontroller through serial port 1; Step 4: MCU and processor data processing The MCU synchronously receives weed position coordinates and posture data from the processor and IMU sensor, performs average filtering on each of the five data sets and stores the results. The MCU also obtains the absolute angle data of the brushless motor provided by the high-precision encoder and performs similar processing. Among them, the posture data from the IMU sensor must first be converted into actual motion speed through integral calculation, and then averaged and filtered; The microcontroller extracts processed position and velocity data at a preset frequency and fuses the data using the Kalman filter algorithm. After calculating the high-precision weed coordinates, the microcontroller transmits them back to the processor via serial port 2. The processor uses an inverse coordinate transformation to convert the weed coordinates into three-dimensional coordinates in the camera coordinate system. The processor then fuses the confidence parameters and timestamp information to generate baseline data. Based on this data, the RRT-MPC fusion algorithm is activated. It combines real-time pose information with historical trajectory characteristics to make dynamic path decisions, generating a laser scanning path that meets crop obstacle avoidance requirements and includes displacement prediction compensation. The processor then transmits motion commands to the microcontroller via serial port 3. Step 5: Precise dynamic laser weeding After receiving the planned coordinates, the MCU uses the FOC control algorithm to fuse the corrected target angle with the motor angle data fed back by the encoder, calculates the digital phase voltage of the brushless motor, and precisely controls the motor's rotation, thereby adjusting the laser reflector to the target angle. The microcontroller sends a start command to the laser, allowing the laser beam to enter the laser reflector. After being deflected by the laser reflector, it is precisely projected to the center of the weed. At the same time, a dynamic compensation mechanism is embedded in the laser projection process. By fusing posture data with visual coordinate feedback, it eliminates projection deviations caused by mechanical vibration and visual processing delays in real time, maintaining millimeter-level dynamic matching accuracy between the laser irradiation center and the weed center. Repeat the above steps until the entire weeding operation is completed.

[0006] Furthermore, during the entire operation process, the single-chip microcomputer controls the weeding device to move forward at a uniform speed of 5 cm / s, and the travel speed can be adjusted to a range of 1-10 cm / s according to operation requirements.

[0007] Furthermore, the weed detection frame output by the model is cropped and the obtained weed image is processed, including: Separate the three channels r, g, and b of the weed image and calculate the super green index , and normalize it to the interval [0, 255]; The 2g-rb color feature factor is used to grayscale the weed image, and the Gaussian weighted average method in the automatic threshold segmentation method is used to perform automatic threshold segmentation on the grayscale image. A minimum area threshold is set for the segmented binary mask to filter out small contours; If there is only one connected domain, the area of the connected domain is directly calculated, the centroid of the connected domain is calculated by using a second-order matrix, and the centroid pixel coordinates of the weed are obtained, which are used for subsequent coordinate conversion. For the case where two or more connected domains remain after area filtering, the centroid of each connected domain is calculated, and the center coordinates of the weed detection frame are calculated so that each connected domain extends along the direction of the respective centroid to the center of the weed detection frame. After the extension processing, the multiple connected domains are merged into one connected domain, and the centroid coordinates are calculated.

[0008] Further, the Gaussian weighted average method in the automatic threshold segmentation method is used to automatically segment the gray image, and the calculation steps are as follows: A Gaussian kernel is defined, which is a k×k matrix G, and the elements of G are calculated from a two-dimensional Gaussian function; Calculate the weighted average, for each pixel , calculate the weighted average of the gray value in its domain ; Calculate the final threshold, which can be adjusted by subtracting a constant from the weighted average value; Apply threshold segmentation, for each pixel , if , the pixel is set to the target foreground (white), otherwise it is set to the background (black); Perform morphological processing on the obtained binary mask, use an elliptical kernel, and sequentially perform closing and opening operations to fill the holes in the connected domain and eliminate noise to obtain the final binary mask.

[0009] Further, the extension processing method includes: First, draw the connecting line between the centroid of each connected domain and the center of the detection frame, obtain the intersection point of the connecting line and the contour of the connected domain, and extend from the intersection point to the center of the detection frame. The extension width is 1 / 6 of the maximum diameter of each connected domain along the direction vector; The maximum diameter calculation method of each connected domain is further described, the image is rotated so that the direction vector is aligned with the horizontal axis, in the rotated image coordinate system, the connected domain is scanned along the direction perpendicular to the direction vector to find the maximum width, and according to the scanning result, the maximum width in the direction is calculated. After extension, the case of multiple connected domains will become one connected domain, and the centroid coordinates of the connected domain are calculated.

[0010] The present invention also adopts the following technical solutions to solve the technical problems: Intelligent real-time dynamic precision laser weed control device, including Image acquisition module, used to collect images of field crops and weeds; The processor processes the collected crop and weed images based on the LW-YOLO model, extracts the weed location coordinates for subsequent laser weeding, and optimally plans the laser scanning path. The laser reflector is used to control the laser irradiation to the target position by rotating the laser reflector, thereby realizing the laser weeding function on a plane 500-600 mm away from the laser source; IMU sensor, used to measure the posture data of the weeding device; A high-precision encoder is used to feedback the angular position information of the brushless motor of the laser reflector; The single-chip microcomputer control module is used to control the phase voltage of the brushless motor and transmit the commands issued by the processor to the sensor and actuator, while receiving sensor feedback data. It also uses the Kalman filter algorithm to calculate the weed position coordinate information and the weeding device posture data, compensates for the time lag error of the visual system and the posture deviation caused by mechanical vibration, and constructs a displacement compensation closed loop through the FOC algorithm to improve the motor positioning accuracy and ensure the realization of dynamic laser weeding.

[0011] Furthermore, the image acquisition module includes a high-resolution camera and an image processing unit for real-time acquisition of field images and image processing.

[0012] Furthermore, a laser head fixing frame is provided on the right side of the shell of the laser reflector, and a laser reflecting device is provided on the left side and the upper side of the interior thereof; The two laser reflection devices include: A brushless motor for driving a laser reflector and a reflector mounting bracket mounted thereon; The brushless motor is used to control the rotation angle of the laser reflector to adjust the optical path of the laser beam; The laser reflector includes a laser reflector x_mirror and a laser reflector y_mirror, wherein the laser reflector x_mirror is responsible for deflecting the laser beam along the x-axis direction, and the laser reflector y_mirror is responsible for deflecting the laser beam along the y-axis direction, and the reflection surfaces of the laser reflectors are kept coplanar with the rotation axes of the brushless motors to which they are connected; The housing is used to carry a brushless motor and a laser head fixing bracket, and is installed on the weeding device.

[0013] Furthermore, a T-shaped guide groove is provided on the right side of the shell of the laser reflector for adjusting the horizontal position of the laser head fixing frame to facilitate the alignment of the laser beam with the center of the laser reflector x_mirror.

[0014] Furthermore, a group of conducting bolt holes arranged in a ring are provided on the right side of the outer shell of the laser reflector, and a bearing seat hole is provided on the opposite side at the same position, which is used to fix the relative position of the laser head fixing frame and the laser reflector outer shell, and to adjust the axial position of the bearing on the other side and the bearing hole seat.

[0015] Furthermore, a quartz glass sheet is provided on the right side inside the shell of the laser reflector, the center of the glass sheet coincides with the axis of the laser head fixing frame, the glass sheet is 10 mm long, 10 mm wide and 2 mm thick, and is used to isolate external dust and debris and to transmit the laser beam.

[0016] Furthermore, a cylindrical strong magnet is installed at the tail of the brushless motor, and the magnet can rotate with the motor shaft. A high-precision encoder is installed 2 mm away from the surface of the magnet. The encoder is fixed in position and uses the SPI communication protocol to interact with the microcontroller. The brushless motor, high-precision encoder and magnet are wrapped in a shell.

[0017] Furthermore, the IMU sensor is used to measure the motion state of the weeding device, including speed and acceleration information in the three axes of x, y, and z.

[0018] Compared with the prior art, the present invention can bring the following beneficial effects: This method integrates image acquisition, target recognition, path planning, and precision targeting technologies. It uses a high-resolution camera to capture images of field crops and weeds, and uses an improved LW-YOLO model for real-time detection. This method improves model recognition accuracy and speed through lightweight optimization. A three-dimensional model of a laser reflector is designed and optimized, and the laser reflector is driven by dual brushless motors, enabling precise two-dimensional positioning. An IMU sensor and a high-precision encoder are also used to improve targeting accuracy. A Kalman filter algorithm is used to establish a composite control system to compensate for motion vibration and displacement errors. The laser reflector is controlled by a single-chip microcomputer, and the laser path is optimized using the RRT and MPC algorithms. Finally, the motor positioning accuracy is improved using the FOC algorithm. This method significantly improves weeding accuracy and efficiency, reduces waiting time during static weeding, and offers the advantages of high precision and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any labor or creativity.

[0020] Figure 1 This is a schematic structural diagram of the laser dynamic weeding device of Example 1; Figure 2 Schematic diagram of the internal structure of the laser reflector of Example 1 (right side view); Figure 3 Schematic diagram of the internal structure of the laser reflector of Example 1 (left side view); Figure 4 This is a schematic diagram of the dynamic weeding process of Example 5; Figure 5 This is a schematic diagram of the weed identification and positioning process of Example 2; Figure 6 This is a grayscale processing diagram of the original image using the super green index as described in Example 2; Figure 7 is a binary image of the grayscale image after morphological processing; Figure 8 is a schematic diagram of the calculated centroid of the binary image; Figure 9 This is a flow chart of centroid processing in the case of multiple connected domains in Example 2; Figure 10 This is a process diagram of a complete identification and positioning process of Example 2. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0022] It should be noted that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments based on the embodiments of the present invention and obtained by researchers of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0023] In order to make the above-mentioned objects, features and advantages of the present invention clearer and easier to understand, the present invention will be further explained in detail below with reference to the accompanying drawings and specific embodiments, taking lettuce as a representative crop.

[0024] Example 1 The application discloses an intelligent real-time dynamic precision laser weeding machine which comprises an image acquisition module 1, a processor 2, a laser reflector 3, a single-chip microcomputer 4, an IMU sensor, a high-precision encoder and a weeding machine 7 and a laser 8.

[0025] Referring to Figures 1 to 3 The image acquisition module 1 is composed of a high-resolution camera and an image processing unit, is used for collecting images of field lettuce and weeds in real time, and returns the images to the processor 2. The processor 2 processes the collected images of lettuce and weeds by using a trained LW-YOLO model, obtains weed coordinates, and converts the weed coordinates from a camera coordinate system to a laser reflector coordinate system; meanwhile, the weed coordinates corrected by the single-chip microcomputer 4 are converted from the laser reflector coordinate system back to the camera coordinate system, laser scanning shortest path planning is carried out in the coordinate system, the path operation adopted is a hybrid form combining RRT algorithm and MPC algorithm; the laser reflector 3 controls the emitted laser to strike on a target position by deflecting the angle of a laser reflector, the target position is 500-600 mm away from the center vertical height of the laser reflector x_mirror 31; the IMU sensor is used for detecting the pose data of the weeding machine 7 and transmitting the pose data to the single-chip microcomputer 4, and the pose data includes speed and acceleration information in x, y and z axial directions; the high-precision encoder is used for detecting the angle position information of brushless motors 32 and 33 in the laser reflector 3; the single-chip microcomputer control module is used for controlling the phase voltage of the brushless motors 32 and 33, transmitting the commands from the processor 2 to the sensors and actuators, and receiving sensor feedback data; meanwhile, the Kalman filtering algorithm is used for operating the weed position data and the pose data of the weeding machine 7, compensating for the attitude deviation caused by the time lag error of the vision system and mechanical vibration, and constructing a displacement compensation closed loop through the FOC algorithm, so that the motor positioning accuracy is improved, and the dynamic laser weeding is realized. The weeding machine 7 is used for carrying the above components and performing dynamic weeding operation in a field.

[0026] A T-shaped guide slot is provided on the right side of the laser reflector 3 housing for adjusting the horizontal position of the laser head holder 5, facilitating alignment of the laser head 6 laser beam with the center of the laser reflector x_mirror 31. A circular array of guide bolt holes is provided on the right side of the laser reflector 3 housing. Bearing seat holes are located on the opposite side of the housing to secure the laser head holder 5 relative to the laser reflector 3 housing and to adjust the axial position of the other bearing 37 relative to the bearing seat. A quartz glass plate 38 is located on the right side of the laser reflector 3 housing, with its center aligning with the axis of the laser head holder 5. The glass plate 38 is 10 mm long, 10 mm wide, and 2 mm thick, providing insulation from external dust and debris while allowing the laser beam to pass through. A brushless motor 32 and 33 are located on the top and left sides of the laser reflector 3 housing, respectively. The vertical distance between the axes of the two brushless motors 32 and 33 is 20 mm. Laser reflectors are mounted on the two brushless motors 32 and 33, respectively. The laser reflector x_mirror31, which is responsible for deflecting the laser beam in the x-axis direction, is 13 mm long and 10 mm wide and is mounted via an x_mirror mounting bracket 36. The laser reflector y_mirror35, which is responsible for deflecting the laser beam in the y-axis direction, is 60 mm long and 20 mm wide and is mounted and fixed via a y_mirror mounting bracket 34.

[0027] The housing of the laser reflector 3 consists of four parts. The right side holds the laser head mount 5 and bearing 37, the left side holds the brushless motor 33 that controls the laser beam's deflection in the y-direction, the bottom is designed as a hollow structure to allow the laser beam to pass through, and the top and front and back sides are integrally designed in an inverted "U" shape to hold the brushless motor 32 that controls the beam's deflection in the x-direction. Once these four parts are assembled, the overall form is a rectangular parallelepiped with effective internal dimensions of 105 mm in length, 100 mm in width, and 78 mm in height. A strong cylindrical magnet is mounted at the tail of each brushless motor 32 and 33, allowing the magnet to rotate with the motor shaft. A high-precision encoder is mounted 2 mm from the magnet surface. The encoder is fixed in position and uses the SPI communication protocol to exchange information with the microcontroller 4. The housing is designed to enclose the motor, high-precision encoder, and magnets.

[0028] Example 2 An intelligent real-time dynamic precision laser weed control method for weed identification and positioning process, referring to Figure 5 , this method specifically includes: Collect raw images containing lettuce and weeds, preprocess and annotate the collected images, and randomly divide the annotated images and their labels into a training set and a validation set in an 8:2 ratio. The image data in the training set and validation set are enhanced separately. The enhancement methods include random rotation, random horizontal and vertical mirror flipping, random brightness change, random contrast change, and adding Gaussian noise. The above five enhancement methods are applied to all images and their labels in turn. Build the LW-YOLO model for lettuce and weed target recognition; When receiving the command to start weeding, the processor 2 sends a power-on instruction to the image acquisition module 1 and the single-chip microcomputer 4; The single chip computer 4 will wake up the IMU sensor, the high-precision encoder and the lawn mower 7, and then the lawn mower 7 will start to move towards the working area; Image acquisition module 1 collects raw images of lettuce and weeds and sends them to processor 2, which uses the LW-YOLO model to recognize the received images; The LW-YOLO model is used for target detection, and each weed detection frame output by the model is cropped. The cropped weed image is then processed to form a connected domain within each detection frame, thereby calculating the pixel coordinates of the centroid of each weed.

[0029] Specifically, the LW-YOLO model dynamically intercepts key frames of the video stream and outputs the pixel-level position coordinates of the weed mass center in the x_mirror coordinate system through feature space recalibration; The method for obtaining the pixel-level position coordinates of the weed mass center is: Detection is performed according to the LW-YOLO model, and the weed detection frame output by the model is cropped. The obtained weed image is processed so that each weed detection frame is processed to obtain a connected domain, thereby calculating the pixel coordinates of the centroid of each weed. The calculation formula is: ; ; ; ; ; ; Wherein, M and N represent the rows and columns of the mask respectively, is the mask image at coordinate point Gray value at, for binary image is 0 or 1, represents the 0th-order moment of the image, represents the first order moment of the image, represents the second first-order moment of the image, is the pixel coordinate of the centroid of the connected domain; The pixel-level position coordinates of the weed mass center are sequentially transformed, and the nine-point calibration method is used to complete the coordinate transformation from the camera coordinate system to the laser reflector x_mirror31 coordinate system.

[0030] Processor 2 adds identification keywords and timestamps to the x and y coordinates of the weeds, packages the data, and sends it to microcontroller 4 through serial port 1.

[0031] Example 3 The LW-YOLO model for identifying lettuce and weed targets is constructed as described in Example 2, including: Design and improve the model framework based on YOLO11n; A small object detection head Xsmall is added to the output layer of the original model with 4 times the downsampling. An efficient multi-scale attention mechanism EMA is added to the previous layer of each detection head. Use ADown convolution module to replace Backbone's downsampling module; Partial convolution (PConv) is used to replace the convolution layer in the residual block Bottleneck in the feature extraction module C3K2.

[0032] Example 4 Performing an image processing operation on each cropped weed image as described in Example 2 includes: Performing Gaussian filtering and median filtering on the weed image to achieve image smoothing and noise removal; Separate the three channels r, g, and b of the weed image and convert them into 32-bit floating-point numbers to avoid overflow problems in integer operations; calculate the super green index. , and its calculation formula is ; and will The array is normalized to the interval [0, 255] for subsequent processing; the normalization formula is 55; in, yes The minimum value in the array, is the maximum value, is the normalized value; The above 2g-rb color feature factor is used to grayscale the weed image, as shown in the following example: Figure 6 As shown, the grayscale image is segmented using the Gaussian weighted average method in the automatic threshold segmentation method. The calculation steps are as follows: ; Define the Gaussian kernel, which is a matrix G of size k×k, whose elements are Calculated by a two-dimensional Gaussian function; where are the coordinates in the Gaussian kernel, is the standard deviation, which is used to control the width of the Gaussian function; Calculate the weighted average: for each pixel , calculate the weighted average of the grayscale values ​​within its area ; ; Calculate the final threshold: subtract a constant from the weighted average To adjust and determine the final threshold ; ; Apply threshold segmentation, for each pixel ,if , then the pixel is set as the target foreground (white), otherwise it is set as the background (black); The binary mask is morphologically processed, and the elliptical kernel is used to perform closing and opening operations in sequence to fill the holes in the connected domain and eliminate noise to obtain the final binary mask, as shown in Figure 2. Figure 7 shown.

[0033] Applying the Canny edge detection algorithm to the binary image to refine the boundary of the weeds and extract the contour; A minimum area threshold is set for the contours to filter out smaller contours.

[0034] Perform conditional judgment on the contour obtained by area filtering. If there is only one connected domain, calculate the area of ​​the connected domain and use the second-order matrix to calculate its centroid to obtain the pixel coordinates of the centroid of the weeds, such as Figure 8 shown.

[0035] If there are still two or more connected domains after area filtering, calculate the centroid of each connected domain and the center coordinates of the cropped weed detection frame respectively, so that each connected domain is extended in the direction of its centroid pointing to the center of the detection frame. The center coordinates of the detection frame are calculated using the following formula ; ; in,( ) is the coordinate of the upper left corner of the weed detection frame output by the model, ( ) is the coordinate of the lower right corner of the weed detection frame.

[0036] The extension method is further described as follows: a line is drawn between the centroid of each connected domain and the center of the detection frame, and an intersection point of the line and the contour of the connected domain is obtained, and the line is extended from the intersection point until it reaches the center of the detection frame; The extension width is 1 / 6 of the maximum diameter of each connected domain along the direction vector.

[0037] The calculation method of the maximum diameter of each connected domain is further explained: first, the image is rotated so that the direction vector is aligned with the horizontal axis. Then, in the rotated image coordinate system, the connected domain is scanned along the direction perpendicular to the direction vector to find its maximum width, and the maximum diameter in this direction is calculated accordingly.

[0038] like Figure 9 As shown in FIG, if there are still two or more connected domains after processing, the multiple connected domains will be merged into one connected domain after the above extension method, and the centroid coordinates of the connected domain will be calculated.

[0039] The pixel coordinates of the centroid of the weeds are transformed from pixel coordinates to an image coordinate system, and then to a camera coordinate system. The camera coordinate system is transformed into an x_mirror coordinate system using a nine-point calibration method.

[0040] Finally, in this embodiment, Figure 10 It shows the whole process of processing a frame of image acquired by a camera from the original image to the final output of the centroid point of the weeds.

[0041] Example 5 After the single chip microcomputer 4 receives the weed position information and the speed data of the weed cutter 7 from the multi-sensor, it performs pre-processing through mean filtering and synchronizes and calibrates the data using the timestamp.

[0042] Specifically, after receiving the weed coordinate data sent by the processor 2, the single-chip microcomputer 4 stores it in the FIFO1 buffer; when the buffer is full, the data therein is averaged and the obtained average value is stored in the array 1; the single-chip microcomputer 4 simultaneously receives the motion speed and acceleration information of the lawn mower 7 provided by the IMU sensor at the same frequency as the coordinate data sent by the processor 2, and converts it into the actual motion speed of the lawn mower 7 through integration operation. The data is stored in the FIFO2 buffer. When the buffer is full, it is averaged and the result is stored in the array 2; the single-chip microcomputer 4 also obtains the absolute angle data of the brushless motors 32 and 33 from the high-precision encoder, and stores the data in the FIFO3 and array 3 respectively according to the same processing method. The above buffers can only accommodate 5 corresponding types of data, while each array can store 2048 corresponding types of data.

[0043] The single-chip computer 4 extracts data from the position and velocity data array at a preset frequency, and performs a fusion operation on it using a simplified Kalman filter formula to obtain the accurate weed position coordinates. At the same time, the three-axis motion speed and angular acceleration data of the weeder 7 are collected in real time through the IMU, and the timestamp synchronization calibration is performed with the camera visual positioning information to construct a Kalman filter state vector containing the plane position deviation and its velocity drift component. The dual observation data of the IMU velocity integral displacement and the camera spatial coordinates are integrated to realize real-time estimation and prediction of the system state and generate a dynamic compensation amount. The compensation amount is mapped and converted into deflection angle instructions of the x and y axes to provide a precise angle reference for subsequent control.

[0044] Based on this, MCU 4 calculates the exact weed position coordinates and transmits the data to processor 2 via the serial port. Processor 2 performs an inverse operation on the received weed coordinates to obtain the three-dimensional spatial coordinates in the camera coordinate system. It then integrates the timestamp and confidence parameters to perform hierarchical path planning, generating path planning data. This data triggers the RRT-MPC collaborative algorithm, which first generates a path connecting the current position and the target point in three-dimensional space using the RRT algorithm. Obstacle avoidance detection and path optimization are performed in conjunction with the crop protection area. The MPC algorithm then constructs a kinematic model of the weeder 7. Based on the current motor speed, position state, and historical trajectory data, it predicts the laser irradiation deviation at the next moment. It then calculates the optimal control variable to compensate for errors caused by vibration and system delays in real time. Finally, it generates high-precision motor control commands and sends them to MCU 4 via serial port 3.

[0045] Specifically, the global planning layer of the hierarchical path planning uses the RRT algorithm to construct a global probabilistic roadmap and combines it with dynamic obstacle topology pruning to generate an initial obstacle avoidance path. The local planning layer employs the MPC rolling-horizon optimization method, incorporating the dynamic model of the mower 7 and the laser mirror angle constraints into the objective function. This allows for real-time smoothing of local trajectories, compensation for disturbances, and calculation of the optimal laser scanning path. After path planning, the weed coordinates are mapped to the laser mirror x_mirror31 coordinate system.

[0046] After receiving the discrete angle control instruction obtained through path planning, the single chip computer 4 uses the real-time moving speed of the lawn mower 7 to perform the lawn mower 7 speed compensation on the target angle according to the same Kalman operation to obtain the compensated accurate target angle; The single chip computer 4 combines the compensated target angle with the angle data of the brushless motors 32 and 33 to establish a feedforward-feedback composite control loop, uses the IMU velocity feedforward to predict the laser reflector compensation angle, combines the Kalman filter feedback to correct the accumulated error, and maps the displacement deviation into the x-axis and y-axis deflection angle instructions; Subsequently, the FOC architecture is adopted to fuse the target angle with the encoder feedback angle. Through PID adjustment, Clarke-Park transformation and SVPWM, the angle error is decoupled to the rotor synchronous coordinate system of the brushless motors 32 and 33. The three-phase drive signal is generated using space vector pulse width modulation, and the digital phase voltage of the brushless motors 32 and 33 is output. The motor phase voltage is adjusted and the brushless motors 32 and 33 are driven by the DRV8303 chip installed on the single-chip microcomputer control module to dynamically adjust the deflection angle of the laser reflector, thereby achieving precise adjustment of the deflection angle of the laser reflector.

[0047] The single chip computer 4 sends an opening command to the laser 8, and the laser 8 emits laser light into the laser reflector 3, and is deflected by the laser reflector and accurately emitted to the center of the weeds; Figure 4 As the weed remover 7 moves, the microcontroller 4 continuously updates the deflection angle control instructions of the laser reflector and simultaneously implements a dynamic compensation mechanism. By fusing posture data with visual coordinate feedback, it eliminates deviations caused by vibration and visual delay in real time, maintains millimeter-level dynamic matching accuracy between the laser irradiation center and the weed center, and removes weeds sequentially according to the planned path. Repeat the above steps until the weeding operation is completed. During the execution of the above steps, the microcontroller 4 controls the weeder 7 to move at a constant speed of 5 cm / s. According to different operation efficiency requirements, the speed can be adjusted within the range of 1-10 cm / s.

[0048] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Intelligent real-time dynamic precision laser weed control method, characterized by: include Step 1: Image acquisition and processing Collect original crop and weed images, perform image preprocessing and object labeling on the collected original images, randomly divide the labeled images and labels into training and validation sets according to proportion, and perform image enhancement; Step 2: Establish the LW-YOLO model for weed target recognition; The LW-YOLO model is designed and improved based on the YOLO11n model. A small object detection head is added to the output layer of the downsampled YOLO11n model, and an efficient multi-scale attention mechanism (EMA) is added to the previous layer of each detection head. Use ADown convolution module to replace Backbone's downsampling module; Partial convolution is used to replace the convolution layer of the residual block Bottleneck in the feature extraction module; Step 3: Get the weed location coordinates After receiving the command to start weeding, the processor sends a power-on command to the image acquisition module and the single-chip microcomputer; The MCU will initialize the IMU sensor and high-precision encoder and control the weeding device to move to the working area; The image acquisition module collects the original crop and weed images and sends them to the processor. The processor uses the LW-YOLO model to identify the received original images. The LW-YOLO model dynamically intercepts key frames of the video stream and outputs the pixel-level position coordinates of the weed mass center in the x_mirror coordinate system through feature space recalibration; The processor adds identification keywords and timestamps to the x and y coordinates of the weeds, packages the data, and sends it to the microcontroller through serial port 1; Step 4: MCU and processor data processing The single chip microcomputer receives the weed position coordinates and posture data from the processor and IMU sensor, performs average filtering on them by group and stores the results; The single chip microcomputer synchronously obtains the absolute angle data of the brushless motor provided by the high-precision encoder and performs similar processing; The microcontroller extracts processed position and velocity data at a preset frequency and fuses the data using the Kalman filter algorithm. After calculating the high-precision weed coordinates, the microcontroller transmits them back to the processor via serial port 2. The processor uses an inverse coordinate transformation to convert the weed coordinates into three-dimensional coordinates in the camera coordinate system. The processor then fuses the confidence parameters and timestamp information to generate baseline data. Based on this data, the RRT-MPC fusion algorithm is activated. Dynamic path planning is performed by combining real-time pose information with historical trajectory characteristics. This generates a laser scanning path that meets crop obstacle avoidance requirements and includes displacement prediction compensation. Finally, the motion command is transmitted to the microcontroller via serial port 3. Step 5: Precise dynamic laser weeding After receiving the planned coordinates, the MCU uses the FOC control algorithm to fuse the corrected target angle with the motor angle data fed back by the encoder, calculates the digital phase voltage of the brushless motor, and controls the rotation of the brushless motor, thereby adjusting the laser reflector to the target angle. The single chip microcomputer sends an on command to the laser, causing the laser beam to enter the laser reflector and be deflected by the laser reflector before being accurately projected to the center of the weed. Repeat the above steps until the entire weeding operation is completed.

2. The intelligent real-time dynamic precision laser weeding method according to claim 1, characterized in that: The method for obtaining the pixel-level position coordinates of the weed mass center is: Detection is performed according to the LW-YOLO model, and the weed detection frame output by the model is cropped. The obtained weed image is processed so that each weed detection frame is processed to obtain a connected domain, thereby calculating the pixel coordinates of the centroid of each weed. The calculation formula is: ； ； ； ； ； ； Wherein, M and N represent the rows and columns of the mask respectively, is the mask image at coordinate point Gray value at, for binary image is 0 or 1, represents the 0th-order moment of the image, represents the first order moment of the image, represents the second first-order moment of the image, is the pixel coordinate of the centroid of the connected domain.

3. The intelligent real-time dynamic precision laser weeding method according to claim 1, characterized in that: The weed detection frame output by the model is cropped, and the image processing operation is performed on the obtained weed image, including: Separate the three channels r, g, and b of the weed image and calculate the super green index , and normalize it to the interval [0, 255]; The 2g-rb color feature factor is used to grayscale the weed image, and the Gaussian weighted average method in the automatic threshold segmentation method is used to perform automatic threshold segmentation on the grayscale image. Set a minimum area threshold for the binary mask obtained by segmentation to filter out smaller contours; Perform conditional judgment on the contour obtained by area filtering. If there is only one connected domain, directly calculate the area of ​​the connected domain and use the second-order matrix to calculate its centroid to obtain the centroid pixel coordinates of the weeds for subsequent coordinate conversion. If there are still two or more connected domains left after area filtering, calculate the centroid of each connected domain and the center coordinates of the weed detection frame respectively, so that each connected domain extends in the direction from its centroid to the center of the weed detection frame. After extension processing, multiple connected domains are merged into one connected domain, and the coordinates of its centroid are calculated.

4. The intelligent real-time dynamic precision laser weeding method according to claim 3, characterized in that: The Gaussian weighted average method in the automatic threshold segmentation method is used to perform automatic threshold segmentation on the grayscale image. The calculation steps are as follows: Define the Gaussian kernel, which is a matrix G of size k×k, whose elements are Calculated by a two-dimensional Gaussian function; Calculate the weighted average, for each pixel , calculate the weighted average of the grayscale values ​​within its area ; Calculate a final threshold value, which can be adjusted by subtracting a constant from the weighted average; Apply threshold segmentation, for each pixel ,if , then the pixel is set as the target foreground, otherwise it is set as the background; The obtained binary mask is morphologically processed, and an elliptical kernel is used to perform closing and opening operations in sequence to fill the holes in the connected domain and eliminate noise to obtain the final binary mask.

5. The intelligent real-time dynamic precision laser weeding method according to claim 3, characterized in that: The extension processing method comprises: First, draw a line connecting the centroid of each connected domain and the center of the detection frame, obtain the intersection of the line and the contour of the connected domain, and extend it from the intersection point to the center of the detection frame and stop; The extension width is 1 / 6 of the maximum diameter of each connected domain along the direction vector; The method for calculating the maximum diameter of each connected domain is further described. The image is rotated so that the direction vector is aligned with the horizontal axis. In the rotated image coordinate system, the connected domain is scanned in a direction perpendicular to the direction vector to find the maximum width. Based on the scanning results, the maximum width in this direction is calculated. After extension, multiple connected domains will become one connected domain, and the coordinates of the centroid of this connected domain will be calculated.

6. Intelligent real-time dynamic precision laser weeding device, characterized by: include Image acquisition module, used to collect images of field crops and weeds; a processor, which processes the collected crop and weed images based on the LW-YOLO model described in claim 1, extracts weed location coordinate information, and simultaneously performs optimal planning of the laser scanning path; A laser reflector is used to control the laser to irradiate the target position by rotating the laser reflector; IMU sensor, used to measure the posture data of the weeding device; A high-precision encoder is used to feedback the angular position information of the brushless motor of the laser reflector; The single-chip microcomputer control module is used to control the phase voltage of the brushless motor and transmit the commands issued by the processor to the sensor and actuator, while receiving sensor feedback data. It also uses the Kalman filter algorithm to calculate the weed position coordinate information and the weeding device posture data, compensates for the time lag error of the visual system and the posture deviation caused by mechanical vibration, and constructs a displacement compensation closed loop through the FOC algorithm to improve the motor positioning accuracy and ensure the realization of dynamic laser weeding.

7. The intelligent real-time dynamic precision laser weeding device according to claim 6, characterized in that: The right side of the laser reflector housing is provided with a laser head fixing frame, and the left side and upper side of the interior are each provided with a laser reflecting device; The two laser reflection devices include: A brushless motor, used for driving a laser reflector and a reflector mounting frame mounted thereon; The brushless motor is used to control the rotation angle of the laser reflector to adjust the optical path of the laser beam; The laser reflector includes a laser reflector x_mirror and a laser reflector y_mirror, wherein the laser reflector x_mirror is responsible for deflecting the laser beam along the x-axis direction, and the laser reflector y_mirror is responsible for deflecting the laser beam along the y-axis direction, and the reflection surfaces of the laser reflectors are kept coplanar with the rotation axes of the brushless motors to which they are connected; The housing is used to carry a brushless motor and a laser head fixing bracket, and is installed on the weeding device.

8. The intelligent real-time dynamic precision laser weeding device according to claim 7, characterized in that: The right side of the laser reflector housing is provided with a T-shaped guide groove for adjusting the horizontal position of the laser head fixing frame to facilitate the alignment of the laser beam with the center of the laser reflector x_mirror; A group of guide bolt holes arranged in a ring are provided on the right side of the laser reflector shell, and a bearing seat hole is provided on the opposite side at the same position, which is used to fix the relative position of the laser head fixing frame and the laser reflector shell, and to adjust the axial position of the bearing on the other side and the bearing hole seat.

9. The intelligent real-time dynamic precision laser weeding device according to claim 7, characterized in that: A quartz glass sheet is provided on the right side of the inner shell of the laser reflector, and the center of the glass sheet coincides with the axis of the laser head fixing frame, which is used to isolate external dust and debris and to transmit the laser beam.

10. The intelligent real-time dynamic precision laser weeding device according to claim 6, characterized in that: The IMU sensor is used to measure the motion state of the weeding device, including the velocity and angular acceleration information in the three axes of x, y, and z.

Images