An artificial intelligence-based field automatic walking laser weeding robot and a control method thereof

Abstract

The present application relates to the technical field of agricultural intelligent equipment, in particular to a field automatic walking laser weeding robot based on artificial intelligence. The robot comprises a main body, a walking system, a sensing and positioning system, a laser weeding system, a control and calculation system and a man-machine interaction system. The sensing and positioning system is used for collecting field working environment information and realizing autonomous positioning; the control and calculation system identifies and distinguishes crops and weeds based on artificial intelligence algorithm, and controls the laser weeding system to carry out fixed-point burning treatment on weeds according to the identification result; the walking system is used for realizing autonomous walking of the robot in the field ridge or between crop rows. The present application reduces the use of chemical herbicides, improves the weeding efficiency and operation safety through the non-contact laser precise weeding mode, and is suitable for various farmland working environments.

Description

Technical Field

[0001] This invention relates to the field of agricultural automation and intelligent equipment technology, specifically to an automatic walking weeding robot in the field based on artificial intelligence visual recognition and laser control technology, and its control method in automatic operation mode and cloud-based remote semi-automatic operation mode. Background Technology

[0002] Currently, agricultural weed control mainly relies on manual weeding, mechanical weeding, and chemical pesticide weeding. Among these methods, manual weeding is labor-intensive, inefficient, and costly; mechanical weeding can easily damage crop roots or seedlings during operation; and the long-term use of chemical herbicides can easily lead to soil pollution, pesticide residues in crops, and damage to the ecological environment. With the development of artificial intelligence, machine vision, and laser technology, some agricultural robots and laser weeding devices based on visual recognition have emerged. However, existing technologies still have the following shortcomings: 1. The equipment is too bulky, and its operating range and height cannot be adjusted. It lacks adaptability to complex field environments, making it difficult to move and operate stably under different terrains and crop row spacings. 2. Existing visual or laser weeding equipment has poor adaptability in complex field environments, limited recognition accuracy, and lacks continuous learning ability. Furthermore, it is almost unusable in actual operations when faced with high-density, large, stubborn weeds that almost cover the ground. 3. The accuracy of crop and weed identification is limited, especially in areas where crops and weeds coexist in close proximity, where accidental damage is likely to occur; 4. Most equipment lacks the ability to self-learn and optimize models based on operational data, making it difficult to continuously improve operational accuracy over time. 5. Existing laser weeding devices mostly use a single scanning or single operation mode, making it difficult to simultaneously achieve both operational efficiency and safety; 6. Limited human-computer interaction and remote control capabilities make it difficult to meet the needs of actual agricultural production for manual intervention and remote collaborative operations. 7. Therefore, there is an urgent need for a laser weeding robot and its control method that can automatically walk in the field environment, has high-precision recognition capabilities, supports automatic and semi-automatic collaborative operation, and has continuous self-learning capabilities. Summary of the Invention

[0003] Purpose of the invention: The purpose of this invention is to provide an artificial intelligence-based autonomous walking laser weeding robot for fields and its control method, which can: 1. Adjustable suspension, variable wheelbase, enabling automatic movement in complex field environments; 2. Achieve precise positioning and intelligent obstacle avoidance through multi-sensor fusion; 3. Accurately distinguish between crops and weeds using artificial intelligence visual algorithms; 4. Employ multiple laser weeding strategies to avoid damaging crops; 5. Supports automatic operation mode and cloud-based remote semi-automatic operation mode; 6. Possesses the ability to self-learn from operational data and continuously optimize models. Technical solution

[0004] (I) Overall Equipment Structure This artificial intelligence-based autonomous laser weeding robot for field use includes the following system modules: 1. Main body of the machine, adjustable suspension, four-motor four-wheel drive system; 2. Sensing and positioning system; 3. Laser weeding system; 4. Control and computing system; 5. Human-computer interaction system; 6. Heat dissipation and power supply system; (II) Sensing and Positioning System The perception and positioning system includes a visual camera, a lidar sensor, and a GPS and BeiDou dual-mode positioning system. Camera layout structure The visual camera includes: 1. Two front-facing cameras; 2. Two rear-facing cameras; 3. Four working cameras are located at the bottom of the device, pointing vertically downwards. The system comprises a front-facing camera, a rear-facing camera, and an infrared radar working together for environmental perception, automatic walking control, and obstacle distance recognition; four bottom-mounted cameras are used for crop and weed identification and location, and the images they capture are seamlessly fused and stitched together to form a continuous-operation field-of-view radar sensor. The lidar sensor is installed on the top of the device and has a scanning frequency of no less than 8 revolutions per second. It is fused and compared with the visual camera data to achieve 360° obstacle recognition and obstacle avoidance positioning. (III) Laser weeding system The laser weeding system includes a first laser emitting component, a second laser emitting component, and a laser gimbal structure. 1. First laser emitting component (area scanning component) It includes at least two high-power laser emitters, which emit in a fan-shaped laser operation line with an adjustable fan angle. It achieves large-area scanning ablation through lateral movement and is used for rapid surface scanning ablation of weeds in high-density, large-area areas outside the crop red line. 2. Second laser emitting component (spot emission component) It includes at least two precision laser emitters that emit linear laser pulses for precise spot ablation of weeds near crop areas. 3. Laser gimbal structure Each laser emitter is mounted on a hemispherical two-degree-of-freedom stepper motor vertical gear set gimbal, enabling precise pointing in any direction within the hemispherical range. (iv) Control Logic and Operating Mode 1. Automatic operation mode In automatic operation mode, the crop type is selected first. The system uses visual and radar fusion to identify the terrain and crop rows, uses artificial intelligence models to distinguish between crops and weeds, automatically generates the operation route through the selected operation area on the GPS map, and controls the laser weeding system to perform weeding operations, while recording and uploading operation data. 2. Semi-automatic operation mode (remote operation via cloud) In semi-automatic operation mode, operators provide manual assistance via a cloud-based remote control system to handle high-value and complex operating environments. The operation involves setting the row spacing and ridge width, then the machine first traverses the work area along a preset route or a manually controlled route. This serves two purposes: first, to preset the optimal route and turning points in complex operating environments; and second, to collect data on crop and weed distribution. This data is then transmitted back to the client, which uses the generated route and bitmap to manually edit the pre-drawn B-Spline work red line for more precise sweeping. After the operator sends the start command, the system uses this B-Spline work red line to delineate the safe boundary between crops and weeds according to the preset path. The area-scanning laser avoids the crop area, while the point-scanning laser only performs point-pulse ablation on the weeds within the boundary. 3. Aircraft movement working logic: Based on the ridge spacing and width captured by the machine's camera, the human-machine interface system provides suggested wheelbase and suspension height data. The operator can adjust the wheelbase and suspension height by adjusting the adjusters on each suspension leg. Four independent motors drive the forward and backward movement of each mechanical leg. The machine's forward and backward movement, as well as turning, can be automatically controlled by the control and computing system or manually controlled via the human-machine interface client. 4. The fuselage's positioning and obstacle avoidance perception system's operating logic: The system achieves autonomous driving and obstacle avoidance through the combined use of two sets of front and rear depth-sensing binocular cameras, a (top-mounted) radar, and satellite positioning. The front and rear cameras respectively capture image information along the forward and backward paths. Using a depth algorithm from the binocular cameras, millimeter-level real-world positioning is achieved. The lidar mounted on the top of the machine performs a 360-degree horizontal scan, acquiring obstacle location data within a 30-meter radius. Working in conjunction with the front and rear camera sets and GPS positioning coordinates, the system calculates the optimal turning trajectory, enabling real-time obstacle avoidance and path maintenance. 5. The working logic of image localization and recognition through camera matrix fusion: The four cameras located at the bottom of the machine are the main sensors for weeding operations. The images captured by the four bottom working cameras are processed by camera calibration, distortion correction, unified plane coordinate mapping, overlapping area feature matching and weighted fusion algorithms to seamlessly stitch multiple local field-of-view images into a continuous and complete operation image (this process algorithm is completed on the host computer). The prerequisites for camera placement include: • The four cameras are fixedly mounted on the bottom of the device. • Arranged in a 2×2 matrix (front left, front right, back left, back right) • The optical axis is basically vertically downward. • There is partial overlap in the field of view within the coverage area. Algorithm Description The local images acquired simultaneously from four channels are merged into a continuous, seamless, and unified coordinate system complete working image (one pixel in the merged image occupies only 0.1~1 mm in the actual working plane) for subsequent processing. • Crop / Weed Identification Laser point calculation • Boundary fitting Overall Algorithm Flow (Overview) Four-camera fusion is not a simple stitching process, but a process of "calibration → correction → alignment → fusion → output". The overall steps are as follows: 1) Camera calibration (offline / power-on initialization) 2) Synchronous image acquisition 3) Distortion correction 4) Unified coordinate system mapping 5) Overlapping region feature matching 6) Calculation of fusion weights 7) Seamless splicing and brightness compensation 8) Output blended image Detailed Algorithm Logic Decomposition Step 1: Camera Calibration Objective: To establish a mapping relationship between pixel coordinates and actual working plane coordinates for each camera. Calibration content For each camera, obtain: 1) Intrinsic parameter matrix: Focal length fx, fy Main points: CX, CY 2) Distortion parameters: radial distortion Tangential distortion External parameters: position and orientation relative to the robot chassis coordinate system Step 2: Frame Synchronization Core Logic Four cameras captured data at the same timestamp. Used for: software timestamp alignment, i.e. forced synchronization of sequence frames. Purpose To avoid the following during robot movement: 1) Misaligned splicing 2) Identify drift 3) Laser positioning error Step 3: Image distortion correction (Undistortion) Perform the following for each video feed: 1) Radial distortion correction 2) Perspective fine-tuning The result is a standard image with accurate geometry and consistent proportions, which is a prerequisite for subsequent precise laser positioning. Step 4: Standardize the operation plane coordinate mapping (Homography) Core idea Map all four feeds to a unified "virtual ground coordinate plane". practice: Calculate a homography matrix H for each camera. Convert pixel coordinates to ground coordinates (unit: μm) This can be achieved by: 1) The four pictures are naturally aligned on the same plane. 2) Subsequent laser point marking = directly taking the coordinates of the fused image. Step 5: Overlap Matching Because the four cameras have overlapping fields of view, precise alignment is required. Within the overlapping region: 1) Extract feature points ORB| FAST | SIFT (or). 2) Feature matching. 3) Calculate minute pose deviations. 4) Perform subpixel-level correction. effect 1) Eliminate installation errors. 2) Eliminate micro-displacements caused by vibration (de-jittering). 3) Ensure the continuity of splicing. Step 6: Calculate the Blending Weight Within the overlapping area, it is not a "hard cut". Strategy: 1) Distance-weighted fusion 2) The closer to the center of the camera, the higher the weight. 3) Gaussian weight decay 4) Gradient smooth transition Mathematical expression (illustrated): I = w1 * I1 + w2 * I2 Where w1 + w2 = 1 Step 7: Seamless splicing and brightness compensation Problem to be solved: 1) Differences in brightness between different cameras 2) Color temperature difference 3) Shadow changes Handling method: 1) Brightness normalization 2) Color histogram matching 3) Adaptive local contrast compensation The final result is a complete image with no obvious boundaries and continuous brightness. Step 8: Output the blended image (Final Output) Output result: 1) A high-resolution continuous operation image 1) Corresponds to a real ground coordinate system As: 1) AI crop / weed recognition input 2) Laser dotting coordinate reference 3) Basics of Boundary Fitting 6. Working Logic of a Two-Degree-of-Freedom Laser Emitter-Driven Servo Gimbal System The host computer calculates the laser target point based on the fused working plane coordinates, and obtains the target azimuth and pitch angles of the two-degree-of-freedom servo gimbal through spatial vector inverse kinematics. The angle commands are then sent to the PLC after compensation, and the PLC implements multi-axis servo closed-loop control to ensure that the laser optical axis is accurately aligned with the target position on the working plane. System Overall Architecture (1) Hardware Module Responsibility Industrial computer (host computer) Visual fusion, coordinate calculation, target planning PLC Real-time servo control, motion execution, safety interlocking Two-degree-of-freedom holder Azimuth axis (Yaw) + pitch axis (Pitch) Laser emission module Actual operation executor (2) Definition of coordinate system The system contains 5 coordinate systems: 1) Fusion of image coordinate systems Unit: pixels (u, v) 2) Working plane coordinate system Unit: μm Real ground coordinates from four cameras 3) Robot body coordinate system With the center of the fuselage as the origin 4) Gimbal mechanical coordinate system Azimuth θ Pitch angle φ 5) Laser optical axis coordinate system Laser emission direction vector (3) Overall control process Blend Images → Target Pixels ↓ Pixel to Ground Coordinate Transformation ↓ Ground coordinates → Inverse solution of gimbal angle ↓ Angle command sent to PLC ↓ Servo Closed-Loop Alignment ↓ Laser trigger (4) Detailed algorithm logic breakdown Step 1: Determining the target point in the merged image (host computer) The host computer identifies the following through AI: 1) Weed target center point 2) Laser points on the work boundary Represented as: P_img = (u, v) Step 2: Image fusion → Working plane coordinates (host computer) Using the homography matrix H already established in the fusion algorithm: [X, Y, 1]^T = H · [u, v, 1]^T get: P_plane = (X, Y) This is a real ground point, measured in μm. Step 3: Working plane coordinates → Gimbal space vector (host computer) Given: Laser emitter spatial fixed point: L0 = (x0, y0, z0) Target ground point: P = (X, Y, 0) Calculate the target direction vector: V = P - L0 Step 4: Direction Vector → Inverse Solution of Gimbal Angle (Host Computer) The core inverse kinematics formula for a two-DOF gimbal: Azimuth (Yaw): θ = atan2(Vy, Vx) Pitch: φ = atan2(Vz, √(Vx2 + Vy2)) get: (θ_target, φ_target) Step 5: Angle Compensation and Calibration Correction (Host Computer) Introducing an error compensation model: θ_cmd = θ_target + Δθ_cal + Δθ_dyn φ_cmd = φ_target + Δφ_cal + Δφ_dyn Source of compensation: 1) Installation deviation 2) Mechanical hysteresis 3) Changes in body posture 4) Historical Error Table (Lookup Table) Step 6: Instruction Issuance and Task Allocation (Host Computer → PLC) The host computer only sends: {θ_cmd, φ_cmd, laser power, operating mode} PLC responsible for: 1) Angular interpolation 2) Speed ​​Planning 3) Limit judgment 4) Safety Interlock Step 7: Servo Closed-Loop Control (PLC) Control structure for each axis: Position ring (angle) ↓ Speed ​​Loop ↓ Current / Torque Loop PLC usage: High-resolution absolute encoder Feedforward + PID Angle in place judgment Step 8: Laser Triggering Logic (PLC) When the following conditions are met: |θ_actual ? θ_cmd|<εθ |φ_actual ? φ_cmd|<εφ The visual system allows ↓ Safe zone confirmed ↓ PLC-triggered laser V. Dynamic Compensation and Prediction During the robot's movement: The host computer executes: 1) Target trajectory prediction 2) Delay compensation 3) Gimbal angular velocity feedforward θ_pred = θ(t + Δt) φ_pred = φ(t + Δt) (V) Artificial Intelligence Recognition and Self-Learning Mechanism 1. The system is based on image recognition algorithms, distinguishing between crops and weeds through crop type presets and exclusion logic. After each semi-automatic operation, the operation data generates a learning log package. Data with higher learning value is prioritized and pushed to the trainer. The AI ​​autonomously performs machine learning for manual review and confirmation, or the system is trained and reviewed manually. The database is updated and the recognition model is optimized, thereby continuously improving the model's recognition accuracy through iteration. Process: Perception → Decision → Execution → Feedback → Learning → Update → Redeployment Beneficial effects Compared with the prior art, the present invention has the following beneficial effects: Achieve high-precision, non-contact laser weeding, effectively avoiding damage to seedlings; Full-coverage sweeping can kill grass seeds, grass shoots, plant pathogen spores, insect eggs, and attached insects on the grass stem and the shallow surface of the working area, solving grass damage while suppressing pests and diseases to a certain extent. It supports automatic and semi-automatic collaborative operations, improving adaptability to complex field environments; A multi-laser operation strategy balances weeding efficiency and operational safety; It has continuous self-learning capabilities, and its accuracy improves with long-term use. Reduce labor input and pesticide use; green and environmentally friendly. Attached Figure Description

[0005] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a schematic diagram of the bottom structure of the present invention; Figure 3 This is a schematic diagram of the bottom four-camera matrix layout of the present invention; Figure 4 This is a schematic diagram of the field operation environment of the present invention; Figure 5 This is a schematic diagram of the adjustable suspension and wheel track adjustment structure; Figure 6 This is a schematic diagram of a multi-sensor fusion obstacle avoidance layout; Figure 7 This is a schematic diagram of the camera image fusion process; Figure 8 This is a diagram of the cloud-based remote semi-automatic operation control interface; Detailed Implementation

[0006] The following description, in conjunction with specific embodiments of the present invention, illustrates that the present invention is not limited to these embodiments. In one embodiment, the laser weeding robot of the present invention is applied in greenhouse or open-field farmland environments, automatically moving along the rows of crops via a four-wheel drive system. A perception and positioning system acquires environmental information in real time, an artificial intelligence recognition model distinguishes between crops and weeds, and controls the laser weeding system to perform area scanning or spot spraying operations according to the operating mode. After the operation is completed, the system uploads the relevant data to a cloud database for subsequent model optimization. In other embodiments, the present invention can adjust camera parameters, laser power and operation strategies according to different crop types, growth stages and terrain conditions to adapt to various agricultural application scenarios. For example, in scallion fields planted on high ridges and deep furrows, weed infestation is very serious, with the main weeds being long-climbing vines such as bindweed and nutgrass. This field has the following characteristics: 1. After several hilling operations, deep furrows are formed between the rows of scallion fields. 2. Dense, stubborn weeds have covered the entire ridge and furrow, obscuring the ground. 3. The bindweed climbs on the leaves of scallions and forms a dense cover on the surface of other weeds. This level of weed infestation is very common in large cash crop fields during the summer's rapid weed growth period. At this time, existing laser weed control methods are no longer adequate for this task due to the following reasons: 1. The ridges are high and the furrows are deep, the ridge spacing is not fixed, and the non-adjustable wheel spacing cannot reach deep into the field. 2. The weed density is too high, and the efficiency of spot spraying cannot meet the workload. 3. The density is too high and the identification features have been lost. 4. Forced operation is extremely inefficient, has a very high accidental injury rate, and results in a loss of economic benefits. 5. On the sloping surface of the ridge, the grass covering the target makes it impossible to determine the obscured target. 6. The grass stalks are too large to be effectively inactivated. 7. Some grass stalks have already produced seeds, which will quickly germinate if they fall to the ground. The intelligent weeding robot described in this case can solve the above problems. 1. Adjustable wheel track can accommodate most ridge widths. 2. The area-scanning laser can perform indiscriminate scanning ablation within the red line range. 3. Area scanning does not require identifying every single plant; it only needs to identify the constructed B-Spline red line for full coverage. The area outside the red line is scanned for complete coverage. Plants inside the red line, right next to the crop, are targeted with pulsed shots. 4. It can quickly deactivate and minimize the accidental damage rate. 5. It can completely cover slopes and inclines, penetrating all the way down to the lowest layer of grass, down to the ground. 6. Use both large-scale surface sweeping and spot spraying to ensure inactivation or elimination of the infestation of large weeds. 7. Inactivate the entire plant, including ground-level grass seeds, insect eggs, and pests attached to weeds.

Claims

1. An artificial intelligence-based autonomous walking laser weeding robot for fields, characterized in that, include: Main body of the fuselage; The following systems are installed on the main body of the machine: a four-motor four-wheel drive system; a sensing and positioning system; and a laser weeding system. A control and computing system; a human-computer interaction system; wherein the robot has at least an automatic operation mode and a semi-automatic operation mode.

2. The laser weeding robot according to claim 1, characterized in that: The height and / or width of the main body of the fuselage are adjustable.

3. The laser weeding robot according to claim 1, characterized in that: The sensing and positioning system includes multiple cameras and at least one radar sensor.

4. The laser weeding robot according to claim 3, characterized in that: The images captured by the camera are processed by a fusion and stitching algorithm and fused with radar data to achieve obstacle recognition and avoidance.

5. The laser weeding robot according to claim 1, characterized in that: The laser weeding system includes a surface-scanning laser emitting component and a point-firing laser emitting component.

6. The laser weeding robot according to claim 5, characterized in that: The laser emitting assembly is mounted on a servo gimbal with at least two degrees of freedom.

7. The laser weeding robot according to claim 6, characterized in that: The control and calculation system is used to calculate the laser target point and solve the azimuth and pitch angles of the servo gimbal.

8. The laser weeding robot according to claim 7, characterized in that: The laser emission component is only triggered to operate when the servo gimbal reaches the preset alignment accuracy condition.

9. The laser weeding robot according to claim 1, characterized in that: In semi-automatic operation mode, a smooth curve is used to define the safe area for laser operation.

10. The laser weeding robot according to claim 1, characterized in that: The control and computing system has job data storage and self-learning functions.

Images