Full-automatic weeding robot with binocular vision and weeding method thereof

Abstract

The invention discloses a full-automatic weeding robot with binocular vision and a weeding method thereof.The system comprises a visual camera system, a control system, a walking system, a weeding mechanical system and a power system, and the visual camera system is used for collecting obstacle information and weed distribution coordinate information in a farmland scene; the control system receives the collected obstacle information and weed distribution coordinate information, calculates and identifies weeds in real time and dynamically avoids obstacles, plans an optimal weeding path according to data provided by the Internet of Things cloud system, and sends corresponding control signals to each execution system; the weeding mechanical system receives the control signal and executes a corresponding weeding operation instruction, and the weeding mechanical system comprises a weeding assembly arranged at the front end of the vehicle body support assembly and a weed smashing assembly connected with the weeding assembly; according to the weeding robot and the weeding method thereof, multiple types of weeds can be accurately recognized, path planning and automatic obstacle avoidance capabilities are achieved, the weeds can be smashed, and soil component information can be detected.

Description

Technical Field

[0001] This invention relates to the field of intelligent agricultural machinery technology for unmanned farms, and in particular to a fully automatic weeding robot with binocular vision and its weeding method. Background Technology

[0002] Currently, unmanned farms are systems composed of a series of intelligent mechanical equipment that automate agricultural processes such as sowing, weeding, fertilizing, watering, and harvesting. Intelligent agricultural machinery uses computer algorithms to simulate the capabilities of the human brain, enabling the machinery to perform the necessary agricultural production tasks.

[0003] In the context of automated weeding machinery, the current technological limitations present several problems:

[0004] 1. Traditional weeding blades are prone to uneven cutting heights due to undulating terrain, resulting in bald patches or weed residue in the lawn; 2. Using pesticides for weeding causes environmental pollution and reduces soil fertility; 3. Manually searching for and regularly weeding large areas is time-consuming and labor-intensive; 4. Existing intelligent weeding machines have unstable signals, are easily affected by obstructions, and have poor multi-machine coordination; 5. Existing intelligent weeding equipment has low intelligence, poor recognition accuracy, can only handle a limited variety of weeds and complex situations, and lacks deep learning capabilities; 6. A large amount of repetitive and disordered work leads to low weeding efficiency and an inability to accurately plan weeding paths; 7. The machines are too bulky, have low adaptability to farmland, and cannot be used for weeding between shorter plants; 8. Weeding machines have a single function and lack agricultural information collection and monitoring capabilities.

[0005] In summary, addressing the existing problems in automated weeding machinery, there is an urgent need to research a novel fully automated weeding robot with binocular vision and its weeding method. This robot should utilize spectral principles to analyze the environment, accurately identify various types of weeds, independently plan its path, and automatically avoid obstacles. The designed mechanical structure should be able to uproot and thoroughly crush weeds, while also loosening the soil and detecting soil composition information, thus solving the problems of environmental pollution from pesticide weeding and the time-consuming and labor-intensive nature of large-scale manual weeding. Summary of the Invention

[0006] To address the environmental pollution caused by pesticide weeding and the time-consuming and labor-intensive nature of manual large-scale weeding in existing technologies, a fully automated weeding robot with binocular vision and its weeding method are proposed.

[0007] In a first aspect, embodiments of this application provide a fully automatic weeding robot with binocular vision. The fully automatic weeding robot includes: a frame system, including: a body support assembly and a track support assembly;

[0008] The visual camera system, mounted on the top of the vehicle frame assembly, is used to collect information on obstacles and the coordinates of weed distribution in farmland scenes. The visual camera system includes: a visual camera, an infrared supplementary lighting component, and a high-resolution image sensor.

[0009] The control system is electrically connected to the vision camera system and the Internet of Things (IoT) cloud system. It receives obstacle information and weed distribution coordinate information collected by the vision camera system, calculates and identifies weeds and dynamically avoids obstacles in real time, and plans the optimal weeding path based on the data provided by the IoT cloud system, and sends corresponding control signals to each execution system. The control system includes: an edge computing unit, a deep learning computing unit, and an on-board control unit.

[0010] The walking system, located on the track support assembly and electrically connected to the control system, is used to receive the travel coordinate control signals sent by the control system and control the weeding robot to move in a specified direction. It includes: track assembly and track drive wheels.

[0011] The weeding machinery system is electrically connected to the control system, receives control signals, and executes corresponding weeding operation commands. The weeding machinery system includes: a weeding component set at the front end of the vehicle frame assembly and a weed-crushing component connected to the weeding component. The weeding component includes: a cutting roller blade assembly and a auger assembly; the weed-crushing component includes: a bucket assembly and a crushing component.

[0012] The power system, electrically connected to the vision camera system, control system, walking system, and weeding machinery system, provides kinetic energy to all systems. The power system includes: a power source, an electric motor, and a transmission device.

[0013] In this embodiment of the invention, the fully automated weeding robot further includes:

[0014] The water and fertilizer monitoring system, electrically connected to the control system, is located inside the weed-crushing component. It is used to collect soil data and transmit it to the Internet of Things cloud system through the control system to automatically generate water and fertilizer replenishment suggestions. The water and fertilizer monitoring system includes multiple sets of sensors.

[0015] In this embodiment of the invention, the above-mentioned mowing roller blade assembly includes: a mowing roller, and mowing roller blades uniformly disposed on the surface of the roller. Two pairs of bevel gears are provided at both ends of the mowing roller blade assembly. A spiral rod assembly is connected to the mowing roller blade assembly through the bevel gears. The spiral rod assembly includes: two spiral rods forming a certain angle with the ground.

[0016] In this embodiment of the invention, the bucket assembly is located below the mowing roller blade assembly. The bucket assembly includes a bucket and a conveyor belt. The front end of the bucket is provided with multiple sharp iron tools for inserting into the soil to pull up weeds. The conveyor belt is connected to the bucket. The auger is inclined to the ground and drills into the roots of the weeds. It rotates in a relatively inward manner to loosen the soil around the roots of the weeds and transports the weeds toward the bucket. The mowing roller blades initially crush the transported weeds and then drop them onto the conveyor belt.

[0017] In this embodiment of the invention, the above-mentioned shredding assembly is connected to a conveyor belt. The shredding assembly includes: a mixing tank, a shredding wheel disposed in the mixing tank, and a screen disposed under the shredding wheel. The conveyor belt transports the initially shredded weeds to the mixing tank. After the shredding wheel shreds the weeds again, the fine mixture leaks out of the mixing tank through the screen.

[0018] In this embodiment of the invention, the multiple sets of sensors of the above-mentioned water and fertilizer detection unit are arranged below the screen in the mixing tank. The sensors are soil moisture sensor, soil nitrogen, phosphorus and potassium sensor and soil pH sensor.

[0019] In this embodiment of the invention, the aforementioned edge computing unit is connected to the Internet of Things cloud system and the vehicle control unit, and is used to receive the collected weed coordinate information, and according to the built-in weed recognition algorithm and the weed image pre-training model based on the convolutional neural network, to identify the weed features and mark the weed location coordinates.

[0020] In this embodiment of the invention, the deep learning computing unit is connected to the edge computing unit and the vehicle control unit. After receiving the collected obstacle information and weed distribution coordinates, it uses a heuristic search algorithm to plan the optimal weeding path.

[0021] Secondly, embodiments of this application provide a weeding method, based on the aforementioned fully automated weeding robot with binocular vision, the method comprising:

[0022] Visual navigation steps: The visual camera system collects information on obstacles and the coordinates of weed distribution in the farmland scene;

[0023] Control steps: The control system receives obstacle information and weed distribution coordinate information collected by the vision camera system, calculates and identifies weeds and dynamically avoids obstacles, and plans the optimal weeding path in real time based on the data provided by the Internet of Things cloud system, and sends corresponding control signals to each execution system.

[0024] Navigation steps: The walking system receives the travel coordinate control signal from the control system and controls the robot to move in the specified direction;

[0025] Weeding process: The weeding machinery system receives the weeding control signal and executes the corresponding weeding operation instructions.

[0026] Water and fertilizer testing steps: Collect soil data and transmit it to the IoT cloud system through the control system to automatically generate water and fertilizer replenishment suggestions.

[0027] In this embodiment of the invention, the above-mentioned visual navigation steps include:

[0028] Visual navigation uses a visual camera to collect information about the lines of the planting field ridges and the surrounding environment. It performs threshold segmentation preprocessing on the video images, performs path recognition and tracking, and obtains information about obstacles and changes in the path curve to guide the weeding robot.

[0029] In this embodiment of the invention, the above control steps include:

[0030] For the acquired image information, median filtering is used to filter the image. A moving window with an odd number of pixels is selected, and the gray value of the center pixel of the window is replaced with the gray value seed of the window to eliminate isolated noise points.

[0031] Mathematical morphological filtering is performed on the binarized filtered image to remove defects and burrs in the image, achieving local background smoothing to ensure the accuracy of recognition.

[0032] After morphological correction of the image, the navigation mark edge detection algorithm is used to perform edge detection on the parts of the image with significant local brightness changes. The detection part is the set of points in the image where the pixel gray level is discontinuous or the gray level changes drastically.

[0033] In this embodiment of the invention, the navigation steps include:

[0034] For the path point set output by the path planning, the differential drive model of the visual navigation kinematic model is used to convert it into motor control commands, and the control algorithm is used to optimize the motor action. During the tracking process, if the visual navigation is detected to deviate from the path centerline, the vehicle control unit adjusts the servo angle or the speed of the left and right wheels according to the deviation value to ensure that the visual navigation returns to the correct path.

[0035] In this embodiment of the invention, the above control steps include:

[0036] When the visual camera system detects an obstacle but does not identify weeds, the control system determines it to be an obstacle-only scene, records the current device position and the next target point on the original path, and sends a deceleration and stop command to the walking system. It calls the path algorithm of the deep learning computing unit to set the obstacle as a forbidden area and plans the optimal path from the current position to the point after the original path breakpoint. When the device reaches the obstacle avoidance deviation, it confirms the position, the control system compares the obstacle avoidance deviation with the deviation of the original path, sends a return to the original path command, and the walking system resumes the original direction of travel.

[0037] In this embodiment of the invention, the above control steps include:

[0038] When the vision camera system detects weeds and no obstacles are detected, the control system determines that it is a weed-only scene, records the current position of the equipment and the break point of the original path, calculates the three-dimensional coordinates of the weeds, sends a deceleration and stop command to the walking system, and starts the preheating of the weeding actuator.

[0039] The dynamic window method is used for weed treatment path planning. The shortest path from the current position to the front of the weed is planned. After the path length is ≤2m, the control system sends a weeding start command to the weeding machinery system. After cutting for a preset time, it resets and records the weed treatment completion signal.

[0040] After weeding is completed, the control system calls the path planning algorithm to plan a path back to the original path breakpoint. The walking system executes the path command, returns to the breakpoint, and resumes driving along the original path.

[0041] In this embodiment of the invention, the above control steps include:

[0042] When the visual camera system simultaneously identifies obstacles and weeds, the control system prioritizes obstacles. It first confirms the location of the obstacle, then the location of the weeds. If the distance between the two is ≤1m, the obstacle is dealt with first; if the distance is >1m, the obstacle can be avoided first and then the weeds can be removed.

[0043] The process for obstacle-only scenarios is as follows: stop, obstacle avoidance planning and execution, the equipment bypasses the obstacle to the point following the original path breakpoint, and the position of weeds is monitored in real time during the process.

[0044] After the equipment reaches the location of the weeds, it stops on the original path, executes the weed-only processing procedure, and then returns to the original route.

[0045] Compared with existing technologies, it has the following outstanding advantages:

[0046] 1) This invention's automatic weeding robot is equipped with tracks, a vision camera, an IoT module, belt drive, a motor, a battery, a gear reducer, an ultrasonic sensor, an algorithm controller, and a soil moisture and fertility detection unit. When this soil information detection weeding vehicle is running, it collects surrounding environmental information through a roof-mounted vision camera. The vision camera includes a TOF depth camera and a binocular structured light depth camera to collect information on weed depth and obstacles. Combining this with the plant area in the RGB image, it identifies weeds using a lightweight YOLOv11 model, outputting the pixel coordinates of the weeds in the image. Then, using camera calibration parameters and GPS-RTK data, it calculates the three-dimensional absolute coordinates (X, Y, Z) of the weeds and feeds this information back to the track motor, controlling it to move forward or backward to approach the weeds. The binocular structured light depth camera detects obstacles, outputting the obstacle's depth and contour coordinates. The ultrasonic sensor detects blind spots in real time. If an obstacle is detected, an emergency warning is triggered, causing the tracks to steer differentially to avoid the obstacle.

[0047] 2) The automatic weeding robot of this invention is equipped with a weeding mechanical device, a spiral drill, a blade, a bucket, a conveyor belt, a mixer, and a screen. When it is close to weeds and needs to use the weeding machine to cut grass or loosen soil, the motor is started to rotate and drive the belt drive. Then, the torque is amplified by the gear reducer and the power is output to the blade. While rotating, the blade drives the bevel gear drive, which then rotates the spiral drills at both ends. The rotating spiral drills are inclined to the ground and drill into the roots of the weeds, roll them up and crush them. The soil and weeds are then sent to the crushing device through the conveyor belt, crushed, and then discharged through the filter screen from the robot body, thus achieving the effect of weeding and loosening soil.

[0048] 3) The automatic weeding robot of this invention supports the identification of soil composition and weeds by using a water and fertilizer detection unit and a TOF depth camera to obtain their information. Combined with pixel targets and robot posture, the three-dimensional coordinates of the weeds are accurately calculated. Under the premise of existing path planning, the path is appropriately changed through real-time image recognition to adjust the path to the movement of weeds. The dynamic window method is used to ensure that the robot stops directly in front of the weeds and stops weeding at the required distance. After weeding is completed, the robot automatically triggers repositioning and path planning to resume normal movement.

[0049] 4) Compared to traditional machine learning-based weeding machines, this invention's automatic weeding robot achieves high-precision control throughout the entire process, from "path tracking" to "weed removal." Precision is a core requirement for agricultural automation equipment. This weeding machine achieves breakthroughs in precision across multiple stages through a deep integration of visual navigation and control algorithms, directly improving weeding effectiveness. High path tracking accuracy prevents crop damage, directly enhancing crop protection. The closed-loop process of "image preprocessing → edge detection → PID control" effectively eliminates path recognition errors caused by environmental noise. The PID controller can control the overshoot after path deviation to ≤5% and the adjustment time to ≤0.5s, ensuring the AGV stably follows the edge of the field ridges and avoids crushing crops due to path deviation. Controllable weed positioning and removal accuracy reduces missed or incorrect removal. This method uses a TOF depth camera to obtain the 3D coordinates of the weeds, and combines this with the robot's posture data to accurately calculate the physical location of the weeds. Subsequently, the path is adjusted using a dynamic window method to ensure that the AGV finally stops directly in front of the weeds. This avoids missing weeds and also prevents the accidental removal of surrounding crops due to positioning deviations. It is especially suitable for scenarios where crops and weeds are densely mixed.

[0050] 5) Compared to other weeding machines, the automatic weeding robot of this invention has a stronger environmental adaptability, overcoming the limitations of complex field scenes and reducing the impact of environmental interference. For example, field environments have pain points such as changes in lighting, random obstacles, and noise interference. This solution improves anti-interference capabilities through the integration of multiple technologies, ensuring stable operation of the equipment. Image preprocessing technology solves the problem of "blurred recognition." Regarding noise in field images, this patent uses median filtering to remove isolated noise points while protecting the boundaries of field ridges and weeds from blurring. Furthermore, for spikes and gaps in binarized images, morphological opening operations are used to achieve local background smoothing, avoiding path recognition failure caused by the accumulation of useless information. Even in low-light conditions such as cloudy days or evenings, path recognizability is maintained.

[0051] 6) This invention's automated weeding robot employs multi-sensor fusion obstacle avoidance to handle both dynamic and static obstacles. For static obstacles, depth information is obtained using a binocular structured light depth camera, and the A* algorithm is used to bypass forbidden areas, balancing detour efficiency and preventing equipment lag. For dynamic obstacles, visual tracking and a dynamic window method are combined to adjust the AGV's speed and path in real time without manual intervention. In blind spots of agricultural robots, this patent integrates visual data and ultrasonic sensors to solve the problem of obstacle detection in visual blind spots such as below the AGV, avoiding the risk of collision with low obstacles.

[0052] 7) This invention's automatic weeding robot is a fully intelligent inspection agricultural machine that significantly reduces labor costs, thus adapting to flexible operation needs. Traditional agricultural machinery relies on manual driving and path adjustment, which is inefficient and has high learning costs. This solution achieves lightweight operation through "visual navigation + automatic planning." Visual navigation automatically generates paths by recognizing the edges of field ridges, eliminating the need for manual placement of auxiliary markers such as magnetic strips or QR codes. If the work area needs to be adjusted, only the new path needs to be re-identified through images, without modifying hardware parameters or recalibrating, making it suitable for the flexible operation needs of small farmers with "multiple plots and multiple crops."

[0053] 8) To prevent path deviations during agricultural machinery operation, this invention's automatic weeding robot incorporates an automatic recovery function to reduce manual intervention. This design allows the AGV to automatically reposition and replan its path after deviating due to obstacle avoidance or terrain disturbances, eliminating the need for manual on-site adjustments. This is particularly suitable for unmanned operations in large-scale farmland, reducing the time cost of manual inspections and troubleshooting. Obstacle avoidance and attitude control effectively reduce maintenance costs. The fusion detection of depth cameras and ultrasonic waves can identify obstacles in advance and trigger deceleration or detours, preventing mechanical damage caused by collisions with hard objects. PID control optimizes motor movement, reducing wear and tear on wheels and the transmission system from rapid acceleration and sharp turns, extending equipment lifespan. Furthermore, it offers enhanced adaptability, precise positioning to prevent accidental weeding, and tracking to prevent crushing. The three-dimensional coordinate positioning of weeds and the "weed removal distance docking" control ensure that the weeding mechanism only acts on the weed area and does not come into contact with the surrounding crops; the high precision of path tracking (deviation ≤3cm) can avoid the AGV from crushing the crop seedlings between the ridges, making it especially suitable for weeding operations of seedling crops.

[0054] 9) In farmland where ecological protection is emphasized (such as organic farmland and green food planting bases), the automatic weeding robot of this invention does not require the use of pesticides. It removes weeds physically, and returning the weeds to the field increases soil organic matter, which is in line with the concept of ecological planting. At the same time, soil testing data can help users avoid over-fertilization and irrigation, reduce agricultural non-point source pollution, and protect the ecological environment of farmland.

[0055] 10) In large-scale unmanned farms with an area of ​​≥100 acres, multiple weeding robots of this invention can be deployed and clustered for scheduling via the Internet of Things (IoT) cloud platform. The cloud system rationally allocates the working area of ​​each weeding robot based on the weed density and soil conditions of different farmland zones, avoiding duplicate or missed operations. Simultaneously, it aggregates soil testing data from all weeding robots to generate a comprehensive soil, water, and fertilizer management plan, achieving intelligent and unmanned farmland management. In scenarios involving the cultivation of cash crops such as vegetable greenhouses and orchards (e.g., citrus or vineyards), the narrow body (customizable width ≤1.2m) and small turning radius (≤1.5m) of this weeding robot adapt to narrow row spacing environments, preventing crop damage. Furthermore, the soil, water, and fertilizer monitoring function can monitor changes in soil water and fertilizer levels in greenhouses and orchards in real time. Combined with the precise planting needs of cash crops (e.g., low fertilizer requirements during the seedling stage and high fertilizer requirements during the fruiting stage), it provides accurate water and fertilizer replenishment recommendations, improving crop yield and quality. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0057] Figure 1a This is a front view of the structure of a fully automatic lawnmower with binocular vision proposed in this invention;

[0058] Figure 1b This is a side view of the structure of a fully automatic lawnmower with binocular vision proposed in this invention;

[0059] Figure 1c This is a rear view of the structure of a fully automatic lawnmower with binocular vision proposed in this invention;

[0060] Figure 2 This is a schematic diagram of the overall control system proposed in this invention.

[0061] Figure 3a This is a partial schematic diagram of the lawn mowing device proposed in this invention;

[0062] Figure 3b This is a partial schematic diagram of the lawnmower device proposed in this invention. Figure 2 ;

[0063] Figure 4 This is a schematic diagram of the internal structure of the stirring device proposed in this invention;

[0064] Figure 5 This is a schematic diagram of the water and fertilizer detection unit proposed in this invention;

[0065] Figure 6 This is a schematic diagram of the weeding method proposed in this invention;

[0066] The components in the diagram are: 1. Weeding machinery, 2. Mowing roller blades, 3. Two augers, 4. Two pairs of bevel gears, 5. Bucket, 6. Conveyor belt, 7. Mixing tank, 8. Shredder wheel, 9. Water and fertilizer detection unit, 10. Screen, 11. Track, 12. Vision camera, 13. Infrared supplementary lighting component, 14. Ultrasonic sensor, 15. Internet of Things cloud, 16. Belt drive, 17. Gear reducer, 18. Adjustment bracket, 19. Battery, 20. Electric motor, 21. Edge computing unit, 22. Deep learning calculator, 23. Vehicle controller. Detailed Implementation

[0067] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0068] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0069] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0070] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0071] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0072] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0073] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0075] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0076] The present invention aims to propose a fully automatic lawnmower with binocular vision and a weeding method thereof, including a weeding mechanical device, which consists of a mowing roller blade and two augers. The mowing roller blade is fitted with two pairs of bevel gears at both ends, and the shaft end is connected to the bevel gears by a key. The augers are inclined at a certain angle to the ground, and the tail shaft end is also connected to the bevel gears by a key to achieve circumferential fixation. The weeding machinery includes a bucket and a conveyor belt. Several sharp iron prongs at the front of the bucket reduce resistance when scooping into the soil. Triangular metal baffles are arranged on both sides of the bucket to prevent harvested weeds and excavated soil from falling out of the machine. The conveyor belt is positioned behind the front of the bucket and below the weeding machinery, used to transport the excavated weeds and soil upwards to a mixing device. The mixing device includes a housing, a shredding wheel, and a screen. The top of the housing has an inlet to receive the conveyor belt, and the bottom has an outlet to discharge the shredded weeds and loosen the soil. The shredding wheel is located inside the housing cavity, shredding the weeds and soil fed into the housing. When sufficiently finely shredded, the weeds and soil can pass through the screen to the outlet. The weeding vehicle moves via tracks and achieves steering using differential speed. The entire vehicle is also equipped with a vision camera, an IoT cloud module, a water and fertilizer monitoring unit, a motor and power supply, and a series of belt drives and gear reduction devices.

[0077] This application describes an automatic weeding robot that can accurately identify weeds through a visual camera and machine learning, plan its path through the Internet of Things (IoT) cloud, avoid obstacles and crops, and uproot the target weeds, crush them together with the soil, and discharge them. During the soil loosening process, the water and fertilizer detection unit detects the moisture and fertility of the soil being uprooted and feeds the agricultural information back to the IoT cloud so that the smart farm can take the next steps in water and fertilizer management.

[0078] The system of this application embodiment will be described in detail below with reference to specific embodiments:

[0079] Example 1

[0080] like Figure 1a , Figure 1b and Figure 1c As shown in the embodiment of this application, a fully automatic weeding robot with binocular vision is proposed. The fully automatic weeding robot includes: a frame system 100, a vision camera system 200, a control system 300, a walking system 400, a weeding machinery system 500, a power system 600, and a water and fertilizer detection system 700.

[0081] The chassis system includes 100: body support assembly and track support assembly;

[0082] The visual camera system 200 is mounted on the top of the vehicle frame assembly via an adjustable bracket 18. It is used to collect obstacle information and weed distribution coordinate information in farmland scenes. The visual camera system 200 includes: a visual camera 12, an infrared supplementary lighting component, and a high-resolution image sensor.

[0083] Specifically, in this embodiment of the invention, the visual camera 12 is the core sensing unit, equipped with an infrared supplementary lighting component and a high-resolution image sensor. It can achieve accurate imaging of farmland scenes under complex lighting conditions such as strong light, cloudy days, and dusk, with an imaging resolution of 1920×1080 pixels and a stable frame rate of 30fps, ensuring real-time capture of the morphological differences between crops and weeds. The visual camera is mounted on an adjustable gimbal structure on the top of the vehicle, with a horizontal rotation angle of 0-360° and a vertical pitch angle of -15° to 60°, covering a monitoring range of 5 meters in radius around the vehicle.

[0084] The visual camera 12 is mounted on an adjustable gimbal structure on the top of the vehicle. It can rotate to cover a monitoring range of 5 meters around the vehicle. It is equipped with an infrared supplementary lighting component and a high-resolution image sensor, which can achieve accurate imaging of farmland scenes under complex lighting conditions such as strong light, cloudy days, and dusk, and capture the morphological differences of crops and weeds in real time.

[0085] like Figure 2 As shown, the control system 300 is electrically connected to the vision camera system and the Internet of Things (IoT) cloud system. It receives obstacle information and weed distribution coordinate information collected by the vision camera system 200, calculates and identifies weeds and dynamically avoids obstacles in real time, and plans the optimal weeding path based on the data provided by the IoT cloud system, and sends corresponding control signals to each execution system. The control system includes: an edge computing unit 21, a deep learning computing unit 22, and an on-board control unit 23.

[0086] In this embodiment of the invention, the edge computing unit 21 is connected to the Internet of Things cloud system 15 and the vehicle control unit 23, and is used to receive the collected weed coordinate information, identify the weed features and mark the weed position coordinates according to the built-in weed recognition algorithm and the weed image pre-training model based on the convolutional neural network.

[0087] Specifically, in this embodiment of the invention, image data is transmitted via vehicle-mounted Ethernet to the edge computing unit 21 associated with the IoT cloud 15. This unit has a built-in weed recognition algorithm—a lightweight model based on a convolutional neural network (CNN). Through prior training on over 100,000 sets of image samples of different crops (wheat, corn, rice, etc.) and weeds (barnyard grass, foxtail grass, etc.), the accuracy rate for weed recognition can reach over 98.5%. The algorithm can automatically extract the leaf shape, texture, and color features of weeds, distinguish between crops and weeds, and mark the location coordinates of the weeds (accuracy error ≤ 5cm).

[0088] In this embodiment of the invention, the deep learning computing unit 22 is connected to the edge computing unit 21 and the vehicle control unit 23. After receiving the collected obstacle information and weed distribution coordinates, it uses a heuristic search algorithm to plan the optimal weeding path.

[0089] Specifically, in this embodiment of the invention, the path planning function is implemented based on the fusion of GIS farmland map and real-time environmental data from the IoT cloud 15. After receiving information on obstacles (stones, field ridges, irrigation equipment, etc.) and weed distribution coordinates from the visual camera 12, the cloud system activates the deep learning calculator 22 and uses the A* heuristic search algorithm to plan the optimal weeding path, with a path planning response time of ≤0.5 seconds. Simultaneously, the system supports dynamic obstacle avoidance. When the vehicle encounters sudden obstacles (such as small field animals or temporarily placed tools) while moving along the tracks 11, the visual camera 12 captures the obstacle information, and the edge computing unit 21 can replan the local path within 0.2 seconds, ensuring that obstacles are avoided and weed removal tasks are not missed.

[0090] The walking system 400 is mounted on the track support assembly and electrically connected to the control system 300. It is used to receive the travel coordinate control signal sent by the control system 300 and control the weeding robot to move in a specified direction. It includes: track assembly 11 and track drive wheels.

[0091] like Figure 3a and Figure 3b As shown, the weeding machinery system 500 is electrically connected to the control system, receives control signals, and executes corresponding weeding operation commands. The weeding machinery system includes: a weeding component set at the front end of the vehicle body support assembly and a weeding shredder component connected to the weeding component. The weeding component includes: a cutting roller blade assembly and a auger assembly; the weeding shredder component includes: a bucket assembly and a shredder assembly.

[0092] The lawn mowing roller blade assembly includes: a lawn mowing roller, and lawn mowing roller blades 2 evenly distributed on the surface of the roller. Two pairs of bevel gears 4 are provided at both ends of the lawn mowing roller blade assembly. The auger assembly is connected to the lawn mowing roller blade assembly through the bevel gears 4. The auger assembly includes: two augers 3 that form a certain angle with the ground.

[0093] The bucket assembly is located below the mowing roller blade assembly. The bucket assembly includes a bucket 5 and a conveyor belt 6. The front end of the bucket 5 is equipped with multiple sharp iron tools for inserting into the soil to pull up weeds. The conveyor belt 6 is connected to the bucket 5. The auger 3 is inclined to the ground and drills into the roots of the weeds. It rotates in a relatively inward manner to loosen the soil around the roots of the weeds and transports the weeds toward the bucket 5. The mowing roller blade 2 initially crushes the transported weeds and then drops them onto the conveyor belt 6.

[0094] like Figure 4As shown, the crushing assembly is connected to the conveyor belt 6. The crushing assembly includes: a mixing tank 7, a crushing wheel 8 disposed in the mixing tank, and a screen 10 disposed under the crushing wheel. The conveyor belt 6 transports the initially crushed weeds to the mixing tank 7. After the crushing wheel 8 crushes the weeds again, the fine mixture leaks out of the mixing tank through the screen 10.

[0095] Specifically, in this embodiment of the invention, the weeding machinery system 500 consists of a linkage structure composed of a mowing roller blade 2 and two augers 3. The mowing roller blade has a diameter of 30cm and 12 sets of high-strength alloy blades evenly distributed on its surface. The blade edges are carbonized, achieving a hardness of HRC60 or higher, and can cut weed stems with a diameter ≤1cm. The two pairs of bevel gears 4 at both ends of the mowing roller blade are made of 45# steel with a heat treatment process and a transmission ratio of 1:2. They are connected to the augers 3 via a key to transmit power, ensuring that the speed of the augers matches the speed of the roller blades. The angle between the two augers and the ground can be infinitely adjusted within the range of 15°-45° via the adjusting bracket 18. The pitch of the auger blades on the surface of the auger is 10cm, and the blade edges have a serrated structure. When rotating, it can loosen the soil around the roots of the weeds and simultaneously transport the weeds towards the bucket 5.

[0096] Specifically, in this embodiment of the invention, the bucket 5 is made of manganese steel welded together, with a bucket opening width of 80cm and a depth of 25cm. The six sharp iron prongs at the front end are made of tungsten steel, each 15cm long, with a tip angle of 30°. The resistance when inserted into the soil is ≤500N, allowing it to penetrate 10-15cm into the soil to uproot weeds. The triangular metal baffles on both sides of the bucket are 20cm high and welded to the bucket body as a whole. The edges of the baffles are rounded to prevent scratching crop roots.

[0097] Specifically, in this embodiment of the invention, the conveyor belt 6 is made of polyurethane, with a width of 60cm and anti-slip protrusions on its surface. The conveying speed can be adjusted from 0.5 to 2m / s via the speed regulator of motor 17. The conveyor belt has an inclination angle of 30°, with one end connected to the bucket outlet and the other end extending to the inlet of the mixing tank 7, ensuring that no weeds and soil mixture is missed during conveying. The mixing tank 7 is made of stainless steel, with a volume of 0.5m³. The top inlet size matches the width of the conveyor belt, and the bottom outlet is equipped with an electric gate to control the discharge speed. The shredding wheel 8 inside the tank has a double-wheel symmetrical structure, with each shredding wheel having a diameter of 20cm and 8 sets of staggered shredding teeth distributed on its surface. The shredding wheel rotates at 1500r / min and is driven by an independent motor via belt drive 14, which can shred the weeds and soil mixture to a fine state with a particle size ≤5mm. The screen 10 is made of stainless steel woven mesh with a mesh size of 5mm. It is installed at an angle of 15° below the crushing wheel, allowing finely crushed mixtures to pass through the screen to the discharge port. Larger impurities that do not pass through the screen (such as stones, weed roots, and hard lumps) can be periodically discharged through the cleaning door on the side of the box. The conveyor belt 6 is used to collect the soil and weeds that are rolled in, preventing them from falling off, and transporting them to the mixer for crushing.

[0098] The power system 600 is electrically connected to the vision camera system, control system, walking system and weeding machinery system, and is used to provide power to all systems. The power system includes: a power source consisting of a battery 19, an electric motor 20, a belt drive 16 and a gear reducer 17.

[0099] Specifically, in this embodiment of the invention, the power source of the weeding vehicle is a battery 19, which uses a lithium iron phosphate battery pack with a capacity of 200Ah and a rated voltage of 48V. It supports fast charging (fully charged in 2 hours) and can work continuously for 8 hours on a full charge. The electrical energy output from the battery supplies the motor 20, which is a permanent magnet synchronous motor with a rated power of 5kW and a rated speed of 3000r / min. It is combined with a belt drive 16 through a gear reducer 17 (reduction ratio of 1:50). The belt drive 16 is responsible for overload protection to prevent debris from getting caught and causing the machine to jam and burn out. The entire mechanical mechanism distributes power to mechanical components such as the mowing roller blades, auger, conveyor belt, and shredder, as well as the drive wheels of the track 11. The track 11 is a rubber track with a width of 20cm and a ground contact length of 1.2m. The track surface has anti-slip patterns, and the ground contact pressure is ≤50kPa, allowing it to travel smoothly on muddy and soft farmland soil without getting stuck. The walking system employs differential steering, achieving steering by adjusting the speed difference between the left and right tracks. The minimum turning radius is 1.5m, meeting the operational needs of narrow-row farmland. Gear reducer 17 amplifies the output torque for better weed harvesting.

[0100] like Figure 5As shown, the water and fertilizer monitoring system 700 includes a water and fertilizer monitoring unit 9 electrically connected to the control system, located inside the weed-crushing assembly. This unit collects soil data and transmits it to an IoT cloud system via the control system to automatically generate water and fertilizer replenishment recommendations. The water and fertilizer monitoring system includes multiple sets of sensors. These sensors, located below the screen inside the mixing tank, are soil moisture sensors, soil nitrogen, phosphorus, and potassium sensors, and soil pH sensors.

[0101] Specifically, in this embodiment of the invention, the water and fertilizer detection unit 9 consists of three sets of sensors: a soil moisture sensor, a soil nitrogen, phosphorus, and potassium sensor, and a soil pH sensor. The sensor probes are made of corrosion-resistant alloy and are installed below the screen 10 inside the mixing tank 7, in direct contact with the finely broken soil. The soil moisture sensor uses the frequency domain reflectance (FDR) principle, with a measurement range of 0-100% volumetric water content and an accuracy of ±2%. The soil nitrogen, phosphorus, and potassium sensors use the ion-selective electrode method, with measurement ranges of 0-500 mg / kg (nitrogen), 0-200 mg / kg (phosphorus), and 0-300 mg / kg (potassium), respectively, and an accuracy of ±5%. The soil pH sensor has a measurement range of 3.5-9.5 and an accuracy of ±0.1 pH. The soil data collected by the sensors is transmitted to the vehicle controller 23 via an RS485 bus. After filtering and calibrating the data, the controller uploads it to the IoT cloud 15 via a 4G / 5G module.

[0102] The cloud system associates the received soil data with farmland zoning information to generate a soil water and fertilizer content heat map. It also automatically generates water and fertilizer replenishment suggestions based on crop growth cycle requirements (such as nitrogen requirements during wheat jointing and potassium requirements during corn grain filling), including parameters such as fertilizer type, fertilizer amount, and irrigation amount. Users can view the data and suggestions through a mobile app or computer backend and remotely control farmland water and fertilizer equipment to perform replenishment operations.

[0103] Example 2

[0104] like Figure 6 As shown, this application embodiment provides a weeding method. Based on the aforementioned fully automated weeding robot with binocular vision, the method includes:

[0105] Visual navigation step 101: The visual camera system collects information on obstacles and the coordinates of weed distribution in the farmland scene;

[0106] Control step 102: The control system receives obstacle information and weed distribution coordinate information collected by the vision camera system, calculates and identifies weeds and dynamically avoids obstacles, and plans the optimal weeding path in real time based on the data provided by the Internet of Things cloud system, and sends corresponding control signals to each execution system.

[0107] Navigation step 103: The walking system receives the travel coordinate control signal from the control system and controls the robot to move in the specified direction;

[0108] Step 104: The weeding machinery system receives the weeding control signal and executes the corresponding weeding operation instructions;

[0109] Water and fertilizer testing step 105: Collect soil data and transmit it to the Internet of Things cloud system through the control system to automatically generate water and fertilizer replenishment suggestions.

[0110] In this embodiment of the invention, the visual navigation step 101 includes:

[0111] Visual navigation uses a visual camera to collect information about the lines of the planting field ridges and the surrounding environment. It performs threshold segmentation preprocessing on the video images, performs path recognition and tracking, and obtains information about obstacles and changes in the path curve to guide the weeding robot.

[0112] In this embodiment of the invention, the control step 102 includes:

[0113] For the acquired image information, median filtering is used to filter the image. A moving window with an odd number of pixels is selected, and the gray value of the center pixel of the window is replaced with the gray value seed of the window to eliminate isolated noise points.

[0114] Mathematical morphological filtering is performed on the binarized filtered image to remove defects and burrs in the image, achieving local background smoothing to ensure the accuracy of recognition.

[0115] After morphological correction of the image, the navigation mark edge detection algorithm is used to perform edge detection on the parts of the image with significant local brightness changes. The detection part is the set of points in the image where the pixel gray level is discontinuous or the gray level changes drastically.

[0116] In this embodiment of the invention, the navigation step 103 includes:

[0117] For the path point set output by the path planning, the differential drive model of the visual navigation kinematic model is used to convert it into motor control commands, and the control algorithm is used to optimize the motor action. During the tracking process, if the visual navigation is detected to deviate from the path centerline, the vehicle control unit adjusts the servo angle or the speed of the left and right wheels according to the deviation value to ensure that the visual navigation returns to the correct path.

[0118] In this embodiment of the invention, the control step 102 includes:

[0119] When the visual camera system detects an obstacle but does not identify weeds, the control system determines it to be an obstacle-only scene, records the current device position and the next target point on the original path, and sends a deceleration and stop command to the walking system. It calls the path algorithm of the deep learning computing unit to set the obstacle as a forbidden area and plans the optimal path from the current position to the point after the original path breakpoint. When the device reaches the obstacle avoidance deviation, it confirms the position, the control system compares the obstacle avoidance deviation with the deviation of the original path, sends a return to the original path command, and the walking system resumes the original direction of travel.

[0120] In this embodiment of the invention, the control step 102 includes:

[0121] When the vision camera system detects weeds and no obstacles are detected, the control system determines that it is a weed-only scene, records the current position of the equipment and the break point of the original path, calculates the three-dimensional coordinates of the weeds, sends a deceleration and stop command to the walking system, and starts the preheating of the weeding actuator.

[0122] The dynamic window method is used for weed treatment path planning. The shortest path from the current position to the front of the weed is planned. After the path length is ≤2m, the control system sends a weeding start command to the weeding machinery system. After cutting for a preset time, it resets and records the weed treatment completion signal.

[0123] After weeding is completed, the control system calls the path planning algorithm to plan a path back to the original path breakpoint. The walking system executes the path command, returns to the breakpoint, and resumes driving along the original path.

[0124] In this embodiment of the invention, the control step 102 includes:

[0125] When the visual camera system simultaneously identifies obstacles and weeds, the control system prioritizes obstacles. It first confirms the location of the obstacle, then the location of the weeds. If the distance between the two is ≤1m, the obstacle is dealt with first; if the distance is >1m, the obstacle can be avoided first and then the weeds can be removed.

[0126] The process for obstacle-only scenarios is as follows: stop, obstacle avoidance planning and execution, the equipment bypasses the obstacle to the point following the original path breakpoint, and the position of weeds is monitored in real time during the process.

[0127] After the equipment reaches the location of the weeds, it stops on the original path, executes the weed-only processing procedure, and then returns to the original route.

[0128] Specifically, in this embodiment of the invention, in order to achieve path planning, given the complexity of the environment, a visual navigation AGV is adopted. Image processing technology is used to identify and track the path, guiding the weeding vehicle. Visual navigation can also acquire obstacle information and path curve changes. Simultaneously, it simplifies path setting and modification, facilitating timely and convenient path changes after obstacle identification. The visual navigation AGV utilizes CCD cameras to collect images of planting field ridges and stripes, employing image processing and analysis to obtain environmental information surrounding the small agricultural machinery. The video images are preprocessed and thresholded for path identification and tracking. This improves the accuracy of path planning and reduces the accumulation of useless information and poor recognizability caused by environmental limitations and random interference.

[0129] Specifically, in the specific embodiment of the present invention, in the image filtering part, in order to achieve the effect of removing noise and protecting the target boundary from becoming blurred, a median filtering method is adopted. Specifically, a moving window containing an odd number of pixels is selected, and the gray value of the center pixel of the window is replaced with the gray value seed of the window, thereby eliminating isolated noise points. Its mathematical expression is as follows:

[0130]

[0131] f(x, y) Original image

[0132] Image after processing g(x,y)

[0133] 2D template

[0134] Even in a binarized image, a few scattered points may still remain, and the edges of the black areas are not very clear, exhibiting jagged edges and gaps. Therefore, mathematical morphological filtering is needed to smooth the local background of the binarized image to ensure accurate recognition. Mathematical morphology mainly involves morphological dilation, morphological difference erosion, opening, closing, and erosion operations. These operations can perform simple smoothing of the image and detect singularities. Based on the binarization results, we need to remove gaps and jagged edges while preserving the original image features. Therefore, opening operations can be performed to make the black edges clearer, facilitating edge detection.

[0135] Specifically, in this embodiment of the invention, after morphological correction of the image, a navigation marker edge detection algorithm is used to detect areas of significant local brightness change in the image. The detected area is a set of points within the image where pixel grayscale is discontinuous or where grayscale changes drastically. Edge detection is used as an important feature for image segmentation and a crucial foundation for image analysis, providing data support for path planning. The path point set output by the path planning module is converted into motor control commands through a differential drive model of the AGV kinematic model. A PID control algorithm is then used to optimize motor actions. For example, during tracking, if the visual system detects that the AGV deviates from the path centerline, the PID controller adjusts the servo angle or the speed of the left and right wheels based on the deviation value to ensure that the AGV returns to the correct path, improving motion stability. The PID parameters are adjusted to ensure that the overshoot is ≤5% and the settling time is ≤0.5s.

[0136] Specifically, in this embodiment of the invention, after path recognition and planning are completed, considering other situations that may occur on the path, many other procedures are extended, such as obstacle recognition and avoidance, and obstacle avoidance path adjustment is added to prevent agricultural machinery from deviating from the predetermined route. Firstly, there is obstacle recognition technology. The three key technologies in obstacle recognition are detection, tracking, and localization. Here, a depth camera based on structured light from binocular vision is selected. The light emitted by the depth camera generates relatively random but fixed spot patterns. After these spots hit the object, their positions captured by the camera vary depending on the distance from the camera. Then, the offset of the half-point of the captured image from the calibrated standard cluster at different positions is calculated. The distance between the object and the camera is calculated using parameters such as the camera position and sensor size. Thus, the obstacle target is extracted, and its position and size in the image are identified, thereby obtaining the obstacle's depth information. Using this depth information, a combination of static and dynamic obstacle avoidance methods is adopted to replan the obstacle avoidance path. In static obstacle avoidance, when a fixed obstacle (such as a rock) is detected, the path planning module uses the A* algorithm, treating the obstacle as a forbidden zone, to replan the optimal detour path from the current position to the target point. Simultaneously, it calculates the length and time cost of the detour path to ensure transportation efficiency. In dynamic obstacle avoidance, if a moving object, such as a small wild animal, is detected, the speed and direction of the obstacle are obtained through visual tracking. A dynamic window method is used to adjust the AGV's speed and path in real time to avoid collisions. For example, when an object moves in front of the AGV, the AGV first slows down and then slightly adjusts its path to the side to avoid it. Robust optimization is employed here, integrating visual data and ultrasonic waves to address blind spots, such as obstacles below the AGV. An anomaly recovery mechanism is added, automatically triggering repositioning and path planning after the AGV deviates from its path to resume normal movement.

[0137] Specifically, in the specific embodiments of the present invention, the avoidance methods and path planning under different scenarios are discussed below:

[0138] Scenario 1: Only obstacles are encountered (no weeds)

[0139] When the binocular camera detects an obstacle and the TOF camera does not identify weeds, the system classifies it as an "obstacle-only scene." Simultaneously, it records the current device position and the next target point on the original path. The processing module then sends a "decelerate and stop" command to the differential motor via the CAN bus, reducing the motor speed from its rated value to 0. The system also records the "breakpoint" on the original path for later regression. The A* algorithm is invoked to designate the obstacle as a "forbidden zone," planning the optimal path from the current position to the point after the breakpoint on the original path, ensuring the detour length is longer than the original path to avoid crop areas. When the device reaches the obstacle avoidance deviation, the IMU and GPS-RTK jointly confirm the position. The processing module compares the obstacle avoidance deviation with the deviation of the original path and sends a "return to original path" command. The motor resumes its original direction of travel and continues along the original path towards the return position.

[0140] Scenario 2: Encountering only weeds (no obstacles)

[0141] When the TOF camera detects weeds, and the binocular camera and ultrasonic sensor do not detect any obstacles, the system determines it to be a "weed-only scene," records the current device position and the breakpoint of the original path, and calculates the 3D coordinates of the weeds. It then sends a "decelerate and stop" command to the motor, stopping the device within 0.8m and simultaneously initiating the preheating of the weeding actuator. Weeding path planning: Using a dynamic window method, a short path is planned from the current position to the front of the weeds. Once the path length is ≤2m, the processing module sends a "weeding start" command to the weeding mechanism. After cutting for 3 seconds, it resets and records the weeding completion signal. After weeding, the processing module calls the path planning algorithm to plan a path from the Papproach back to the original breakpoint. The motor executes the path command, and upon returning to the breakpoint, confirms the deviation from the original path is ≤2cm, then resumes the original path.

[0142] Scenario 3: Encountering both obstacles and weeds simultaneously

[0143] The system defaults to "obstacle priority handling." It first uses a binocular camera to confirm the obstacle's location, then a TOF camera to confirm the weed's location. If the distance between the two is ≤1m, the obstacle is handled first; if the distance is >1m, it can be handled in the order of "obstacle avoidance first, then weed removal." Similar to the "stop-obstacle avoidance planning-obstacle avoidance execution" process in Scenario 1, the device avoids the obstacle to the point following the break in the original path, monitoring the weed's location in real time. After reaching the weed's location, the device stops on the original path and executes the "weed removal" process in Scenario 2. Then, it returns to the original route, similar to Scenario 1 and Scenario 2.

[0144] Image data is transmitted via vehicle-mounted Ethernet to the edge computing unit 21 associated with the IoT cloud 15. This unit has a built-in weed recognition algorithm—a lightweight model based on a convolutional neural network (CNN). Through prior training on multiple sets of image samples of different crops and weeds, the accuracy rate of weed recognition can reach over 98.5%. The algorithm can automatically extract the leaf shape, texture, and color features of weeds, distinguish between crops and weeds, and mark the location coordinates of the weeds (accuracy error ≤ 5cm). According to claim 1, a fully automatic lawnmower with binocular vision and its weeding method are characterized in that: the path planning function is realized by fusing the GIS farmland map and real-time environmental data of the IoT cloud 15. After receiving the obstacle (stones, field ridges, irrigation equipment, etc.) information and weed distribution coordinates fed back by the visual camera 12, the cloud system activates the deep learning calculator 22 and uses the A* heuristic search algorithm to plan the optimal weeding path, with a path planning response time ≤ 0.5 seconds. Meanwhile, the system supports dynamic obstacle avoidance. When the vehicle encounters sudden obstacles (such as small animals in the field or temporarily placed tools) while moving along the tracks 11, the visual camera 12 captures the obstacle information, and the edge computing unit 21 can replan the local path within 0.2 seconds to ensure that the obstacle is avoided and the weed clearing task is not missed.

[0145] As described above, the method of the present invention can be implemented well.

[0146] In summary, this fully automatic lawnmower with binocular vision and its weeding method accurately analyzes and locates weeds in the surrounding environment through a vision camera 12, infrared supplementary lighting component 13, ultrasonic sensor 14, and adjustment bracket 18 at the top of the device. Then, the edge computing unit 21, deep learning calculator 22, and vehicle controller 23 calculate and determine the most reasonable running trajectory, controlling the drive track 11 to move towards the target. When the weeding machine 500 approaches the target weeds, its mowing roller blades 2 drive two augers 3 through bevel gears 4 at both ends, tilting... Inserted into the ground, the weeds are uprooted and crushed, then fall onto the bucket 5 and conveyor belt 6, transporting them to the inlet of the mixing tank 7. After being crushed by the internal crushing wheel 8, they pass through the screen 10 and are discharged to the outlet. The mixing tank contains a water and fertilizer detection unit 9, which detects the soil composition and uploads the agricultural information to the Internet of Things cloud 15. The mechanical components of the entire device also include a belt drive 16 and a gear reducer 17, which are used to transmit and amplify the energy output from the battery 19 and the motor 20, ultimately achieving the purpose of precise and effective weeding and collecting soil information.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A fully automated weeding robot equipped with binocular vision, characterized in that, The fully automated weeding robot includes: a chassis system, including: a body support assembly and a track support assembly; A visual camera system is installed on the top of the vehicle body support assembly to collect obstacle information and weed distribution coordinate information in farmland scenes. The visual camera system includes: a visual camera, an infrared supplementary lighting component, and a high-resolution image sensor. The control system is electrically connected to the vision camera system and the Internet of Things (IoT) cloud system. It receives obstacle information and weed distribution coordinate information collected by the vision camera system, calculates and identifies weeds and dynamically avoids obstacles in real time, and plans the optimal weeding path based on the data provided by the IoT cloud system, and sends corresponding control signals to each execution system. The control system includes: an edge computing unit, a deep learning computing unit, and an on-board control unit. The walking system, which is installed on the track support assembly and electrically connected to the control system, is used to receive the travel coordinate control signal sent by the control system and control the weeding robot to move in a specified direction. It includes: track assembly and track drive wheels. The weeding machinery system is electrically connected to the control system, receives the control signal, and executes corresponding weeding operation commands. The weeding machinery system includes: a weeding component disposed at the front end of the vehicle body support assembly and a weed-crushing component connected to the weeding component. The weeding component includes: a cutting roller blade assembly and a auger assembly; the weed-crushing component includes: a bucket assembly and a crushing component. The power system, electrically connected to the vision camera system, control system, walking system, and weeding machinery system, is used to provide kinetic energy to all systems. The power system includes a power source, an electric motor, and a transmission device.

2. The fully automated weeding robot with binocular vision according to claim 1, characterized in that, The fully automated weeding robot also includes: A water and fertilizer monitoring system, electrically connected to the control system, is installed inside the weed-crushing component. It is used to collect soil data and transmit it to the Internet of Things cloud system through the control system to automatically generate water and fertilizer replenishment suggestions. The water and fertilizer monitoring system includes multiple sets of sensors.

3. The fully automated weeding robot with binocular vision according to claim 1 or 2, characterized in that, The mowing roller blade assembly includes: a mowing roller, and mowing roller blades evenly disposed on the surface of the roller. Two pairs of bevel gears are provided at both ends of the mowing roller blade assembly. The auger assembly is connected to the mowing roller blade assembly through the bevel gears. The auger assembly includes: two augers that form a certain angle with the ground.

4. The fully automated weeding robot with binocular vision according to claim 3, characterized in that, The bucket assembly is located below the mowing roller blade assembly. The bucket assembly includes a bucket and a conveyor belt. The front end of the bucket is equipped with multiple sharp iron tools for inserting into the soil to pull up weeds. The conveyor belt is connected to the bucket. The auger is inclined to the ground and drills into the roots of the weeds. It rotates in a relatively inward manner to loosen the soil around the roots of the weeds and transports the weeds toward the bucket. The mowing roller blades work with the bucket to initially crush the transported weeds and then drop them onto the conveyor belt.

5. The fully automated weeding robot with binocular vision according to claim 4, characterized in that, The shredding assembly is connected to the conveyor belt. The shredding assembly includes: a mixing tank, a shredding wheel disposed in the mixing tank, and a screen disposed under the shredding wheel. The conveyor belt transports the initially shredded weeds to the inlet of the mixing tank. After the weeds are shredded again by the shredding wheel, the fine mixture leaks through the screen to the outlet of the mixing tank.

6. The fully automated weeding robot with binocular vision according to claim 2, characterized in that, The multiple sensors of the water and fertilizer detection unit are located below the screen inside the mixing tank. The sensors are soil moisture sensor, soil nitrogen, phosphorus and potassium sensor and soil pH sensor.

7. The fully automated weeding robot with binocular vision according to claim 1, characterized in that, The edge computing unit connects to the IoT cloud system and the vehicle control unit, and is used to receive the collected weed coordinate information. Based on the built-in weed recognition algorithm and the weed image pre-training model based on the convolutional neural network, it identifies the weed features and marks the weed location coordinates.

8. The fully automated weeding robot with binocular vision according to claim 1, characterized in that, The deep learning computing unit is connected to the edge computing unit and the vehicle control unit. It is used to receive the collected obstacle information and weed distribution coordinates, and then use a heuristic search algorithm to plan the optimal weeding path.

9. A weeding method, comprising a fully automated weeding robot with binocular vision as described in any one of claims 1-8, characterized in that, The method includes: Visual navigation steps: The visual camera system collects obstacle information and weed distribution coordinate information in the farmland scene; Control steps: The control system receives obstacle information and weed distribution coordinate information collected by the vision camera system, calculates and identifies weeds and dynamically avoids obstacles, and plans the optimal weeding path in real time based on the data provided by the Internet of Things cloud system, and sends corresponding control signals to each execution system. Navigation steps: The walking system receives the travel coordinate control signal from the control system and controls the robot to move in the specified direction; Weeding steps: The weeding machinery system receives the weeding control signal and executes the corresponding weeding operation instructions; Water and fertilizer testing steps: Soil data is collected and transmitted to the Internet of Things cloud system through the control system to automatically generate water and fertilizer replenishment suggestions.

10. The weeding method according to claim 9, characterized in that, The visual navigation steps include: Visual navigation uses the visual camera to collect information on the lines of the planting field ridges and the surrounding environment, performs threshold segmentation preprocessing on the video images, identifies and tracks the path, and obtains information on obstacles and changes in the path curve to guide the weeding robot.

11. The weeding method according to claim 9, characterized in that, The control steps include: For the acquired image information, median filtering is used to filter the image. A moving window with an odd number of pixels is selected, and the gray value of the center pixel of the window is replaced with the gray value seed of the window to eliminate isolated noise points. Mathematical morphological filtering is performed on the binarized filtered image to remove defects and burrs in the image, achieving local background smoothing to ensure the accuracy of recognition. After morphological correction of the image, the navigation mark edge detection algorithm is used to perform edge detection on the parts of the image with significant local brightness changes. The detection part is the set of points in the image where the pixel gray level is discontinuous or the gray level changes drastically.

12. The weeding method according to claim 9, characterized in that, The navigation steps include: For the path point set output by the path planning, the visual navigation kinematic model is converted into motor control commands through differential drive model, and the motor action is optimized by control algorithm. During the tracking process, if the visual navigation is detected to deviate from the path centerline, the vehicle control unit adjusts the servo angle or the speed of the left and right wheels according to the deviation value to ensure that the visual navigation returns to the correct path.

13. The weeding method according to claim 11, characterized in that, The control steps include: When the visual camera system detects an obstacle but does not identify weeds, the control system determines it to be an obstacle-only scene, records the current device position and the next target point on the original path, and sends a deceleration and stop command to the walking system. It calls the path algorithm of the deep learning computing unit to set the obstacle as a forbidden area, plans the optimal path from the current position to the point after the original path breakpoint, bypassing the obstacle. When the device reaches the obstacle avoidance deviation, it confirms the position, the control system compares the obstacle avoidance deviation with the deviation of the original path, sends a return to the original path command, and the walking system resumes the original direction of travel.

14. The weeding method according to claim 11, characterized in that, The control steps include: When the visual camera system detects weeds and no obstacles are detected, the control system determines that it is a weed-only scene, records the current device position and the breakpoint of the original path, calculates the three-dimensional coordinates of the weeds, sends a deceleration and stop command to the walking system, and starts the preheating of the weeding actuator. The dynamic window method is used to plan the path for weed treatment. The shortest path from the current position to the front of the weed is planned. After the path length is ≤2m, the control system sends a weeding start command to the weeding machinery system. After cutting for a preset time, it resets and records the weed treatment completion signal. After weeding is completed, the control system calls the path planning algorithm to plan a path back to the original path breakpoint. The walking system executes the path command, returns to the breakpoint, and resumes driving along the original path.

15. The weeding method according to claim 11, characterized in that, The control steps include: When the visual camera system simultaneously identifies obstacles and weeds, the control system prioritizes obstacles. It first confirms the location of the obstacle, then confirms the location of the weeds. If the distance between the two is ≤1m, the obstacle is prioritized; if the distance is >1m, the obstacles can be avoided first, and then the weeds can be removed. The process for obstacle-only scenarios is as follows: stop, obstacle avoidance planning and execution, the equipment bypasses the obstacle to the point following the original path breakpoint, and the position of weeds is monitored in real time during the process. After the device reaches the location of the weeds, it stops on its original path, executes the weed processing procedure for the weed-only scenario, and then returns to its original route.

Images