Laser weeding robot weeding method and system based on three-dimensional perception

Abstract

The invention discloses a weeding method and system of a laser weeding robot based on three-dimensional perception, and aims to solve the problem of weeding in a high-stalk close planting field. The method comprises the steps that S1, field RGB images and depth information are collected through a depth camera in a side view mode, a target weed is recognized through a deep learning model, and the three-dimensional coordinates of the target weed in a robot coordinate system are solved; s2, the intelligent navigation module carries out autonomous advancing and positioning of the robot; s3, the heterogeneous computing platform solves the yaw angle and the pitch angle of the holder according to the weed coordinates, and optimizes a multi-target irradiation sequence; and S4, the microcontroller unit drives the two-degree-of-freedom holder and the laser generator to complete fixed-point clearing. The system integrates a three-dimensional sensing module, an intelligent navigation module, a laser execution module and the like on a crawler-type chassis. The whole-course unmanned, precise and efficient weeding operation is realized.

Description

Technical Field

[0001] This invention relates to the field of agricultural robot technology, specifically to a laser weeding robot weeding method and system based on three-dimensional perception. Background Technology

[0002] In large-scale agricultural production, weed control in the field is a crucial aspect. Currently, it mainly relies on manual weeding and chemical weeding. Manual weeding is inefficient and costly, while chemical weeding easily leads to pesticide residues, environmental pollution, and increased weed resistance. Although intelligent weeding equipment has emerged, existing technologies are mostly designed for low-growing, wide-row crops and are difficult to apply to tall, densely planted crops. The main technical bottlenecks are as follows: the dense canopy of tall crops causes severe shading, and the visual characteristics of crops and weed seedlings are similar, resulting in low accuracy of vision-based recognition algorithms; the complex field environment, with its changing light and irregular crop rows, places extremely high demands on the robot's autonomous navigation and precise positioning capabilities; laser weeding requires millisecond-level precise positioning of weed growth points, and the uncertainties in the aforementioned recognition and navigation directly lead to weeding failures.

[0003] Therefore, existing technologies lack a laser weeding robot weeding method and system that can overcome the interference of complex environmental conditions in densely planted fields and integrate high-precision weed identification, three-dimensional precise positioning and intelligent autonomous navigation. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to propose a weeding method and system based on a three-dimensional perception-based laser weeding robot. Addressing the difficulties in weed identification, low navigation and positioning accuracy, and poor weeding precision in tall, densely planted crop fields, this invention integrates three-dimensional environmental perception, intelligent navigation decision-making, and laser precision execution technology to achieve fully unmanned, high-precision, and non-chemical intelligent weeding, including: 1) Solve the problem of missed or false detection of weeds caused by the canopy shading of tall crops, and improve the robustness of target weed identification in complex environments; 2) Overcome the interference of changes in field lighting and terrain undulations on robot navigation, and ensure the stability of the travel path and the reliability of obstacle avoidance; 3) Through multi-module collaborative control, the laser beam can achieve millisecond-level precise spot removal of weed growth points, avoiding chemical residues and environmental pollution.

[0005] This was achieved through the following technical solutions: First, a laser weeding robot method based on three-dimensional perception is proposed, with the following steps: S1. The three-dimensional perception module uses a depth camera to collect RGB images and depth information of the field environment from the side. The high-performance processing unit runs a deep learning model trained on a dedicated dataset to identify the target weeds and transforms the coordinates of the target weeds to the robot's body coordinate system {B} through a fixed coordinate system transformation matrix. S2. Based on the SLAM algorithm, the robot's real-time localization is calculated, a map is built, and a navigation path is determined from the fusion data of multiple sensors in the intelligent navigation module. The robot's real-time pose information is then sent to the heterogeneous computing and control platform. S3. The high-performance processing unit calculates the target yaw angle θ and pitch angle φ of the two-degree-of-freedom gimbal in the laser execution module, plans the irradiation sequence of the laser generator in the laser execution module, and generates corresponding travel commands based on the coordinates of the target weeds in the {B} system and the robot's real-time pose information. When performing the calculation, if multiple target weeds are identified at the same time, the irradiation sequence is optimized based on the principle of minimizing the total cost of gimbal rotation. S4. The microcontroller unit controls the joint rotation of the two-degree-of-freedom gimbal and drives the laser beam of the laser generator to remove the target weeds at a fixed point based on the movement command.

[0006] Optionally, in step S1, before the deep learning model identifies the target weeds, data augmentation processing is performed on the RGB image and depth information. This data augmentation includes geometric transformation, photometric transformation, noise injection, and simulation of complex weather conditions. Data augmentation enhances the generalization ability and robustness of the deep learning model under different field environments, ensuring accurate weed identification.

[0007] Optionally, the coordinate system transformation matrix fixed in step S1 is: {B} is the robot's body coordinate system; {C} is the camera coordinate system; T C B Given a 4×4 homogeneous transformation matrix, given by the rotation matrix R B C Translation vector t B C Composition; R B C Let t be a 3×3 orthogonal matrix representing the rotation from the robot body coordinate system {B} to the two-DOF gimbal coordinate system {G}; B C Let be a 3×1 vector representing the coordinates of the origin of the two-degree-of-freedom gimbal coordinate system {G} in the robot body coordinate system {B}. This is a zero vector used to maintain matrix dimension matching. It establishes precise coordinate system transformation relationships to ensure accurate coordinate mapping from camera observations to the robot body, providing reliable position input for subsequent control.

[0008] Optionally, in step S3, the pose of the two-degree-of-freedom gimbal relative to the robot's body coordinate system is fixed, and the transformation matrix between the two-degree-of-freedom gimbal and the robot's body coordinate system is: , where {G} is a two-degree-of-freedom gimbal coordinate system.

[0009] Optionally, before the fixed-point removal in step S4, the coordinate system of the target weeds is transformed to the two-degree-of-freedom gimbal coordinate system using the homogeneous coordinate formula of the inverse coordinate transformation in three-dimensional space. The formula for the transformation is: ,in,( ) -1 for The inverse matrix; Xg, Yg, and Zg represent the homogeneous coordinates of the target weed in the robot's body coordinate system; Xg, Yg, and Zg represent the rectangular coordinates of the target weed in the two-degree-of-freedom gimbal coordinate system. Represented as a column vector, T This indicates transpose.

[0010] Optionally, the rectangular coordinates (Xg, Yg, Zg) in the two-degree-of-freedom gimbal coordinate system can be converted into spherical coordinate parameters. The joint angles of the two-degree-of-freedom gimbal are calculated based on the spherical coordinate parameters. The conversion formula from three-dimensional rectangular coordinates to spherical coordinates is as follows: , where r represents the distance from the target point to the rotation center of the two-degree-of-freedom gimbal; Indicates the yaw angle; Indicates pitch angle; atan2 ( Y g ，X g ) represents the arctangent function in the four quadrants.

[0011] Optionally, when multiple target weeds are identified simultaneously in step S3, the formula for optimizing the total cost of the laser irradiation sequence π is: ,in, This represents the total rotation cost corresponding to the sequence π; π = (π1, π2, ..., πn) represents the operation order of the target point; The target point for the k-th operation is represented; Current represents the initial pose of the two-DOF gimbal. Indicates starting from the k-th target point Switch to the (k+1)th target point The term represents the joint rotation cost of a two-degree-of-freedom platform; the summation term represents the cumulative rotation cost between adjacent target points in the sequence. When encountering multiple target weeds, the total rotation cost of the gimbal is minimized by optimizing the irradiation sequence, thereby reducing movement time and energy consumption and improving continuous weeding efficiency.

[0012] Optionally, the formula for the joint rotation cost of a two-degree-of-freedom platform is: ,in, This represents the two-degree-of-freedom gimbal yaw angle corresponding to target points i and j; : Represents the two-degree-of-freedom gimbal pitch angle corresponding to target points i and j; : Represents the weighting coefficient, used to adjust the cost percentage of yaw and pitch joint rotation.

[0013] Secondly, a laser weeding robot system based on three-dimensional perception for performing the above method is proposed, including: a three-dimensional perception module, a laser execution module, an intelligent navigation module, and a heterogeneous computing and control platform, each module or platform being integrated on a tracked mobile chassis.

[0014] Optionally, the three-dimensional perception module, including a depth vision sensor and a deep learning model, is used to identify target weeds and output their corresponding three-dimensional coordinates in the robot coordinate system; the laser execution module, including a two-degree-of-freedom gimbal and a laser generator, is used to receive movement commands and perform precise weeding; the intelligent navigation module is used to calculate the robot's real-time pose information from the fusion data of multiple sensors and to coordinate with other modules; the heterogeneous computing and control platform adopts a hierarchical heterogeneous architecture, including a high-performance processing unit and a microcontroller unit, to coordinate the operation and control of each module.

[0015] The beneficial effects of this invention compared to the prior art are: 1. High recognition accuracy: By combining a side-view depth sensor with a deep learning model trained on a dedicated dataset, the problem of tall crops blocking the view and the visual similarity between weeds and crops is effectively overcome, improving the accuracy of weed recognition by more than 30%.

[0016] 2. Strong system synergy: The intelligent navigation module is deeply integrated with the perception and execution modules, and the robot's speed is dynamically adjusted according to the number of weeds identified, realizing closed-loop optimization of "perception-decision-execution" and improving work efficiency by 50%.

[0017] 3. Precise and pollution-free weed removal: Laser non-contact removal avoids soil compaction and chemical residues. The gimbal's multi-target optimization sequence ensures that the laser beam is precisely aimed at the weed growth point, and the removal error is controlled within the centimeter level.

[0018] 4. Wide environmental adaptability: Data augmentation and multi-sensor fusion navigation enable the robot to adapt to complex field conditions such as changes in lighting and terrain undulations, supporting fully unmanned operation of tall, densely planted crops such as corn and sugarcane.

[0019] This invention provides an efficient and reliable weed control solution for green agriculture through system-level innovation in three-dimensional perception, intelligent decision-making, and precise execution. Attached Figure Description

[0020] Figure 1 A flowchart of a weeding method using a laser weeding robot based on three-dimensional perception; Figure 2 A schematic diagram of the framework of a laser weeding robot system; Figure 3 This is a physical image of a laser weeding robot system. Figure 4 A schematic diagram of a camera and gimbal coordinate system; Figure 5 This is a schematic diagram of a coordinate transformation; Figure 6 A diagram illustrating the process of solving yaw and pitch angles; Figure 7 This refers to a portion of the original images in a dataset. Figure 8 These are images that have undergone data augmentation processing on a portion of the original images. Detailed Implementation

[0021] The following will be based on embodiments of the present invention. Figures 1 to 8 The technical solutions in the embodiments of the present invention will be described in detail below.

[0022] like Figure 1 As shown, a weeding method using a laser weeding robot based on three-dimensional perception is provided; as Figure 2 The image shown is a schematic diagram of the framework of a laser weeding robot system; combined with Figure 1 and Figure 2 This weeding solution integrates modules such as 3D perception, intelligent navigation, and laser execution onto a tracked chassis. It optimizes the data collection and processing methods, plans the weeding path, accurately identifies target weeds, and performs weeding operations in a fully unmanned manner.

[0023] The weeding method specifically includes the following steps: S1. The 3D perception module uses a depth camera (equipped with a depth vision sensor) to acquire RGB images and depth information of the field environment in real time from a side-view perspective. The side-view perspective overcomes the occlusion problem of traditional top-down or close-range direct views, allowing for better capture of field weeds. Then, the high-performance processing unit runs a deep learning model trained on a dedicated dataset (side-view weed image dataset) to identify the target weeds and transforms the coordinates of the target weeds to the robot's body coordinate system {B} using a fixed coordinate transformation matrix.

[0024] In this embodiment, the 3D perception module includes a depth camera (such as a binocular camera, structured light camera, or ToF camera) and a deep learning model. The depth camera uses side-view to acquire RGB images and depth information of the field environment. The RGB images are used to capture features such as color and texture of crops and weeds, while the depth information provides spatial distance data. The dataset acquired through side-view acquisition is divided into a training dataset and a dataset for actual experiments. The acquisition is conducted in real tall-stalk crop (such as corn) fields to ensure that the data matches the actual working environment.

[0025] like Figure 7 As shown, this is a portion of the original images in a dataset; such as Figure 8 The image shown is a portion of the original image after data augmentation processing; combined with... Figure 7 and Figure 8 As shown, to improve model robustness, this scheme also requires data augmentation processing on the original images in the dataset, including: geometric transformation (rotation and flipping to simulate terrain tilt), photometric transformation (adjusting brightness and contrast to simulate lighting changes), noise injection (enhancing sensor fault tolerance), and simulation of complex weather (rain, fog, etc.). The augmented images are then uniformly resolved to 640×640 pixels, and a dedicated dataset is constructed. Deep learning models (such as the YOLO series) are trained on this dedicated dataset, focusing on learning the distinguishing features of crops and weeds from a side-view perspective (such as stem thickness and leaf shape), outputting weed category and two-dimensional pixel coordinates. Combining depth information, the weed coordinates are transformed to the robot's body coordinate system {B} using a coordinate system transformation matrix. The fixed coordinate system transformation matrix is: {B} is the robot's body coordinate system; {C} is the camera coordinate system; T C B Given a 4×4 homogeneous transformation matrix, given by the rotation matrix R B C Translation vector t B C Composition; R B C Let t be a 3×3 orthogonal matrix representing the rotation from the robot body coordinate system {B} to the two-DOF gimbal coordinate system {G}; B C Let be a 3×1 vector representing the coordinates of the origin of the two-degree-of-freedom gimbal coordinate system {G} in the robot body coordinate system {B}. It is a zero vector used to maintain matrix dimension matching.

[0026] Homogeneous coordinates are a coordinate system used in computer graphics, robotics, and computer vision to uniformly represent geometric transformations (translation, rotation, scaling, and projection). Let the homogeneous coordinates of the target weeds in the camera coordinate system {C} be... . Let be the homogeneous coordinates of the target point in the C-camera coordinate system; The three-dimensional rectangular coordinate components of the target point in the C-camera coordinate system; T is the transpose symbol; 1 is the identifier for homogeneous coordinates (a non-zero constant, usually taken as 1), used to incorporate translation transformations into matrix multiplication operations.

[0027] against It is the coordinate in the robot's body coordinate system. It can be calculated using the following matrix multiplication: .

[0028] By using the coordinate system transformation formula in homogeneous coordinates, the position of the target weed in the three-dimensional image acquired by the depth vision sensor can be converted into three-dimensional spatial coordinates relative to the robot. This provides a unified reference framework for the subsequent kinematic calculation of the two-degree-of-freedom gimbal control based on the robot's body coordinate system, ensuring the pointing control accuracy of the laser beam.

[0029] S2. Based on the SLAM algorithm, the robot's real-time localization is calculated, a map is built, and a navigation path is determined from the fusion data of multiple sensors in the intelligent navigation module. This enables autonomous navigation and obstacle avoidance along crop rows, and the robot's real-time pose information is sent to the heterogeneous computing and control platform.

[0030] The intelligent navigation module integrates multiple sensors, including LiDAR, IMU, and odometer, and improves positioning reliability through multi-source sensor fusion. The fusion method adopts the well-known Extended Kalman Filter (EKF) or factor graph optimization: LiDAR provides obstacle distance information, IMU compensates for attitude changes, and odometer records travel data. The three are fused to correct errors of a single sensor (such as odometer slippage).

[0031] Based on fused data, SLAM algorithms (such as Cartographer or LOAM) are run to construct real-time 2D / 3D field maps and achieve centimeter-level positioning. The map identifies crop rows as navigation references, adapting to tall, densely planted environments.

[0032] Path planning is divided into two layers: global and local. Global planning generates the optimal coverage path based on the farmland boundary; local planning adjusts the direction of travel in real time to ensure that the robot travels stably along the center line of the crop rows and provides a side view for 3D perception. During dynamic obstacle avoidance, traversable obstacles (small stones) are directly crossed, while detour paths are planned for insurmountable obstacles (field ridges).

[0033] In this embodiment, the navigation system module can also be linked with the three-dimensional perception module and the laser execution module. That is, the chassis speed is dynamically adjusted according to the number of weeds identified (speed is reduced when there are multiple targets and increased when there are no targets). The aircraft attitude is stabilized by IMU data and the angle of the laser generator in the laser execution module is adjusted to ensure the accuracy of data acquisition and aiming.

[0034] S3. The high-performance processing unit calculates the target yaw angle θ and pitch angle φ of the two-degree-of-freedom gimbal in the laser execution module, plans the irradiation sequence of the laser generator in the laser execution module, and generates corresponding travel commands based on the coordinates of the target weeds in the {B} system and the robot's real-time pose information. When performing the calculation, if multiple target weeds are identified at the same time, the irradiation sequence is optimized based on the principle of minimizing the total cost of gimbal rotation.

[0035] like Figure 4 The diagram shown is a schematic of a camera and gimbal coordinate system; as shown Figure 5 The image shown is a schematic diagram of a coordinate transformation; combined with... Figure 4 and Figure 5 As shown, during coordinate transformation, the coordinates of the target weeds in the {B} system are first transformed to the gimbal coordinate system {G}. The transformation matrix of the gimbal relative to the {B} system is: The laser emission point coincides with the rotation center of the two-degree-of-freedom gimbal. Let the coordinates of the laser emission point (the end of the gimbal) in the two-degree-of-freedom gimbal coordinate system be O. G =[0,0,0,1] T In order to align the laser beam with the target point P in the robot's body coordinate system B =[X b ,Y b Z b ,1] T This point needs to be transformed into a two-degree-of-freedom gimbal coordinate system, that is: , ( ) -1 for The inverse matrix is ​​used to implement the inverse operation of the "G→B" transformation (i.e., B→G). Xg, Yg, and Zg represent the homogeneous coordinates of the target weed in the robot's body coordinate system; Xg, Yg, and Zg represent the rectangular coordinates of the target weed in the two-degree-of-freedom gimbal coordinate system. Represented as a column vector, T This indicates transpose. That is, the homogeneous coordinates of the target point in the G coordinate system after transformation, which are ultimately represented as column vectors. .

[0036] In robot systems, the target point in the robot's body coordinate system is transformed into coordinates in a two-degree-of-freedom gimbal coordinate system, providing a foundation for calculating the joint angles of the two-degree-of-freedom gimbal; the inverse matrix of the homogeneous transformation matrix ( ) -1 Essentially, it's the reverse operation of the transformation from the G coordinate system to the B coordinate system, first canceling out... The translation component in the coordinate system is then canceled out by the rotation component, thus mapping the point from the B coordinate system to the G coordinate system.

[0037] like Figure 6 The diagram shown illustrates the process of solving yaw and pitch angles; combined with... Figure 6 As shown, in a two-degree-of-freedom gimbal coordinate system, the target point The spherical coordinate parameters can be directly used to calculate the joint angles of a two-DOF gimbal. Let the spherical coordinates of the target point be... The joint angles of the two-degree-of-freedom gimbal are calculated based on the spherical coordinate parameters. The conversion formula from three-dimensional rectangular coordinates to spherical coordinates is as follows: , where r represents the distance from the target point to the rotation center of the two-degree-of-freedom gimbal; This represents the yaw angle, i.e., the Y-axis around the two-degree-of-freedom gimbal coordinate system {G}. G The rotation angle of the shaft; Indicates pitch angle; atan2 ( Y g ，X g () represents the arctangent function in the four quadrants, which can be directly determined. The quadrants (range (-π, π]) are used to avoid angular ambiguity; The two methods yield the same results, but the arctan form is better suited to avoid calculation problems when r=0.

[0038] When a single frame image identifies multiple (let the number be n) target weeds, an optimal laser irradiation sequence needs to be planned to minimize the total angular cost of the two-degree-of-freedom gimbal rotation (equivalent to the total operation time).

[0039] Let the set of all target points in the robot's body coordinate system be P = {P1} B P2 B ,…,P n B For each target point P i B By transforming coordinates and solving in spherical coordinates, the corresponding joint angles (θi, ϕi) of the two-degree-of-freedom gimbal can be obtained. Let the joint angles corresponding to the current pose of the two-degree-of-freedom gimbal be (θi, ϕi). current， ϕ current As the starting state for sequence planning, the rotation cost of a two-DOF gimbal from target point i to target point j is defined as the angle difference between the two points in joint space: ,in, : Represents the total value of the two-degree-of-freedom gimbal rotation when switching from target point i to target point j; This represents the two-degree-of-freedom gimbal yaw angle corresponding to target points i and j; : Represents the two-degree-of-freedom gimbal pitch angle corresponding to target points i and j; : Represents the weighting coefficient, used to adjust the cost ratio of yaw and pitch joint rotation (e.g., if a certain joint moves slower, the corresponding weight can be increased).

[0040] By calculating the rotation cost between any two target points, the illumination / operation sequence with the minimum total cost is found (e.g., greedy algorithm, dynamic programming). Target switching with the lower rotation cost is prioritized to reduce gimbal movement time and improve operational efficiency.

[0041] Let the illumination sequence of the target points be π = (π1, π2, ..., π). n ), where πk represents the index of the k-th irradiated target point, then the formula for the total cost of the laser irradiation sequence π is: , : Represents the total rotation cost corresponding to sequence π; Current: Represents the initial pose (current position) of the two-degree-of-freedom gimbal. : Represents the cost of a single rotation from state / target point a to b (corresponding to the previous cost). (Formula); The summation term in the total cost formula represents the cumulative rotation cost between adjacent target points in the sequence.

[0042] In this embodiment, the total cost consists of two parts: the cost from the initial pose to the first target point and the cumulative cost between adjacent target points in the sequence. A smaller total cost indicates less total rotation of the two-degree-of-freedom gimbal, resulting in higher operational efficiency. When the number of targets n is less than or equal to a preset threshold N, such as N=5 or N=7, an exact optimization algorithm is used to solve for the optimal sequence. For example, an exhaustive search method is used to traverse all n! possible permutations and directly select the sequence with the minimum total cost. The first approach is suitable for extremely small-scale scenarios where n ≤ 5; the second approach, dynamic programming, uses "a subset of illuminated targets + current position" as the state and recursively calculates the minimum cost through a state transition equation, reducing the computational complexity of exhaustive search and making it suitable for scenarios where 5 < n ≤ 7. This strategy guarantees a theoretically optimal solution, and the computational cost is controllable in small-scale scenarios.

[0043] When the number of targets n exceeds the threshold N, a greedy algorithm can be used to solve for an approximately optimal sequence to meet the requirements of real-time operation. This sequence is determined by the current pose (θ) of the two-degree-of-freedom gimbal. current， ϕ current Starting from a point, select the unilluminated target point with the minimum rotation cost as the next point and remove it from the sequence. Repeat this process until all target points have been traversed and the unilluminated target point with the minimum rotation cost is empty, resulting in an approximately optimal sequence. .

[0044] The laser actuation module plans the optimal irradiation sequence based on the heterogeneous computing and control platform. or The system sequentially adjusts the gimbal attitude and initiates laser irradiation without repeatedly waiting for instructions, achieving a continuous process of "aiming-irradiation-target switching". Furthermore, the rotation of the two-degree-of-freedom gimbal and the timing of laser irradiation are strictly coordinated. The laser is only activated after the two-degree-of-freedom gimbal is in position, and the two-degree-of-freedom gimbal immediately turns to the next target after irradiation ends, avoiding deviations caused by overlapping actions and balancing operational efficiency and accuracy.

[0045] S4. The microcontroller unit controls the joint rotation of the two-degree-of-freedom gimbal and drives the laser beam of the laser generator to remove the target weeds at a fixed point based on the movement command.

[0046] The microcontroller unit (MCU) drives the gimbal to rotate to the target angle via a servo control interface, ensuring fast response and resistance to bumps. The laser is activated after the gimbal reaches its position, with the irradiation duration set (in milliseconds) according to the type of weeds. Energy input is precisely controlled via a hardware timer to avoid chemical residue. The heterogeneous computing platform adopts a layered architecture: the high-performance processing unit runs complex algorithms (such as perception and navigation), while the MCU performs real-time control. The two are synchronized through a high-speed communication interface to achieve a closed loop of "perception-decision-execution".

[0047] like Figure 3 The image shown is a physical diagram of a laser weeding robot system; combined with Figure 2 and Figure 3 As shown in the figure, this embodiment also proposes a laser weeding robot system based on three-dimensional perception for performing the above-mentioned weeding method, including: a three-dimensional perception module, a laser execution module, an intelligent navigation module, and a heterogeneous computing and control platform, each module or platform being integrated on a tracked mobile chassis. The three-dimensional perception module includes a depth vision sensor and a deep learning model, used to identify target weeds and output their corresponding three-dimensional coordinates in the robot coordinate system; the laser execution module includes a two-degree-of-freedom gimbal and a laser generator, used to receive movement commands and perform precise weeding; the intelligent navigation module is used to calculate the robot's real-time pose information from the fusion data of multiple sensor sources and to coordinate with other modules; the heterogeneous computing and control platform adopts a hierarchical heterogeneous architecture, including a high-performance processing unit and a microcontroller unit, used to coordinate the computation and control of each module. The function and beneficial effects of this system can be referred to the description in the aforementioned weeding method, and will not be repeated here.

[0048] In summary, this invention achieves fully unmanned, high-precision, and non-chemical intelligent weeding by integrating 3D environmental perception, intelligent navigation decision-making, and laser precision execution technology. By combining a side-view depth sensor with a deep learning model trained on a dedicated dataset, it effectively overcomes the challenges of tall crop shading and the visual similarity between weeds and crops, improving weed identification accuracy by over 30% and demonstrating high precision. The intelligent navigation module, 3D perception module, and execution module work in close coordination, dynamically adjusting the robot's speed based on the number of weeds identified, achieving closed-loop optimization of "perception-decision-execution," effectively improving operational efficiency and demonstrating strong system synergy. Laser non-contact removal avoids soil compaction and chemical residues, while the gimbal's multi-target optimization sequence ensures the laser beam is precisely aimed at the weed growth point, controlling the removal error to the centimeter level, resulting in precise and pollution-free weeding. It supports fully unmanned operation of tall, densely planted crops such as corn and sugarcane, exhibiting broad environmental adaptability and significant advancements.

[0049] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A laser weeding robot weeding method based on three-dimensional perception, characterized in that, The steps are as follows: S1. The three-dimensional perception module uses a depth camera to collect RGB images and depth information of the field environment from the side. The high-performance processing unit runs a deep learning model trained on a dedicated dataset to identify the target weeds and transforms the coordinates of the target weeds to the robot's body coordinate system {B} through a fixed coordinate system transformation matrix. S2. Based on the SLAM algorithm, the robot's real-time localization is calculated, a map is built, and a navigation path is determined from the fusion data of multiple sensors in the intelligent navigation module. The robot's real-time pose information is then sent to the heterogeneous computing and control platform. S3. The high-performance processing unit calculates the target yaw angle θ and pitch angle φ of the two-degree-of-freedom gimbal in the laser execution module, plans the irradiation sequence of the laser generator in the laser execution module, and generates corresponding travel commands based on the coordinates of the target weeds in the {B} system and the robot's real-time pose information. When performing the calculation, if multiple target weeds are identified at the same time, the irradiation sequence is optimized based on the principle of minimizing the total cost of gimbal rotation. S4. The microcontroller unit controls the joint rotation of the two-degree-of-freedom gimbal and drives the laser beam of the laser generator to remove the target weeds at a fixed point based on the movement command.

2. The laser weeding robot weeding method based on three-dimensional perception according to claim 1, characterized in that, In step S1, before the deep learning model identifies the target weeds, the RGB image and depth information are first subjected to data augmentation processing, which includes geometric transformation, photometric transformation, noise injection, and simulation of complex weather.

3. The laser weeding robot weeding method based on three-dimensional perception according to claim 1, characterized in that, The coordinate system transformation matrix fixed in step S1 is: {B} is the robot's body coordinate system; {C} is the camera coordinate system; Given a 4×4 homogeneous transformation matrix, derived from the rotation matrix... Translation vector t B C Composition; R B C Let t be a 3×3 orthogonal matrix representing the rotation from the robot body coordinate system {B} to the two-DOF gimbal coordinate system {G}; B C Let be a 3×1 vector representing the coordinates of the origin of the two-degree-of-freedom gimbal coordinate system {G} in the robot body coordinate system {B}. It is a zero vector used to maintain matrix dimension matching.

4. The laser weeding robot weeding method based on three-dimensional perception according to claim 1, characterized in that, In step S3, the pose of the two-degree-of-freedom gimbal relative to the robot's body coordinate system is fixed, and the transformation matrix between the two-degree-of-freedom gimbal and the robot's body coordinate system is: , where {G} is a two-degree-of-freedom gimbal coordinate system.

5. The weeding method of the laser weeding robot based on three-dimensional perception according to claim 1, characterized in that, Before the fixed-point removal in step S4, the coordinate system of the target weeds is transformed to the two-degree-of-freedom gimbal coordinate system using the homogeneous coordinate formula of the inverse coordinate system transformation in three-dimensional space. The formula for the transformation is: ,in,( ) -1 for The inverse matrix; Xg, Yg, and Zg represent the homogeneous coordinates of the target weed in the robot's body coordinate system; Xg, Yg, and Zg represent the rectangular coordinates of the target weed in the two-degree-of-freedom gimbal coordinate system. Represented as a column vector, T This indicates transpose.

6. The laser weeding robot weeding method based on three-dimensional perception according to claim 5, characterized in that, Convert the rectangular coordinates (Xg, Yg, Zg) in the two-degree-of-freedom gimbal coordinate system to spherical coordinate parameters. The joint angles of the two-degree-of-freedom gimbal are calculated based on the spherical coordinate parameters. The conversion formula from three-dimensional rectangular coordinates to spherical coordinates is as follows: , where r represents the distance from the target point to the rotation center of the two-degree-of-freedom gimbal; Indicates the yaw angle; Indicates pitch angle; atan2 ( Y g ，X g ) represents the arctangent function in the four quadrants.

7. The laser weeding robot weeding method based on three-dimensional perception according to claim 1, characterized in that, When multiple target weeds are identified simultaneously in step S3, the formula for optimizing the total cost of the laser irradiation sequence π is: ,in, This represents the total rotation cost corresponding to the sequence π; π = (π1, π2, ..., πn) represents the operation order of the target point; The target point for the k-th operation is represented; Current represents the initial pose of the two-DOF gimbal. Indicates starting from the k-th target point Switch to the (k+1)th target point At that time, the joint rotation cost of the two-degree-of-freedom platform; the summation term represents the cumulative rotation cost between adjacent target points in the sequence.

8. The laser weeding robot weeding method based on three-dimensional perception according to claim 7, characterized in that, The formula for the joint rotation cost of a two-degree-of-freedom platform is: ,in, This represents the two-degree-of-freedom gimbal yaw angle corresponding to target points i and j; : Represents the two-degree-of-freedom gimbal pitch angle corresponding to target points i and j; : Represents the weighting coefficient, used to adjust the cost percentage of yaw and pitch joint rotation.

9. A laser weeding robot system based on three-dimensional perception for performing the method according to any one of claims 1-8, characterized in that, include: The three-dimensional perception module, laser execution module, intelligent navigation module, and heterogeneous computing and control platform are all integrated on the tracked mobile chassis.

10. The system according to claim 9, characterized in that, The 3D perception module, including a depth camera and a deep learning model, is used to identify target weeds and output their corresponding 3D coordinates in the robot's coordinate system; the laser execution module, including a two-degree-of-freedom gimbal and a laser generator, is used to receive movement commands and perform precise weeding; the intelligent navigation module is used to calculate the robot's real-time pose information from the fusion data of multiple sensors and to coordinate with other modules; the heterogeneous computing and control platform adopts a hierarchical heterogeneous architecture, including a high-performance processing unit and a microcontroller unit, to coordinate the computation and control of each module.

Images