Laser weeding machine control method and system based on binocular vision

Abstract

The invention provides a laser weeding machine control method and system based on binocular vision, and relates to the technical field of control, and the method comprises the steps: collecting a farmland image through a binocular camera, and obtaining a three-dimensional space coordinate; segmenting the left view image to extract a crop region and a weed region, and identifying a weed category; calculating three-dimensional position coordinates of the individual weeds and judging whether the individual weeds are located in a safe area; the angle of the laser transmitter is controlled according to the weed position, laser parameters are determined based on the weed category, and precise directional ablation removal of weeds is achieved. According to the invention, intelligent precise weeding without chemical agents is realized, and the farmland management efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of control technology, and in particular to a control method and system for laser weeders based on binocular vision. Background Technology

[0002] With the advancement of agricultural modernization, automated weeding technology has become an important direction for solving the problem of weeds in farmland. Traditional farmland weeding mainly relies on manual weeding or chemical herbicides. Manual weeding is labor-intensive and inefficient, while chemical herbicides, although highly efficient, cause environmental pollution and soil degradation. In recent years, laser weeding technology has received widespread attention as an environmentally friendly and efficient physical weeding method. This technology uses a high-energy laser beam to precisely irradiate the base of the weed stems, achieving weed control without chemical residues, effectively reducing pesticide use, and aligning with the trend of green agriculture development.

[0003] In the development of laser weeding robots, machine vision technology is crucial for weed identification and localization. Currently, monocular vision systems are used in various agricultural robots, but they have limitations in accurately distinguishing weeds from crops and in three-dimensional spatial localization. With the development of computer vision technology, binocular stereo vision systems, capable of simultaneously acquiring two-dimensional image information and three-dimensional spatial information of target objects, provide a new technological means for precision agriculture operations.

[0004] However, existing laser weeding technologies still have some technical shortcomings and deficiencies. First, current technologies struggle to accurately distinguish between different types of weeds and crops during weed identification, especially when weeds and crops are similar in appearance, leading to misjudgments and resulting in the wrong removal of crops or the presence of weeds. Second, traditional laser weeding systems lack the ability to precisely locate the three-dimensional spatial position of individual weeds, making it difficult to determine the relative position of weeds and crops, and to determine whether weeds are in a safe area for removal, increasing the risk of accidentally damaging crops. Summary of the Invention

[0005] The present invention provides a laser weeding machine control method and system based on binocular vision, which can solve the problems in the prior art.

[0006] A first aspect of the present invention provides a method for controlling a laser weeder based on binocular vision, comprising: Left and right view images of a farmland area are acquired using a binocular camera, and a disparity map between the left and right view images is calculated based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area. The left view image is segmented to extract crop and weed regions. The weed regions are then classified and identified according to preset plant morphology parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image. The two-dimensional pixel coordinates of the weed area are correlated and mapped with the disparity values ​​of the corresponding positions in the disparity map. Combined with the intrinsic and extrinsic parameter matrix of the binocular camera, the three-dimensional spatial coordinates of each individual weed are calculated. The distance between the individual weed and the crop root in three-dimensional space is used to determine whether the individual weed is in a safe area that can be cleared. For individual weeds located in a safe, removable area, the spatial relationship between the laser emitter and the individual weed is calculated based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path to the base of the weed stem. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. The laser emitter is then controlled to emit a laser beam towards the base of the stem according to the laser power parameters and irradiation duration parameters, thereby performing directional ablation and removal of the weed.

[0007] Left and right view images of a farmland area are acquired using a binocular camera. A disparity map between the left and right view images is calculated based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area, including: The left and right view images of a farmland area are simultaneously acquired by a binocular camera within a preset time interval, and the distortion correction processing of the left and right view images is performed to obtain the corrected left and right view images; Epipolar correction is performed on the corrected left and right view images to ensure that corresponding feature points in the corrected left and right view images are located on the same horizontal scan line. Feature points are extracted from the corrected left view image, and corresponding points matching the feature points are searched on the corresponding scan line of the corrected right view image. The disparity value is calculated based on the pixel position difference between the feature points and the corresponding points to generate the disparity map. The disparity value of each pixel in the disparity map is calculated with the baseline distance and focal length parameters of the stereo camera to obtain the three-dimensional spatial coordinates of the target object corresponding to that pixel in the stereo camera coordinate system.

[0008] The left view image is segmented to extract crop and weed regions. The weed regions are then classified and identified based on preset plant morphology parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image, including: The left view image is converted from RGB color space to HSV color space, and the hue component, saturation component, and lightness component are extracted respectively. Based on a preset vegetation color feature threshold, the hue component and the saturation component are binarized, and the pixels in the left view image are segmented into vegetation pixels and background pixels to generate an initial vegetation mask. The initial vegetation mask is applied to the left view image, and the image sub-regions corresponding to the areas covered by the vegetation mask are extracted as the vegetation regions to be classified. Connectivity analysis is performed on the vegetation region to be classified, each connected component is labeled, and morphological characteristic parameters of each connected component are calculated. The morphological characteristic parameters include the area, perimeter, aspect ratio, and compactness of the connected component. The morphological feature parameters of each connected component are matched with the preset crop morphological feature parameters for similarity. When the similarity is higher than the preset crop matching threshold, the connected component is marked as a crop region; otherwise, it is marked as a weed region. For the connected components marked as weed areas, their texture feature parameters and edge feature parameters are extracted. The morphological feature parameters, texture feature parameters, and edge feature parameters are combined into a feature vector. The feature vector is input into a preset classification model, and the classification model outputs the weed category identifier corresponding to the weed area based on the feature vector. Calculate the weighted average of the coordinates of all pixels in the weed region, and use the weighted average as the centroid of the weed region. The coordinates of the centroid are the two-dimensional pixel coordinates of the weed region in the left view image.

[0009] The two-dimensional pixel coordinates of the weed area are correlated and mapped with the disparity values ​​at corresponding positions in the disparity map. Combined with the intrinsic and extrinsic parameter matrices of the binocular camera, the three-dimensional spatial coordinates of each individual weed are calculated. The determination of whether a weed is within a safe, removable area is based on its distance from the crop roots in three-dimensional space, including: The weed region is segmented into instances, each weed individual is assigned a unique identifier, and the two-dimensional pixel coordinates of the centroid of the pixel region occupied by each weed individual in the left view image are calculated. The disparity value corresponding to the two-dimensional pixel coordinates of the centroid position is read from the disparity map. When the disparity value is lower than the validity threshold, the median disparity value of the pixels with valid disparity values ​​within the pixel area occupied by the weed individual is used instead. Based on the intrinsic parameter matrix of the stereo camera, the two-dimensional pixel coordinates of the centroid position and the corresponding disparity value are input into the coordinate reconstruction formula to calculate the three-dimensional coordinates of the weed individual in the camera coordinate system; the extrinsic parameter matrix of the stereo camera is then applied to map the three-dimensional coordinates from the camera coordinate system to the farmland world coordinate system. The crop region is subjected to skeleton extraction to obtain the central axis structure of the crop plant. The position of the crop root is determined at the bottom of the central axis structure, and the three-dimensional spatial coordinates of the crop root in the farmland world coordinate system are obtained by parallax map and coordinate transformation. Construct a safety protection sphere with the three-dimensional spatial coordinates of each crop root as the center and a preset safety distance threshold as the radius. Determine whether the three-dimensional spatial coordinates of an individual weed are outside any safety protection sphere. If it is outside all safety protection spheres, then the individual weed is determined to be in a safe area that can be cleared.

[0010] The process involves obtaining the central axis structure of the crop plant, determining the location of the crop roots at the bottom of the central axis structure, and obtaining the three-dimensional spatial coordinates of the crop roots in the farmland world coordinate system through disparity mapping and coordinate transformation. Specifically, this includes: Morphological refinement is performed on the crop region in the left view image to extract the skeletal lines of the crop plant; branch analysis is performed on the skeletal lines to identify the main branches and lateral branches of the crop plant; based on the continuity and vertical extension length of the branches, the main branches that run through the longitudinal direction of the crop plant are selected as the central axis structure. On the central axis structure, the skeleton pixels are traversed from top to bottom along the vertical direction to determine the bottom pixel of the central axis structure as the bottom position of the central axis. Starting from the bottom of the central axis, the search extends vertically downwards in the left view image to detect the boundary line between the crop area and the background area. The intersection of the boundary line and the horizontal coordinate of the bottom of the central axis is determined as the two-dimensional pixel coordinates of the crop root position in the left view image. The disparity value corresponding to the two-dimensional pixel coordinates of the crop root location is indexed in the disparity map; when the disparity value is invalid or below the validity threshold, a neighborhood window is set around the two-dimensional pixel coordinates of the crop root location, and pixels with valid disparity values ​​within the neighborhood window are extracted. The weighted average of these valid disparity values ​​is calculated as the valid disparity value of the crop root location. Based on the intrinsic and extrinsic parameter matrices of the binocular camera, the two-dimensional pixel coordinates of the crop root location and the corresponding effective disparity value are transformed to obtain the three-dimensional spatial coordinates of the crop root in the farmland world coordinate system.

[0011] For individual weeds located within a safe, removable area, the spatial relationship between the laser emitter and the weed is calculated based on its three-dimensional coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path towards the base of the weed's stem. The corresponding laser power and irradiation duration parameters are determined based on the weed category identifier, specifically including: Obtain the installation position coordinates of the laser emitter's emission point in the farmland world coordinate system, and calculate the three-dimensional position vector between the installation position coordinates and the three-dimensional spatial position coordinates of the individual weeds; The three-dimensional position vector is decomposed into horizontal and vertical components. Based on the angle between the projection direction of the horizontal component in the horizontal plane and the preset reference direction, the required horizontal rotation angle of the laser emitter is calculated. Based on the ratio of the vertical component to the magnitude of the three-dimensional position vector, the required pitch angle of the laser emitter relative to the horizontal plane is obtained through inverse trigonometric function calculation. Based on the weed category identifier, the relative height ratio parameter of the stem base of the weed of that category is retrieved from a preset weed morphological feature database. The vertical component of the three-dimensional spatial coordinates of the individual weed is multiplied by the relative height ratio parameter of the stem base to obtain the vertical coordinate correction value of the stem base position. Keeping the horizontal component of the three-dimensional spatial coordinates of the individual weed unchanged, the vertical coordinate correction value replaces the original vertical component to form the three-dimensional coordinates of the stem base position; recalculate the target position vector between the installation position coordinates and the three-dimensional coordinates of the stem base position, and update the horizontal rotation angle and the pitch angle according to the target position vector; Based on the weed category identifier and the magnitude of the target location vector, query the corresponding laser power parameters and irradiation duration parameters.

[0012] A second aspect of the present invention provides a laser weeding machine control system based on binocular vision, comprising: The first unit is used to acquire left and right view images of a farmland area using a binocular camera, and calculate the disparity map between the left and right view images based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area; The second unit is used to perform image segmentation processing on the left view image, extract crop areas and weed areas, and classify and identify the weed areas according to preset plant morphological feature parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image; The third unit is used to correlate and map the two-dimensional pixel coordinates of the weed area with the disparity values ​​of the corresponding positions in the disparity map, and calculate the three-dimensional spatial coordinates of each weed individual by combining the intrinsic and extrinsic parameter matrix of the binocular camera. It also determines whether the weed individual is in a safe area that can be cleared based on the distance between the weed individual and the crop root in three-dimensional space. The fourth unit is used to calculate the spatial relationship between the laser emitter and the individual weed located in a safe area for removal based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path to the base of the stem of the individual weed. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. The laser emitter is then controlled to emit a laser beam to the base of the stem according to the laser power parameters and irradiation duration parameters to perform directional ablation and removal of the individual weed.

[0013] A third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] Fourth aspect of the present invention, A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] The beneficial effects of this application are as follows: By acquiring the precise three-dimensional spatial coordinates of target objects within farmland using a binocular vision system, the technical shortcomings of insufficient spatial positioning accuracy in traditional monocular vision systems are overcome, providing an accurate spatial positioning basis for laser weeding.

[0016] By combining preset plant morphological characteristic parameters to classify and identify weeds, it can accurately distinguish different types of weeds and intelligently adjust laser parameters according to the weed category, thereby improving the targeting and efficiency of weed control.

[0017] By calculating the distance between individual weeds and crop roots in three-dimensional space, a mechanism for determining safe areas for weed removal was established, effectively avoiding accidental damage to crops during laser weeding and improving weeding safety.

[0018] It achieves precise alignment between the laser emitter and the base of the individual weed stems, and adaptively adjusts the laser power and irradiation duration parameters according to the weed type, ensuring weed control while avoiding energy waste and excessive soil scorching. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the laser weeding machine control method based on binocular vision according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0022] Figure 1 This is a flowchart illustrating the laser weeding machine control method based on binocular vision according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: Left and right view images of a farmland area are acquired using a binocular camera, and a disparity map between the left and right view images is calculated based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area. The left view image is segmented to extract crop and weed regions. The weed regions are then classified and identified according to preset plant morphology parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image. The two-dimensional pixel coordinates of the weed area are correlated and mapped with the disparity values ​​of the corresponding positions in the disparity map. Combined with the intrinsic and extrinsic parameter matrix of the binocular camera, the three-dimensional spatial coordinates of each individual weed are calculated. The distance between the individual weed and the crop root in three-dimensional space is used to determine whether the individual weed is in a safe area that can be cleared. For individual weeds located in a safe, removable area, the spatial relationship between the laser emitter and the individual weed is calculated based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path to the base of the weed stem. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. The laser emitter is then controlled to emit a laser beam towards the base of the stem according to the laser power parameters and irradiation duration parameters, thereby performing directional ablation and removal of the weed.

[0023] In one optional implementation, left and right view images of a farmland area are acquired using a binocular camera, and a disparity map between the left and right view images is calculated based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area, including: The left and right view images of a farmland area are simultaneously acquired by a binocular camera within a preset time interval, and the distortion correction processing of the left and right view images is performed to obtain the corrected left and right view images; Epipolar correction is performed on the corrected left and right view images to ensure that corresponding feature points in the corrected left and right view images are located on the same horizontal scan line. Feature points are extracted from the corrected left view image, and corresponding points matching the feature points are searched on the corresponding scan line of the corrected right view image. The disparity value is calculated based on the pixel position difference between the feature points and the corresponding points to generate the disparity map. The disparity value of each pixel in the disparity map is calculated with the baseline distance and focal length parameters of the stereo camera to obtain the three-dimensional spatial coordinates of the target object corresponding to that pixel in the stereo camera coordinate system.

[0024] A binocular camera system is deployed in farmland areas. This system consists of two parallel cameras mounted at a fixed distance, typically 10-20 cm apart (baseline distance) to ensure sufficient parallax information for depth calculation. The cameras are usually mounted at a height of 2-3 meters to cover a sufficient area of ​​farmland. The camera focal length is selected based on the actual monitoring range, generally between 5-8 mm, with a field of view of approximately 60-80 degrees.

[0025] Simultaneously acquire left and right view images of the farmland area within a preset time interval (e.g., every 1 second). Synchronous acquisition is achieved through hardware trigger signals, ensuring that the left and right cameras capture images at the same time, with a time synchronization error controlled within 1 millisecond. The acquired image resolution is typically 1920×1080 pixels or 2592×1944 pixels to guarantee sufficient detail.

[0026] The acquired raw images undergo distortion correction. Camera distortion mainly includes radial and tangential distortion, which is corrected using pre-calibrated camera intrinsic parameter matrices and distortion coefficients. Distortion correction employs a polynomial model; radial distortion is typically represented by a third-order polynomial. The correction process uses a mapping function to reproject each pixel of the distorted image onto the corrected image plane. The corrected image eliminates edge curvature and deformation, conforming to the pinhole camera model.

[0027] Next, epipolar correction is performed to ensure that corresponding points in the corrected left and right images lie on the same horizontal scan line. Epipolar correction is achieved by calculating the fundamental and essential matrices, and then using a rotation matrix to project the two images onto a common plane. This step reduces the computational complexity of subsequent stereo matching, simplifying the two-dimensional search problem into a one-dimensional search problem. After correction, corresponding points of the same object in the left and right images have the same y-coordinate, with only the x-coordinate differing.

[0028] Feature points are extracted from the corrected left-side image. Various methods are employed for feature point extraction, such as SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Feature Transform), or ORB (Oriented Fast Rotation BRIEF) algorithms. For farmland scenes, due to the presence of numerous regions with similar textures, local gradient information can be incorporated to enhance feature point recognition. Typically, 500-1000 stable feature points are extracted from each image, with each feature point containing location coordinates and descriptor information.

[0029] For each feature point in the left image, a matching point is searched on the corresponding horizontal scan line in the right image. The matching search uses a sliding window method, with a window size typically of 9×9 or 11×11 pixels. For each candidate matching point, its similarity to the feature point in the left image is calculated, using metrics such as Normalized Cross Correlation (NCC), Sum of Absolute Differences (SAD), or Sum of Squared Differences (SSD). To improve matching accuracy, a left-right consistency check can also be applied; that is, after finding the best match for the feature point in the left image, a reverse matching process is performed from the right image back to the left image to verify whether it points to the original point.

[0030] The disparity value is calculated based on the horizontal positional difference between matching point pairs. For a point with coordinates (x_l, y) in the left image, its matching point in the right image has coordinates (x_r, y), so the disparity value d = x_l - x_r. After the disparity calculation, a disparity map of the same size as the original image is generated, where each pixel value represents the disparity magnitude at that location. To eliminate possible false matches, median filtering or bilateral filtering can be used to smooth the disparity map.

[0031] Finally, the disparity map is converted into 3D coordinate information. Based on the pinhole camera model, for a point with a disparity value of d, its depth Z can be calculated using the formula Z = f × b / d, where f is the camera focal length (in pixels) and b is the baseline distance of the stereo camera. After obtaining the depth Z, the X and Y coordinates in the world coordinate system can be further calculated: X = (x_l - c_x) × Z / f, Y = (y - c_y) × Z / f, where c_x and c_y are the optical center coordinates of the camera. In this way, the coordinates (X, Y, Z) of the target object within the farmland area in 3D space are obtained.

[0032] In one optional implementation, the left view image is segmented to extract crop and weed regions, and the weed regions are classified and identified according to preset plant morphological feature parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image, including: The left view image is converted from RGB color space to HSV color space, and the hue component, saturation component, and lightness component are extracted respectively. Based on a preset vegetation color feature threshold, the hue component and the saturation component are binarized, and the pixels in the left view image are segmented into vegetation pixels and background pixels to generate an initial vegetation mask. The initial vegetation mask is applied to the left view image, and the image sub-regions corresponding to the areas covered by the vegetation mask are extracted as the vegetation regions to be classified. Connectivity analysis is performed on the vegetation region to be classified, each connected component is labeled, and morphological characteristic parameters of each connected component are calculated. The morphological characteristic parameters include the area, perimeter, aspect ratio, and compactness of the connected component. The morphological feature parameters of each connected component are matched with the preset crop morphological feature parameters for similarity. When the similarity is higher than the preset crop matching threshold, the connected component is marked as a crop region; otherwise, it is marked as a weed region. For the connected components marked as weed areas, their texture feature parameters and edge feature parameters are extracted. The morphological feature parameters, texture feature parameters, and edge feature parameters are combined into a feature vector. The feature vector is input into a preset classification model, and the classification model outputs the weed category identifier corresponding to the weed area based on the feature vector. Calculate the weighted average of the coordinates of all pixels in the weed region, and use the weighted average as the centroid of the weed region. The coordinates of the centroid are the two-dimensional pixel coordinates of the weed region in the left view image.

[0033] Image segmentation is performed on the left-view image by converting it from the RGB color space to the HSV color space. Images in the RGB color space consist of three channels: red, green, and blue. This representation is not intuitive for vegetation segmentation. After converting the image to the HSV color space, the hue, saturation, and value components can be extracted separately. The HSV color space is more suitable for vegetation segmentation because vegetation typically exhibits a concentrated distribution characteristic in the hue component.

[0034] In the HSV color space, the hue and saturation components are binarized based on pre-defined vegetation color feature thresholds. Specifically, the hue threshold range is set to [30°, 90°], which covers the hue values ​​of most green vegetation; simultaneously, the saturation threshold is set to 0.2, meaning pixels with a saturation greater than 0.2 are more likely to belong to vegetation areas. By combining these two thresholds, the pixels in the left-view image can be divided into vegetation pixels and background pixels, thus generating an initial vegetation mask.

[0035] The generated initial vegetation mask is applied to the left view image, and the image sub-regions corresponding to the mask-covered areas are extracted as the vegetation regions to be classified. Specifically, the original image pixels corresponding to the pixel positions with a value of 1 in the initial vegetation mask are extracted to form new image sub-regions. This step allows subsequent processing to be performed only on the vegetation regions, effectively reducing computation and improving classification accuracy.

[0036] Connectivity analysis is performed on the extracted vegetation regions to be classified, and each connected component is labeled. The connectivity analysis uses the eight-neighborhood connectivity criterion, which considers whether eight neighboring pixels of a pixel belong to the same region. Through breadth-first search or equivalent labeling, connected pixels in the binary image are grouped, and each group is assigned a unique identifier, forming multiple connected components.

[0037] For each connected component, morphological feature parameters are calculated, including area, perimeter, aspect ratio, and compactness. Area is obtained by calculating the total number of pixels within the connected component; perimeter is calculated using an edge tracking algorithm to determine the boundary length of the connected component; aspect ratio is calculated by dividing the length of the longest side by the shortest side of the smallest bounding rectangle of the connected component; and compactness is calculated using the formula 4π × area / perimeter. 2 The calculation reflects the regularity of the shape of the connected components.

[0038] The morphological feature parameters of each connected component are matched with preset crop morphological feature parameters based on similarity. These preset crop morphological feature parameters are determined in advance based on the typical morphological characteristics of a specific crop. Taking rice as an example, its morphological feature parameters might be set as follows: area range of [800, 3000] pixels, aspect ratio range of [3.0, 7.0], and compactness range of [0.3, 0.6]. The similarity matching uses a weighted Euclidean distance calculation method. When the calculated similarity is higher than the preset crop matching threshold, the connected component is marked as a crop region; otherwise, it is marked as a weed region.

[0039] For the connected components marked as weeds, texture and edge feature parameters are further extracted to enable more accurate weed category identification. Texture feature parameters are extracted using the gray-level co-occurrence matrix, including statistics such as energy, contrast, homogeneity, and entropy; edge feature parameters are extracted using the Canny edge detection algorithm, including features such as edge density and edge orientation histogram. The morphological, texture, and edge feature parameters are combined into a feature vector to form the feature representation for weed identification.

[0040] The combined feature vectors are input into a pre-defined classification model to identify the category of weeds. The pre-defined classification model can be a Support Vector Machine (SVM), Random Forest, or Deep Neural Network, etc. Taking SVM as an example, a radial basis function is used as the kernel function, and the parameters C and gamma are optimized through cross-validation to construct a classifier that can effectively distinguish different weed categories. The model outputs a weed category label, such as "broadleaf weed," "grass weed," or a specific weed species such as "purslane" or "foxtail grass."

[0041] To determine the precise location of the weed area in the left-view image, a weighted average of the coordinates of all pixels within the weed area is calculated as the centroid of that area. During the calculation, either pixel grayscale values ​​or green component values ​​can be used as weights to make the centroid position more aligned with the center of the weeds. Specifically, the centroid x-coordinate is calculated as follows: the sum of the x-coordinates of all pixels multiplied by their respective weights, divided by the total weights. The centroid y-coordinate is calculated similarly. The final centroid coordinates are the two-dimensional pixel coordinates of the weed area in the left-view image.

[0042] Through the above steps, the crop and weed areas in the left-view image were segmented, the weed types were identified, and the two-dimensional pixel coordinates of the weed areas in the image were determined, providing basic data support for subsequent accurate weed identification and localization.

[0043] In one optional implementation, the two-dimensional pixel coordinates of the weed area are correlated and mapped with the disparity values ​​at corresponding positions in the disparity map. Combined with the intrinsic and extrinsic parameter matrix of the binocular camera, the three-dimensional spatial coordinates of each individual weed are calculated. Determining whether a weed is within a safe, removable area based on its distance from the crop roots in three-dimensional space includes: The weed region is segmented into instances, each weed individual is assigned a unique identifier, and the two-dimensional pixel coordinates of the centroid of the pixel region occupied by each weed individual in the left view image are calculated. The disparity value corresponding to the two-dimensional pixel coordinates of the centroid position is read from the disparity map. When the disparity value is lower than the validity threshold, the median disparity value of the pixels with valid disparity values ​​within the pixel area occupied by the weed individual is used instead. Based on the intrinsic parameter matrix of the stereo camera, the two-dimensional pixel coordinates of the centroid position and the corresponding disparity value are input into the coordinate reconstruction formula to calculate the three-dimensional coordinates of the weed individual in the camera coordinate system; the extrinsic parameter matrix of the stereo camera is then applied to map the three-dimensional coordinates from the camera coordinate system to the farmland world coordinate system. The crop region is subjected to skeleton extraction to obtain the central axis structure of the crop plant. The position of the crop root is determined at the bottom of the central axis structure, and the three-dimensional spatial coordinates of the crop root in the farmland world coordinate system are obtained by parallax map and coordinate transformation. Construct a safety protection sphere with the three-dimensional spatial coordinates of each crop root as the center and a preset safety distance threshold as the radius. Determine whether the three-dimensional spatial coordinates of an individual weed are outside any safety protection sphere. If it is outside all safety protection spheres, then the individual weed is determined to be in a safe area that can be cleared.

[0044] In farmland weed identification and removal systems, to accurately locate safe-to-remove weeds, it is necessary to convert two-dimensional image information into three-dimensional spatial coordinates and assess the spatial relationship between weeds and crops. The technical solution to achieve this goal is described in detail below.

[0045] First, the acquired left-view image from the stereo camera is segmented into instances of the weed region. A deep learning segmentation network (such as Mask R-CNN or DeepLabv3+) is used to segment the weeds in the image at the pixel level, assigning a unique identifier to each individual weed. The output of this process is a mask image of the same size as the original image, where different weed individuals are labeled with different pixel values.

[0046] For each identified weed individual, the centroid position of the pixel region it occupies in the left-view image is calculated. The centroid is calculated as follows: for the i-th weed individual, assuming its pixel set is Pi, its centroid coordinates (cx_i, cy_i) are obtained by calculating the average of the coordinates of all pixels belonging to that weed region. This step outputs the two-dimensional centroid pixel coordinates of each weed individual.

[0047] Find the disparity value d_i corresponding to the centroid location of each weed in the acquired disparity map. The disparity map is generated from the left and right image pairs of a stereo camera using a stereo matching algorithm (such as SGBM or BM algorithm). If the disparity value at a certain weed centroid location is lower than a preset validity threshold (e.g., 5 pixels), it indicates that the disparity calculation at that point is inaccurate. In this case, an alternative strategy is needed: extract the set of all pixels with valid disparity values ​​within the pixel region occupied by the individual weed, calculate the median of these disparity values, and use it as the disparity value of the weed centroid.

[0048] 3D coordinate reconstruction is performed based on the intrinsic parameter matrix K of the stereo camera. Assuming the baseline length of the stereo camera is b and the focal length is f, the 3D coordinates (X_i, Y_i, Z_i) of individual weed i in the camera coordinate system are calculated as follows: Z_i = f xb / d_i, X_i = (cx_i - u0) x Z_i / f, Y_i = (cy_i - v0) x Z_i / f, where (u0, v0) are the coordinates of the principal point of the image, usually close to the center of the image.

[0049] The 3D coordinates of the weeds are transformed from the camera coordinate system to the farmland world coordinate system. This transformation requires the application of the extrinsic parameter matrix [R|T] of the stereo camera, where R is the rotation matrix and T is the translation vector. The transformation formula is: [X_w, Y_w, Z_w,1]^T = [R|T]^-1 x [X_i, Y_i, Z_i, 1]^T. After the transformation, the position coordinates (X_w, Y_w, Z_w) of the individual weeds in the farmland world coordinate system are obtained.

[0050] To determine the location of crops, the morphological skeleton of the identified crop regions is first extracted. Binarization and thinning algorithms (such as the Zhang-Suen thinning algorithm) are used to extract the central axis structure of the crops. In the obtained skeleton structure, the set of points locating the lowest part is taken as the crop root region, and the center point of the root region is calculated as the crop root location coordinates (rx, ry).

[0051] Using the same method as for weed treatment, parallax maps and coordinate transformations are employed to convert the two-dimensional pixel coordinates of the crop roots into three-dimensional spatial coordinates (rX_w, rY_w, rZ_w) in the farmland world coordinate system. This step is performed on every detected individual crop in the farmland.

[0052] Construct crop safety protection zones. Using the three-dimensional spatial coordinates of each crop root as the center, and setting a preset safety distance threshold d_safe (e.g., 15 cm) as the radius, construct a safety protection sphere. The safety protection sphere represents the area where weeding should not be performed to avoid damaging the crop root system.

[0053] Finally, determine whether each weed is located within a safe zone. For the i-th weed, calculate its Euclidean distance to the root of each crop: dist_i,j = sqrt((X_w_i - rX_w_j) 2 + (Y_w_i - rY_w_j) 2 + (Z_w_i - rZ_w_j) 2If there exists any crop j such that dist_i,j <= d_safe, then the weed is considered to be located within the safe protection sphere of a certain crop and should not be removed; otherwise, the weed is determined to be in a safe area that can be removed and can be removed.

[0054] In practical applications, the safety distance threshold can be adjusted according to crop type and growth stage. For example, for crops with well-developed root systems, the safety distance can be set larger; for crops in the seedling stage, the safety distance can be appropriately reduced. In addition, the shape of the safety zone can be further optimized according to topographic features and soil conditions. For example, under sloping conditions, the safety zone can be appropriately expanded downhill to protect the root system that may be affected by soil erosion.

[0055] In one optional implementation, the central axis structure of the crop plant is obtained, the location of the crop roots is determined at the bottom of the central axis structure, and the three-dimensional spatial coordinates of the crop roots in the farmland world coordinate system are obtained through parallax mapping and coordinate transformation. Specifically, this includes: Morphological refinement is performed on the crop region in the left view image to extract the skeletal lines of the crop plant; branch analysis is performed on the skeletal lines to identify the main branches and lateral branches of the crop plant; based on the continuity and vertical extension length of the branches, the main branches that run through the longitudinal direction of the crop plant are selected as the central axis structure. On the central axis structure, the skeleton pixels are traversed from top to bottom along the vertical direction to determine the bottom pixel of the central axis structure as the bottom position of the central axis. Starting from the bottom of the central axis, the search extends vertically downwards in the left view image to detect the boundary line between the crop area and the background area. The intersection of the boundary line and the horizontal coordinate of the bottom of the central axis is determined as the two-dimensional pixel coordinates of the crop root position in the left view image. The disparity value corresponding to the two-dimensional pixel coordinates of the crop root location is indexed in the disparity map; when the disparity value is invalid or below the validity threshold, a neighborhood window is set around the two-dimensional pixel coordinates of the crop root location, and pixels with valid disparity values ​​within the neighborhood window are extracted. The weighted average of these valid disparity values ​​is calculated as the valid disparity value of the crop root location. Based on the intrinsic and extrinsic parameter matrices of the binocular camera, the two-dimensional pixel coordinates of the crop root location and the corresponding effective disparity value are transformed to obtain the three-dimensional spatial coordinates of the crop root in the farmland world coordinate system.

[0056] In the precise operation of agricultural robots, identifying the location of crop roots and obtaining their three-dimensional coordinates is crucial for targeted operations. This embodiment provides a crop root localization method based on binocular vision, which accurately obtains the three-dimensional spatial coordinates of crop roots in the farmland world coordinate system through crop plant morphological analysis and parallax information processing.

[0057] The left-side image of the crop, captured by a stereo camera, is obtained, and the central axis structure of the crop plant is extracted. The crop region in the left-side image is preprocessed, including grayscale conversion, binarization, and denoising, to obtain a clear binary image of the crop region. Then, a morphological thinning algorithm is used to process the binary image, progressively thinning the robust crop region into one-pixel-wide skeleton lines. The Zhang-Suen thinning algorithm is used during the thinning process; this algorithm iteratively removes boundary pixels while maintaining the image's topological structure, ultimately generating a complete skeleton line representation of the crop plant.

[0058] Branching analysis of the skeletal lines is performed to identify plant structure. By detecting bifurcation points in the skeletal lines, they are divided into multiple segments. The length, direction, and connectivity of each segment are calculated. Main branches are typically longer and extend vertically, while lateral branches are relatively shorter and often extend horizontally or obliquely. Based on these differences, main branches are selected using the following criteria: vertical extension length greater than a preset threshold (e.g., one-third of the image height), angle with the vertical direction less than a preset angle threshold (e.g., 20 degrees), and good continuity. The main branches that meet these criteria are connected to form a central axis structure running longitudinally through the plant.

[0059] After obtaining the central axis structure, determine the location of the crop roots. Traverse the skeleton pixels along the central axis vertically from top to bottom, recording the coordinates of each pixel. When the last pixel is reached, mark it as the bottom position of the central axis. This position is usually close to, but not necessarily exactly, the root of the crop, because parts of the plant's roots may be buried in the soil or obscured in the image.

[0060] To more accurately locate the crop roots, the search extends vertically downwards in the left-view image, starting from the bottom of the central axis. During the search, the boundary between the crop area and the background area (usually soil) is detected by calculating the grayscale changes or color feature differences of each row of pixels. When a significant feature change is detected, that location is determined as the boundary line. The intersection of this boundary line and the horizontal coordinate of the bottom of the central axis is determined as the location of the crop roots, and its two-dimensional pixel coordinates (u_root, v_root) in the left-view image are recorded.

[0061] After obtaining the two-dimensional coordinates of the root, its depth information needs to be determined. The disparity value `disp_root` corresponding to the crop root position (u_root, v_root) is indexed in a pre-calculated disparity map. The disparity map is obtained by processing the stereo image pairs using a stereo matching algorithm (such as BM, SGBM, or AD-Census), representing the difference in horizontal position between corresponding points in the left and right views.

[0062] Sometimes, due to occlusion, changes in lighting, or insufficient texture, the disparity value at the root location may be invalid (e.g., zero or negative) or have low reliability (e.g., less than a preset validity threshold). In this case, a neighborhood window (e.g., 5x5 or 7x7 pixels) is set around the root location, and all pixels with valid disparity values ​​within this window are extracted. These valid disparity values ​​are then weighted and averaged, with the weights determined based on the distance from the pixel to the center point or the confidence level of the disparity value, to obtain the valid disparity value disp_valid at the crop root location.

[0063] Finally, based on the calibration parameters of the stereo camera, the position of the crop roots is transformed from the image coordinate system to the farmland world coordinate system. First, the three-dimensional coordinates of the roots in the camera coordinate system are calculated based on the camera intrinsic parameter matrix and disparity values: Camera coordinates X_c = (u_root - cx) xb / disp_valid; Camera coordinates Y_c = (v_root - cy) xb / disp_valid; Camera coordinates Z_c = fxb / disp_valid; Where cx and cy are the pixel coordinates of the camera's optical center, f is the camera's focal length, and b is the baseline distance of the binocular camera.

[0064] Then, using the camera extrinsic parameter matrix (containing the rotation matrix R and translation vector T), the 3D coordinates in the camera coordinate system are transformed to the farmland world coordinate system: World coordinates X_w = R11 x X_c + R12 x Y_c + R13 x Z_c + T1; World coordinates Y_w = R21 x X_c + R22 x Y_c + R23 x Z_c + T2; World coordinates Z_w = R31 x X_c + R32 x Y_c + R33 x Z_c + T3; Through the above steps, the precise three-dimensional spatial coordinates (X_w, Y_w, Z_w) of the crop roots in the farmland world coordinate system are obtained, providing crucial information for subsequent precise operations by agricultural robots, such as fertilization, weeding, or irrigation. In practical applications, the farmland world coordinate system is typically defined as follows: the X-axis is along the crop row direction, the Y-axis is perpendicular to the ground and upwards, the Z-axis is perpendicular to the crop row direction, and the origin can be set at a fixed reference point in the farmland.

[0065] In one optional implementation, for a weed located within a safe, removable area, the spatial relationship between the laser emitter and the weed is calculated based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path towards the base of the weed's stem. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. Specifically, this includes: Obtain the installation position coordinates of the laser emitter's emission point in the farmland world coordinate system, and calculate the three-dimensional position vector between the installation position coordinates and the three-dimensional spatial position coordinates of the individual weeds; The three-dimensional position vector is decomposed into horizontal and vertical components. Based on the angle between the projection direction of the horizontal component in the horizontal plane and the preset reference direction, the required horizontal rotation angle of the laser emitter is calculated. Based on the ratio of the vertical component to the magnitude of the three-dimensional position vector, the required pitch angle of the laser emitter relative to the horizontal plane is obtained through inverse trigonometric function calculation. Based on the weed category identifier, the relative height ratio parameter of the stem base of the weed of that category is retrieved from a preset weed morphological feature database. The vertical component of the three-dimensional spatial coordinates of the individual weed is multiplied by the relative height ratio parameter of the stem base to obtain the vertical coordinate correction value of the stem base position. Keeping the horizontal component of the three-dimensional spatial coordinates of the individual weed unchanged, the vertical coordinate correction value replaces the original vertical component to form the three-dimensional coordinates of the stem base position; recalculate the target position vector between the installation position coordinates and the three-dimensional coordinates of the stem base position, and update the horizontal rotation angle and the pitch angle according to the target position vector; Based on the weed category identifier and the magnitude of the target location vector, query the corresponding laser power parameters and irradiation duration parameters.

[0066] Obtain the installation coordinates of the laser emitter in the farmland world coordinate system, assuming the laser emitter is located at coordinates (X_l, Y_l, Z_l). Simultaneously, obtain the three-dimensional spatial coordinates (X_w, Y_w, Z_w) of the target weed individual and its corresponding weed category identifier using a visual recognition system. Based on these two sets of coordinates, calculate the three-dimensional position vector V = (X_w - X_l, Y_w - Y_l, Z_w - Z_l) between the laser emitter and the weed individual.

[0067] The three-dimensional position vector V is decomposed into a horizontal component V_h = (X_w - X_l, Y_w - Y_l, 0) and a vertical component V_v = (0, 0, Z_w - Z_l). In the horizontal plane, the angle α between the horizontal component V_h and a preset reference direction (e.g., north in farmland) is calculated. This angle can be obtained by combining the vector dot product and cross product; the result is the horizontal rotation angle that the laser emitter needs to adjust. For example, if V_h is 30 degrees east of north, the laser emitter needs to rotate 30 degrees east.

[0068] The pitch angle β is calculated using the ratio of the length of the vertical component V_v to the magnitude of the three-dimensional position vector V. Specifically, the ratio |V_v| / |V| is calculated, and then the pitch angle β = arcsin(|V_v| / |V|) is obtained through the arcsine function. If Z_w is greater than Z_l, the pitch angle is positive (upward); otherwise, it is negative (downward).

[0069] To achieve precise targeting of the base of weed stems, target location correction is necessary. Based on the weed category identifier, the relative height ratio parameter h_ratio of the stem base for that weed category is retrieved from a pre-defined weed morphological feature database. This parameter represents the relative proportion of the stem base position within the overall plant height, and the h_ratio value varies for different weed types. For example, for broadleaf weeds, h_ratio might be 0.05, indicating that the stem base is located at the bottom 5% of the plant; while for some grass weeds, h_ratio might be 0.10.

[0070] Calculate the vertical coordinate correction value Z_base for the base of the weed stem. Assuming the weed grows on the ground, with the ground's vertical coordinate as Z_ground, then Z_base = Z_ground + (Z_w - Z_ground) × h_ratio. In practical applications, this can be simplified to Z_base = Z_ground + H × h_ratio, where H is the weed height.

[0071] Keeping the horizontal coordinates of the individual weeds unchanged, the vertical coordinates are corrected to Z_base, forming the three-dimensional coordinates (X_w, Y_w, Z_base) of the stem base. Then, the target position vector V_target = (X_w - X_l, Y_w - Y_l, Z_base - Z_l) between the laser emitter and the stem base is recalculated. Based on V_target, the horizontal rotation angle and pitch angle are updated according to the aforementioned method to ensure that the laser beam is accurately pointed at the base of the weed stem.

[0072] Based on the weed category identifier and the modulus |V_target| (i.e., laser propagation distance) of the target location vector V_target, a preset parameter table is consulted to determine appropriate laser power and irradiation duration parameters. These parameters take into account the sensitivity of different weed species to heat energy and the characteristic of laser energy attenuation with distance. For example, for weeds at greater distances, it may be necessary to increase laser power or extend irradiation time; for more resistant weed species, corresponding parameters also need to be adjusted to ensure effective eradication.

[0073] In practical applications, to improve system efficiency, a database can be established to correlate weed types, laser propagation distances, and optimal power and duration parameters. When weeds are detected, the system automatically queries this database without real-time calculations, thus accelerating the response time.

[0074] Once the laser parameters are determined, the control system drives the servo motor to adjust the horizontal rotation and pitch angles of the laser emitter, directing the laser beam towards the base of the target weed stem, and initiates laser irradiation according to the determined power and duration parameters. After irradiation is complete, the system can immediately switch to the next target weed, achieving continuous and efficient weed control.

[0075] In addition, during laser irradiation, the system can monitor temperature changes in the target area in real time and adjust laser parameters in a timely manner to avoid excessive irradiation that could damage the soil or surrounding crops, thus ensuring precise and safe weed control.

[0076] The present invention provides a binocular vision-based laser weeding machine control system, comprising: The first unit is used to acquire left and right view images of a farmland area using a binocular camera, and calculate the disparity map between the left and right view images based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area; The second unit is used to perform image segmentation processing on the left view image, extract crop areas and weed areas, and classify and identify the weed areas according to preset plant morphological feature parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image; The third unit is used to correlate and map the two-dimensional pixel coordinates of the weed area with the disparity values ​​of the corresponding positions in the disparity map, and calculate the three-dimensional spatial coordinates of each weed individual by combining the intrinsic and extrinsic parameter matrix of the binocular camera. It also determines whether the weed individual is in a safe area that can be cleared based on the distance between the weed individual and the crop root in three-dimensional space. The fourth unit is used to calculate the spatial relationship between the laser emitter and the individual weed located in a safe area for removal based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path to the base of the stem of the individual weed. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. The laser emitter is then controlled to emit a laser beam to the base of the stem according to the laser power parameters and irradiation duration parameters to perform directional ablation and removal of the individual weed.

[0077] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0078] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0079] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A laser weeding machine control method based on binocular vision, characterized in that, include: Left and right view images of a farmland area are acquired using a binocular camera, and a disparity map between the left and right view images is calculated based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area. The left view image is segmented to extract crop and weed regions. The weed regions are then classified and identified according to preset plant morphology parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image. The two-dimensional pixel coordinates of the weed area are correlated and mapped with the disparity values ​​of the corresponding positions in the disparity map. Combined with the intrinsic and extrinsic parameter matrix of the binocular camera, the three-dimensional spatial coordinates of each individual weed are calculated. The distance between the individual weed and the crop root in three-dimensional space is used to determine whether the individual weed is in a safe area that can be cleared. For individual weeds located in a safe, removable area, the spatial relationship between the laser emitter and the individual weed is calculated based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path to the base of the weed stem. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. The laser emitter is then controlled to emit a laser beam towards the base of the stem according to the laser power parameters and irradiation duration parameters, thereby performing directional ablation and removal of the weed.

2. The method according to claim 1, characterized in that, Left and right view images of a farmland area are acquired using a binocular camera. A disparity map between the left and right view images is calculated based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area, including: The left and right view images of a farmland area are simultaneously acquired by a binocular camera within a preset time interval, and the distortion correction processing of the left and right view images is performed to obtain the corrected left and right view images; Epipolar correction is performed on the corrected left and right view images to ensure that corresponding feature points in the corrected left and right view images are located on the same horizontal scan line. Feature points are extracted from the corrected left view image, and corresponding points matching the feature points are searched on the corresponding scan line of the corrected right view image. The disparity value is calculated based on the pixel position difference between the feature points and the corresponding points to generate the disparity map. The disparity value of each pixel in the disparity map is calculated with the baseline distance and focal length parameters of the stereo camera to obtain the three-dimensional spatial coordinates of the target object corresponding to that pixel in the stereo camera coordinate system.

3. The method according to claim 1, characterized in that, The left view image is segmented to extract crop and weed regions. The weed regions are then classified and identified based on preset plant morphology parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image, including: The left view image is converted from RGB color space to HSV color space, and the hue component, saturation component, and lightness component are extracted respectively. Based on a preset vegetation color feature threshold, the hue component and the saturation component are binarized, and the pixels in the left view image are segmented into vegetation pixels and background pixels to generate an initial vegetation mask. The initial vegetation mask is applied to the left view image, and the image sub-regions corresponding to the areas covered by the vegetation mask are extracted as the vegetation regions to be classified. Connectivity analysis is performed on the vegetation region to be classified, each connected component is labeled, and morphological characteristic parameters of each connected component are calculated. The morphological characteristic parameters include the area, perimeter, aspect ratio, and compactness of the connected component. The morphological feature parameters of each connected component are matched with the preset crop morphological feature parameters for similarity. When the similarity is higher than the preset crop matching threshold, the connected component is marked as a crop region; otherwise, it is marked as a weed region. For the connected components marked as weed areas, their texture feature parameters and edge feature parameters are extracted. The morphological feature parameters, texture feature parameters, and edge feature parameters are combined into a feature vector. The feature vector is input into a preset classification model, and the classification model outputs the weed category identifier corresponding to the weed area based on the feature vector. Calculate the weighted average of the coordinates of all pixels in the weed region, and use the weighted average as the centroid of the weed region. The coordinates of the centroid are the two-dimensional pixel coordinates of the weed region in the left view image.

4. The method according to claim 1, characterized in that, The two-dimensional pixel coordinates of the weed area are correlated and mapped with the disparity values ​​at corresponding positions in the disparity map. Combined with the intrinsic and extrinsic parameter matrices of the binocular camera, the three-dimensional spatial coordinates of each individual weed are calculated. The determination of whether a weed is within a safe, removable area is based on its distance from the crop roots in three-dimensional space, including: The weed region is segmented into instances, each weed individual is assigned a unique identifier, and the two-dimensional pixel coordinates of the centroid of the pixel region occupied by each weed individual in the left view image are calculated. The disparity value corresponding to the two-dimensional pixel coordinates of the centroid position is read from the disparity map. When the disparity value is lower than the validity threshold, the median disparity value of the pixels with valid disparity values ​​within the pixel area occupied by the weed individual is used instead. Based on the intrinsic parameter matrix of the stereo camera, the two-dimensional pixel coordinates of the centroid position and the corresponding disparity value are input into the coordinate reconstruction formula to calculate the three-dimensional coordinates of the weed individual in the camera coordinate system; the extrinsic parameter matrix of the stereo camera is then applied to map the three-dimensional coordinates from the camera coordinate system to the farmland world coordinate system. The crop region is subjected to skeleton extraction to obtain the central axis structure of the crop plant. The position of the crop root is determined at the bottom of the central axis structure, and the three-dimensional spatial coordinates of the crop root in the farmland world coordinate system are obtained by parallax map and coordinate transformation. Construct a safety protection sphere with the three-dimensional spatial coordinates of each crop root as the center and a preset safety distance threshold as the radius. Determine whether the three-dimensional spatial coordinates of an individual weed are outside any safety protection sphere. If it is outside all safety protection spheres, then the individual weed is determined to be in a safe area that can be cleared.

5. The method according to claim 4, characterized in that, The process involves obtaining the central axis structure of the crop plant, determining the location of the crop roots at the bottom of the central axis structure, and obtaining the three-dimensional spatial coordinates of the crop roots in the farmland world coordinate system through disparity mapping and coordinate transformation. Specifically, this includes: Morphological refinement is performed on the crop region in the left view image to extract the skeletal lines of the crop plant; branch analysis is performed on the skeletal lines to identify the main branches and lateral branches of the crop plant; based on the continuity and vertical extension length of the branches, the main branches that run through the longitudinal direction of the crop plant are selected as the central axis structure. On the central axis structure, the skeleton pixels are traversed from top to bottom along the vertical direction to determine the bottom pixel of the central axis structure as the bottom position of the central axis. Starting from the bottom of the central axis, the search extends vertically downwards in the left view image to detect the boundary line between the crop area and the background area. The intersection of the boundary line and the horizontal coordinate of the bottom of the central axis is determined as the two-dimensional pixel coordinates of the crop root position in the left view image. The disparity value corresponding to the two-dimensional pixel coordinates of the crop root location is indexed in the disparity map; when the disparity value is invalid or below the validity threshold, a neighborhood window is set around the two-dimensional pixel coordinates of the crop root location, and pixels with valid disparity values ​​within the neighborhood window are extracted. The weighted average of these valid disparity values ​​is calculated as the valid disparity value of the crop root location. Based on the intrinsic and extrinsic parameter matrices of the binocular camera, the two-dimensional pixel coordinates of the crop root location and the corresponding effective disparity value are transformed to obtain the three-dimensional spatial coordinates of the crop root in the farmland world coordinate system.

6. The method according to claim 1, characterized in that, For individual weeds located within a safe, removable area, the spatial relationship between the laser emitter and the weed is calculated based on its three-dimensional coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path towards the base of the weed's stem. The corresponding laser power and irradiation duration parameters are determined based on the weed category identifier, specifically including: Obtain the installation position coordinates of the laser emitter's emission point in the farmland world coordinate system, and calculate the three-dimensional position vector between the installation position coordinates and the three-dimensional spatial position coordinates of the individual weeds; The three-dimensional position vector is decomposed into horizontal and vertical components. Based on the angle between the projection direction of the horizontal component in the horizontal plane and the preset reference direction, the required horizontal rotation angle of the laser emitter is calculated. Based on the ratio of the vertical component to the magnitude of the three-dimensional position vector, the required pitch angle of the laser emitter relative to the horizontal plane is obtained through inverse trigonometric function calculation. Based on the weed category identifier, the relative height ratio parameter of the stem base of the weed of that category is retrieved from a preset weed morphological feature database. The vertical component of the three-dimensional spatial coordinates of the individual weed is multiplied by the relative height ratio parameter of the stem base to obtain the vertical coordinate correction value of the stem base position. Keeping the horizontal component of the three-dimensional spatial coordinates of the individual weed unchanged, the vertical coordinate correction value replaces the original vertical component to form the three-dimensional coordinates of the stem base position; recalculate the target position vector between the installation position coordinates and the three-dimensional coordinates of the stem base position, and update the horizontal rotation angle and the pitch angle according to the target position vector; Based on the weed category identifier and the magnitude of the target location vector, query the corresponding laser power parameters and irradiation duration parameters.

7. A binocular vision-based laser weeding machine control system, used to implement the method as described in any one of claims 1-6, characterized in that, include: The first unit is used to acquire left and right view images of a farmland area using a binocular camera, and calculate the disparity map between the left and right view images based on a stereo matching algorithm to obtain the three-dimensional spatial coordinates of the target object within the farmland area; The second unit is used to perform image segmentation processing on the left view image, extract crop areas and weed areas, and classify and identify the weed areas according to preset plant morphological feature parameters to obtain weed category identifiers and their two-dimensional pixel coordinates in the left view image; The third unit is used to correlate and map the two-dimensional pixel coordinates of the weed area with the disparity values ​​of the corresponding positions in the disparity map, and calculate the three-dimensional spatial coordinates of each weed individual by combining the intrinsic and extrinsic parameter matrix of the binocular camera. It also determines whether the weed individual is in a safe area that can be cleared based on the distance between the weed individual and the crop root in three-dimensional space. The fourth unit is used to calculate the spatial relationship between the laser emitter and the individual weed located in a safe area for removal based on its three-dimensional spatial coordinates. By controlling the pitch and horizontal rotation angles of the laser emitter, the laser beam is directed along a preset optical path to the base of the stem of the individual weed. The corresponding laser power parameters and irradiation duration parameters are determined according to the weed category identifier. The laser emitter is then controlled to emit a laser beam to the base of the stem according to the laser power parameters and irradiation duration parameters to perform directional ablation and removal of the individual weed.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images