Camera parameter self-calibration row-control weeding method and system based on interactive point selection

Abstract

The invention discloses a camera parameter self-calibration row-control weeding method and system based on interactive point selection, a user only clicks a plurality of feature points on a seedling strip image shot by a camera and inputs row spacing, all installation parameters such as a camera pitch angle and seedling height can be automatically calculated, and accurate mapping from the image to a physical world is established; on the basis, the system recognizes the position of the seedling belt in real time, calculates the transverse physical deviation between the position and the ideal position, and controls a transverse moving mechanism of the weeding machine to carry out precise row alignment; according to the method, rapid, non-inductive and high-precision calibration of camera parameters is realized, the automation degree, row control precision and operation efficiency of weeding operation are improved, and meanwhile, the technical threshold of system use and maintenance is reduced.

Description

Technical Field

[0001] This invention relates to the fields of agricultural machinery automation and intelligent vision technology, specifically to a method and system for row weeding based on interactive point selection and camera parameter self-calibration. Background Technology

[0002] Currently, common mechanical weeding operation modes include Figure 1 As shown, a weeding implement is towed by a tractor. This weeding implement typically includes a beam mechanism and multiple weeding units mounted on it. Each unit is equipped with blades that operate in the furrow or ridge area. Its working principle is to use the blades to turn the soil between the crop rows to remove weeds. This method has a fundamental limitation: the trajectory of the weeding operation depends entirely on the tractor's travel path, requiring it to strictly follow the crop row trajectory formed during sowing.

[0003] However, in actual agricultural production, even when mechanical sowing is used, the sown crop rows are often slightly curved and not ideally straight because it is difficult for the tractor to maintain an absolutely straight path. When weeding, whether relying on manual driving or using a navigation system for automatic driving, the tractor's actual driving trajectory relative to the crop rows will inevitably deviate to a certain extent. This deviation will directly cause the blades on the weeding implements to deviate from the predetermined positions between rows or in the furrows, posing a risk of scratching the crop seedlings, which affects the quality of the operation.

[0004] To address the aforementioned row alignment accuracy issues, vision-assisted row weeding systems have emerged. These systems use cameras to capture real-time images of the seedling strips in front of the machine, and then use visual algorithms to detect the position of the seedling strips relative to the machine. This allows the machine to move laterally, ensuring that the blades are accurately aligned between the rows for precise weeding. However, before deploying such vision systems, it is essential to first obtain the camera's installation parameters, such as the camera's height, pitch angle, and the distance from the camera's optical center to the seedling strip plane. These parameters form the mathematical basis for establishing an accurate mapping between image pixel coordinates and real-world coordinates.

[0005] Currently, obtaining these installation parameters typically relies on tedious manual measurement and data entry. For example, it requires on-site measurement of multiple physical quantities, such as camera installation height, seedling height, and the ground position corresponding to the image center. Taking the measurement of corn seedling height as an example, due to the complex morphology of crop stems and leaves, there is a lack of standardized selection criteria for measurement points, such as choosing the highest leaf tip or stem fork. Different operators often make different judgments. Figure 2As shown, this subjectivity introduces significant human measurement errors. All measurement data needs to be entered one by one during system initialization, which is time-consuming and inefficient. More importantly, once the camera position changes due to maintenance, collision, or replacement, the entire measurement and data entry process must be repeated, causing significant inconvenience to the actual application and maintenance of the system.

[0006] Therefore, in the existing technology, both the traditional traction weeding method and the current vision-assisted row-alignment system have shortcomings in terms of operational accuracy or system usability that urgently need improvement. There is an urgent need in the field for a camera parameter self-calibration technology that can be fast, accurate and without complicated manual measurement. Summary of the Invention

[0007] This invention provides a method and system for row weeding based on interactive point selection and camera parameter self-calibration;

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for row weeding based on interactive point selection and camera parameter self-calibration includes the following steps:

[0010] S1, acquire farmland images containing at least two parallel seedling strips using a camera;

[0011] S2, display the image on the interactive interface, receive the two points selected by the user for each seedling strip on the image, and receive the row spacing of the seedling strip input by the user;

[0012] S3, based on the position of each selected point in the image, determine the seedling strip to which each selected point belongs and its longitudinal position on the seedling strip;

[0013] S4 uses the row spacing of the seedling strip as a geometric constraint in the physical world coordinate system to perform distortion correction on the image pixel coordinates of the selected points.

[0014] Based on the distortion-corrected coordinates and their attribution relationships, the camera's installation parameters are calculated, including at least the camera's pitch angle.

[0015] The geometric constraint is that the ground row spacing calculated from the coordinates of the upper selected points of the two different seedling strips is equal to the ground row spacing calculated from the coordinates of the lower selected points, and both are equal to the input seedling strip row spacing.

[0016] Preferably, step S3 specifically includes:

[0017] Divide the selection points into upper and lower selection points using the central axis of the image as the boundary;

[0018] The selected points at the top and bottom are sorted according to their horizontal coordinate values ​​to establish the correspondence between each selected point and the adjacent seedling strips on the left and right.

[0019] Preferably, in step S4, the installation parameters also include the camera's height above the seedling relative to the plane of the seedling strip, calculated using the following formula:

[0020]

[0021] Where cam_h is the height of the camera above the seedling, z is the distance from the camera center along the optical axis to the ground, and φ is the angle between the camera optical axis and the ground, which is obtained by solving based on the selected point coordinates and geometric constraints.

[0022] Preferably, in step S4, the formula for calculating the camera pitch angle is:

[0023]

[0024] Where pitch is the camera tilt angle.

[0025] Preferably, in step S2, the user inputs multiple selections of the same seedling strip location, the set of image pixel coordinates of the multiple selections is optimized to obtain optimized coordinates, and the calculation in step S4 is performed using the optimized coordinates;

[0026] Optimization processes include:

[0027] Based on the image pixel coordinate set selected from multiple points, abnormal coordinate points are identified and eliminated. The arithmetic mean of the x and y coordinates of the remaining valid coordinate points is calculated, and the average value is used as the optimized coordinate.

[0028] Preferably, it further includes:

[0029] S5. If the camera installation status changes or the preset operation time is reached, repeat steps S1 to S4 to update the installation parameters.

[0030] Preferably, it further includes:

[0031] S6 uses the calculated installation parameters to establish a mapping relationship between the image pixel coordinate system and the physical world coordinate system, and generates and overlays distorted curves on the farmland image.

[0032] Preferably, it further includes a row control step:

[0033] S7 acquires real-time farmland images and identifies the location of seedling strips in real time;

[0034] S8. Based on the mapping relationship established in step S6, the image pixel coordinates of the real-time seedling strip position are converted into the coordinates of the real-time seedling strip position in the physical world coordinate system, and the coordinate difference between the coordinates and the target physical world coordinates corresponding to the distorted curve in the horizontal direction is calculated to obtain the physical horizontal deviation.

[0035] S9 generates a control signal based on the physical lateral deviation, driving the lateral movement mechanism of the weeding machine to correct the row deviation.

[0036] Preferably, in step S7, when identifying the real-time location of the seedling strip, the seedling strip and non-seedling strip areas are simultaneously distinguished;

[0037] In step S9, based on the identification results of non-seedling areas, an auxiliary control signal is generated to drive the working parts on the weeding machine to perform targeted clearing operations on the non-seedling areas.

[0038] Preferably, a camera parameter self-calibration system based on interactive point selection is used to implement the above method.

[0039] The beneficial effects of this invention are as follows:

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention achieves rapid, convenient, and high-precision self-calibration of camera parameters. It eliminates the cumbersome process of relying on manual multi-point physical measurements using tools such as measuring tapes in traditional vision systems. Operators only need to click on a few feature points on the seedling strip image displayed on the device screen and input the known row spacing. All core installation parameters, including camera pitch angle and seedling height, can be automatically calculated within seconds. The initialization time, which used to take more than ten minutes, is reduced to a few seconds. It eliminates human errors caused by subjective selection of measurement points and inaccurate readings, ensuring the objectivity and accuracy of calibration parameters and laying a reliable mathematical foundation for subsequent vision control.

[0042] To improve the row alignment accuracy and reliability of weeding operations, the world coordinate mapping relationship established based on high-precision self-calibration parameters can calculate the physical position deviation of the seedling strip relative to the machine in real time and accurately, and generate precise control commands accordingly. This enables the weeding blades to dynamically and accurately follow the trajectory of the seedling strip, effectively avoiding the problem of seedling scraping caused by traditional fixed attachment methods or navigation path deviations, thus improving the quality of weeding operations and crop safety.

[0043] To enhance the system's adaptability and ease of use, this invention incorporates a parameter update mechanism. When the camera's installation status changes or the accumulated operation time reaches a threshold, it can guide users to quickly recalibrate, ensuring the system's long-term stability. At the same time, by providing a visual verification curve of the calibration results, it empowers users to intuitively judge the calibration quality, improving the user-friendliness of human-computer interaction. These designs enable the system to adapt to complex field operation environments and reduce maintenance difficulty and technical barriers.

[0044] Expanding the functional boundaries of intelligent weeding, this invention further utilizes the same vision system to simultaneously identify seedling strips and non-seedling strip areas, enabling the system to not only control the machine to align laterally but also generate targeted removal commands for weeds within the rows, driving laser, micro-spraying, or mechanical devices for precise strikes. This achieves intelligent integration of navigation and plant protection functions, improves the overall operational efficiency of the equipment, and provides an effective technical means to reduce the use of chemical pesticides.

[0045] In summary, this invention solves the core problems of cumbersome deployment and limited accuracy of vision-based row control systems through an innovative interactive calibration method, and on this basis, constructs a high-precision, adaptive, and functionally integrated intelligent weed control method and system. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of existing towed mechanical weeding operations;

[0047] Figure 2 This is a schematic diagram illustrating the subjective errors inherent in manual measurement of seedling height in existing technologies.

[0048] Figure 3 This is a schematic diagram of the overall structure of the visual-assisted row weeding system according to an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of the view of the farmland ahead, captured by a camera;

[0050] Figure 5 A schematic diagram of an interface for users to interactively select points on an image displayed on a tablet.

[0051] Figure 6 This is a schematic diagram illustrating the sorting and coordinate labeling of user-selected points.

[0052] Figure 7 This is a side view of the camera imaging geometry model on which the present invention is based;

[0053] Figure 8 This is a schematic diagram of a distorted curve generated and overlaid on a real-time image based on calibration parameters for verification purposes.

[0054] Figure 9 This is a flowchart of a method for row weeding based on interactive point selection camera parameter self-calibration provided in Embodiment 1 of the present invention. Detailed Implementation

[0055] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, exemplary embodiments will be described in detail below. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of methods and systems consistent with some aspects of this application as detailed in the appended claims.

[0056] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0057] To better illustrate the purpose, technical solutions, and advantages of this application, the following description, in conjunction with specific embodiments and comparative examples, aims to provide a detailed understanding of the content of this application, rather than limiting it. All other embodiments obtained by those skilled in the art without inventive effort are within the protection scope of this application. Unless otherwise specified, the experimental reagents and instruments involved in the implementation of this application are commonly used reagents and instruments. In this application, the technical features described in an open-ended manner include both closed-ended technical solutions composed of the listed features and open-ended technical solutions that include the listed features.

[0058] The following are preferred embodiments, which provide a detailed description of the specific implementation methods, features, and effects of the present invention.

[0059] Example 1

[0060] This embodiment provides a method for row weeding based on interactive point selection and camera parameter self-calibration, including the following steps:

[0061] S1, acquire images of farmland containing at least two parallel seedbeds using a camera, such as Figure 3 As shown, a camera is installed in front of the weeding machine. During operation, the camera captures real-time images of the farmland scene, including crop seedlings, in front of it, obtaining data such as... Figure 4 The image of farmland shown;

[0062] S2, the image is displayed on the interactive interface and user input is received. The farmland images acquired by the camera are displayed in real time on the interactive interface of a device such as a tablet computer. The user can click on the seedling strips displayed in the image through the interactive interface to select two points for each seedling strip in the field of view, such as... Figure 5As shown, at the same time, users also need to input the known row spacing of the seedling strip on the interactive interface. This row spacing is a fixed parameter that is determined when the farmland is sown and can be easily obtained from agronomic requirements.

[0063] S3. Determine the affiliation of each selected point based on its position in the image. After receiving all selected points generated by the user's click, first use the central axis of the image display area as the boundary to divide all selected points into upper selected points belonging to the upper half of the image and lower selected points belonging to the lower half of the image.

[0064] Subsequently, the two sets of data, the upper selection points and the lower selection points, were sorted from smallest to largest according to the horizontal coordinate values ​​of the selection points in the image pixel coordinate system.

[0065] This differentiation and sorting operation uniquely establishes the correspondence between each selected point and a specific seedling strip, clarifying whether it is located at the top or bottom of the seedling strip, such as... Figure 6 As shown, the upper point P11 and lower point P12 of the first seedling strip, as well as the upper point P21 and lower point P22 of the second seedling strip can be clearly obtained.

[0066] S4. Using the previously input seedling strip row spacing as the known geometric constraints in the physical world coordinate system, calculate the camera installation parameters.

[0067] Call the pre-calibrated camera internal parameters, including distortion correction parameters, to perform distortion correction processing on the original image pixel coordinates of all selected points, and convert them into distortion-free coordinates in the image physical coordinate system;

[0068] Based on the selected point coordinates after distortion correction and the specific seedling strip and longitudinal position information to which they belong, the camera installation parameters are calculated. The core geometric constraint on which this invention is based is that the row spacing between two different seedling strips is fixed and known in the physical world coordinate system.

[0069] Therefore, the ground row spacing calculated from the coordinates of the upper selected points of two different seedling strips in the image must be equal to the ground row spacing calculated from the coordinates of the lower selected points, and both of these must be equal to the seedling strip row spacing previously entered by the user.

[0070] By utilizing this key constraint and combining it with the geometric model of camera imaging, key installation parameters such as the camera's pitch angle can be directly solved without having to manually measure the camera height and seedling height as in traditional methods.

[0071] Specifically, this implementation method allows users to interactively click on a few points on the seedling strip in the image and combine this with easily obtainable row spacing parameters to automatically calibrate key parameters such as camera pitch angle. This method avoids the tedious, subjective, and time-consuming manual measurement steps in traditional vision systems, reducing calibration time from more than ten minutes to a simple click operation, improving calibration efficiency and accuracy, and lowering the technical threshold for system use.

[0072] Furthermore, after receiving the two selected points for each seedling strip from the user through the interactive interface, step S3 is executed to organize the seemingly disordered selected points into an ordered whole, clarifying the affiliation of each point. This is done automatically based on the position information of the selected points in the image, specifically including:

[0073] Using the central axis of the displayed farmland image as the boundary, all selected points generated by user clicks are divided into two sets: points located above the central axis of the image are classified as upper selected points, while points located below the central axis are classified as lower selected points. This is to separate feature points of the same seedling strip at different longitudinal positions in the image.

[0074] Sort the points in the upper and lower selection sets respectively according to their x-coordinate values ​​in the image pixel coordinate system from smallest to largest;

[0075] By sorting by horizontal axis, all selected points can be naturally arranged in order from left to right;

[0076] By combining the above two steps, a precise correspondence between each selected point and its adjacent seedling strips on the left and right is automatically established;

[0077] For example, for the sorted set of upper selection points, the point with the smallest x-coordinate is identified as the point above the leftmost seedling strip, the point with the second smallest x-coordinate is identified as the point above the seedling strip immediately to its right, and so on. The lower selection points are matched according to the same logic.

[0078] like Figure 6 As shown, it can be clearly output that the coordinates of point P11 above the first seedling strip are (u 11 ,v 11 The coordinates of point P12 below are (u 12 ,v 12 The coordinates of point P21 above the second seedling strip are (u 21 ,v 21 The coordinates of point P22 below are (u 22 ,v 22 );

[0079] From a set of disordered pixel coordinates, we can clearly obtain which coordinate points are on each seedling strip, and whether each coordinate point corresponds to the upper or lower half of the seedling strip;

[0080] Specifically, through a two-step process of first distinguishing vertically and then sorting horizontally, the user's random click input is transformed into structured data with clear geometric and logical relationships. This step is the foundation for subsequent calculations, ensuring that the system can correctly understand the user's interaction intent and accurately associate the pixels in the image with specific seedlings in the physical world, providing a reliable data premise for subsequent solution of camera parameters using geometric constraints.

[0081] Furthermore, in step S4, the installation parameters also include the camera's height above the seedling relative to the seedling plane, calculated using the following formula:

[0082]

[0083] Where cam_h is the height of the camera above the seedling, z is the distance from the center of the camera along the optical axis to the ground, and φ is the angle between the optical axis of the camera and the ground, which is solved according to the selected point coordinates and geometric constraints.

[0084] The process of solving for the distance parameter z and the included angle φ is based on the image geometric model and physical world constraints. The core lies in establishing equations using the coordinates of four specific points clicked by the user. Figure 5 The screen shown shows four points obtained by clicking two points on each of the two seedling strips. For clarity, subscripts are used to mark these points. The first subscript number indicates the seedling strip number, and the second subscript number indicates the vertical position of the point on the seedling strip.

[0085] P11 represents the point above the first seedling strip, and its pixel coordinates are (u 11 ,v 11 The corresponding ordinate of the distortion-free image is y. u11 The distortion correction parameter used is γ 11 ;

[0086] P12 represents the point below the first seedling strip, and its pixel coordinates are (u 12 ,v 12 );

[0087] P21 represents the point above the second seedling strip, and its pixel coordinates are (u 21 ,v 21 The corresponding ordinate of the distortion-free image is y. u21 The distortion correction parameter used is γ 21 ;

[0088] P22 represents the point below the second seedling strip, and its pixel coordinates are (u 22 ,v 22 The corresponding ordinate of the distortion-free image is y. u22 ;

[0089] The specific derivation steps are as follows:

[0090] Based on the pinhole camera model and the camera's installation geometry relative to the ground, establish the mapping relationship between the image coordinate system and the ground coordinate system;

[0091] The geometric model of camera imaging on which the calculation is based is as follows: Figure 7 As shown, the dashed line represents the camera's optical axis. Let the distortion-corrected coordinates in the image coordinate system be (x...). u ,y u ), whose corresponding point in the ground coordinate system is (x w ,y w If f is the camera focal length, then the following relationship exists:

[0092]

[0093] By eliminating ground coordinate y w x can be derived u The expression is:

[0094]

[0095] Next, establish the transformation relationship between the image pixel coordinate system and the image coordinate system, let (x i ,y i ) represents the image pixel coordinates, γ is the parameter generated based on pre-calibrated image distortion parameters used for correction, and d x and d y These represent the physical dimensions of each pixel in the camera, both horizontally and vertically, also known as the pixel size. x and I y If the image size is specified, then a transformation relationship exists:

[0096]

[0097]

[0098] Where (x) d ,y d () represents the coordinates in the image coordinate system that include distortion;

[0099] The formula for calculating parameter γ is:

[0100]

[0101] k1 and k2 are pre-calibrated parameters. There will be slight differences in γ at different positions. γ at any position in the image can be calculated in advance based on the image size and used in real time.

[0102] Based on the above transformation relationship, the ground coordinates x can be solved in reverse.w Expressions for image pixel coordinates and mounting parameters:

[0103]

[0104] Based on the agronomical characteristics of sowing operations, the row spacing between adjacent seedbeds is constant in the physical world. Let this actual row spacing be r. For the points clicked by the user above and below the two seedbeds respectively, what are their corresponding ground coordinates x? w The differences should all be equal to the row spacing r, thus establishing the geometric constraint equation:

[0105]

[0106] ground coordinates x w Substituting the expression into the constraint equation, and substituting the pixel x-coordinate difference obtained from the user's click and the corresponding image y-coordinate, we obtain the specific equation. By solving this equation, we can calculate the angle φ between the camera's optical axis and the ground.

[0107]

[0108] Furthermore, the camera pitch angle is obtained:

[0109]

[0110] After obtaining the included angle φ, the distance parameter z can be calculated:

[0111]

[0112] Finally, the precise camera height above the seedling, cam_h, is obtained.

[0113] Further, in step S2, the user inputs multiple point selections for the same seedling strip location, the set of image pixel coordinates of the multiple selected points is optimized to obtain optimized coordinates, and the calculation in step S4 is performed using the optimized coordinates.

[0114] When a user views the camera's live feed or captured images on a tablet, the system prompts the user to click multiple times to select the same relative position on the same seedling strip.

[0115] For example, if a user clicks three or five times consecutively on the area above the first seedling strip, the system records these multiple point selection operations and obtains the pixel coordinates corresponding to each click on the image, forming a set of image pixel coordinates containing multiple coordinate points;

[0116] The system performs optimization processing on the set of pixel coordinates of the image to eliminate the impact of random errors or misoperations that may exist in a single click.

[0117] The optimization process first identifies and removes abnormal coordinate points from the set. Abnormal coordinate points are those coordinate points that are significantly deviated from the area formed by most points. The cause may be accidental touch by the user or local interference in the image. The identification method can be the statistical distance method, for example, calculating the median of the horizontal and vertical coordinates of all points, and judging those points whose distance from the median point exceeds a preset threshold as abnormal points and removing them.

[0118] After outlier removal, the arithmetic mean of the x and y coordinates of the remaining valid coordinates in the set is calculated.

[0119] Suppose the set of valid coordinate points contains n points, with x-coordinates u1, u2, ..., u3. n The vertical coordinates are v1, v2, ..., v n Then the x-coordinate U of the optimized coordinates optimized The ordinate V is equal to the sum of all valid x-coordinates divided by n. optimized The average coordinate obtained by dividing the sum of all valid ordinates by n is the optimal estimate representing the location of the seedling, and is called the optimized coordinate.

[0120] These optimized coordinates were then used to replace the original coordinates for a single click and served as one of the input data for calculating the camera mounting parameters in step S4.

[0121] The coordinates used in the formula calculation in the subsequent step S4, such as the coordinates P11 above the first seedling strip, are the optimized coordinates obtained here, rather than any single click coordinates. The above optimization process is repeated for multiple point selections made by the user for the upper and lower positions of each seedling strip, thereby providing a set of optimized and interference-resistant coordinate data for step S4.

[0122] Specifically, by introducing a multiple selection and optimization mechanism, this method reduces the risk of increased error in the entire calibration process due to accidental deviations from a single click, improves the accuracy and reliability of the camera parameter self-calibration method in actual use, makes the system more tolerant to user operations, and ensures the precision of vision-based row weeding control.

[0123] Furthermore, the method also includes:

[0124] S5. After the camera installation status changes or the preset operation time is reached, steps S1 to S4 are executed again to update the installation parameters.

[0125] When the camera's installation status changes, for example, when the weeding equipment is working in the field, the mechanical structure that fixes the camera may become slightly loose or displaced due to bumps or collisions, causing the camera's pitch angle or height above the seedlings to change compared to the previously calibrated parameters.

[0126] At this point, the original installation parameters can no longer accurately reflect the true geometric relationship between the camera and the seedling belt. If continued use is used, it will introduce control errors. Therefore, it is necessary to prompt the user or automatically trigger the recalibration process.

[0127] Another scenario is directly related to the operation time. A preset operation time threshold is set. This threshold is empirically set based on the slow changes that may occur due to continuous operation, such as the thermal expansion of the mechanical structure and the cumulative vibration displacement. When the cumulative operation time reaches the preset time, the recalibration process is automatically started regardless of whether the installation structure has changed significantly, in order to prevent potential accuracy decay.

[0128] When either of these two scenarios is triggered, the camera will again capture and display the current image containing the seedling strip, guiding the user to perform interactive point selection operations and receiving the row spacing information input by the user. Based on the new point selection coordinates and row spacing, the camera's pitch angle, distance parameters, and camera height above the seedling will be recalculated according to the aforementioned calculation method, and the installation parameters stored in the system will be updated accordingly. If the optimization features are applied, the point selection coordinates will also be optimized during the recalibration process.

[0129] Furthermore, the method also includes:

[0130] S6. After calculating and updating the camera installation parameters according to the methods of steps S1 to S5, establish the mapping relationship between the image pixel coordinate system and the physical world coordinate system, generate and overlay a distorted curve on the farmland image to realize the visualization verification of the calibration results and provide a mapping basis for subsequent control.

[0131] The calculated camera mounting parameters include the camera pitch angle and the camera height above the seedling (cam_h). The mathematical model of this mapping relationship is based on the pinhole camera model and camera mounting geometry. The derivation process involves the transformation relationship between image distortion-free coordinates, pixel coordinates, camera focal length, pixel size, distortion correction parameters, and world coordinates.

[0132] Based on this mapping relationship, the straight line trajectory of the seedling in the physical world is projected in reverse into the image pixel coordinate system;

[0133] Due to the inherent optical distortion of camera lenses, this projection is not a straight line in the image, but a curved line with a distorted shape.

[0134] Perform the aforementioned calculations to perform back projection, and generate a curve corresponding to the expected position of the seedling strip on the real-time acquired farmland image based on the current calibration parameters;

[0135] Using image overlay technology, the calculated, distorted curve is drawn in real time and displayed on the camera screen shown on the user's tablet, such as... Figure 8 As shown, this curve provides users with intuitive visual feedback;

[0136] Users can quickly verify the accuracy of the current camera calibration parameters by observing the degree of agreement between the actual seedling strip and the superimposed curve. If the curve and the seedling strip basically coincide, it means that the calibration is accurate; if there is a significant deviation, the recalibration process can be triggered.

[0137] Furthermore, the establishment of this mapping relationship and the determination of the curve's position on the image directly provide a calculation benchmark for subsequent visual row control. By detecting the pixel deviation between the actual seedling strip image features and this predefined curve in real time, and combining it with the established coordinate mapping relationship, the physical world deviation of the seedling strip relative to the machine can be calculated, driving the machine's lateral movement mechanism to perform precise row weeding.

[0138] Furthermore, the above method also includes row control steps:

[0139] S7 acquires real-time farmland images and identifies the location of seedling strips in real time;

[0140] By continuously acquiring real-time images of farmland ahead using calibrated cameras, and using image processing and recognition algorithms, such as those based on color features, morphological features, or machine learning models, the current position of each seedling strip is automatically identified from the real-time images, and the image pixel coordinates representing its position are output. This identification process is the basis for subsequent calculation of deviations.

[0141] This embodiment employs an image recognition algorithm based on color features and morphological processing to acquire a real-time color farmland image captured by a camera. This image typically contains green crop seedlings and a soil background with distinct color differences.

[0142] The first step is image preprocessing and color space conversion. In order to better distinguish between vegetation and soil, the image is converted from the common RGB color space to the HSV color space. The HSV color space consists of three components: hue, saturation, and lightness. It can more effectively separate color information from light intensity and improve the robustness of identifying green vegetation under different natural lighting conditions.

[0143] The second step is threshold segmentation based on color features. In the HSV space, green vegetation areas will concentrate in a specific range of values ​​in the hue channel. The hue threshold range applicable to the target crop, such as corn seedlings, is determined in advance through experiments.

[0144] Using this threshold range, the image's tone channel is binarized. Pixels with tone values ​​within this range are set to white (foreground, representing possible seedlings), and the remaining pixels are set to black (background, representing soil), thus obtaining an initial binarized segmentation image.

[0145] The third step is to apply morphological image processing to optimize the segmentation results. The initial binary image usually contains noise, such as scattered weeds, shadows, and discontinuities in the foreground region caused by gaps between seedlings and leaves. The system performs the following morphological operations in sequence:

[0146] Morphological opening is performed, which involves erosion followed by dilation. Small rectangular or circular structuring elements are used for erosion to eliminate isolated noise points and small non-target areas in the image. Then, the same structuring elements are used for dilation to restore the size of the over-eroded main area of ​​the seedling.

[0147] Then, morphological closing operations are performed, which involves first expanding and then eroding. Structural elements of slightly larger size are used for expansion to fill the holes and breaks caused by leaf gaps inside the same seedling strip, connecting them into a more complete connected area. Then, erosion is performed to smooth the area outline and restore the approximate size.

[0148] The fourth step is contour extraction and seedling strip position determination. For the binary image after morphological optimization, the contour search algorithm is used to extract the contours of all white connected regions. Based on prior knowledge, such as the known approximate row spacing, the minimum width and length of the seedling strip in the image, the extracted contours are filtered.

[0149] The filtering criteria may include the area of ​​the bounding rectangle of the contour, its width, aspect ratio, and its vertical position in the image. Contours that conform to the preset geometric features are determined to be valid seedling areas.

[0150] The fifth step is to calculate the representative coordinates of the seedling strip. For each connected region that is determined to be a valid seedling strip, calculate its minimum bounding rectangle or take its bottom center point. Since it is closer to the crop roots and its position is more stable, the pixel coordinates of this point are output as the recognition position coordinates of the seedling strip in the image at this moment, providing input for subsequent steps.

[0151] This algorithm initially locks the target through color space conversion and threshold segmentation, eliminates noise and enhances regional connectivity through morphological operations, and finally obtains stable seedling location coordinates through geometric feature screening. The whole process balances recognition accuracy and real-time processing efficiency, can adapt to the complexity of farmland environment, and provides a reliable foundation for subsequent coordinate mapping and deviation calculation.

[0152] S8. Based on the mapping relationship established in step S6, the image pixel coordinates of the real-time seedling strip position are converted into the coordinates of the real-time seedling strip position in the physical world coordinate system, and the coordinate difference between the coordinates and the target physical world coordinates corresponding to the distorted curve in the horizontal direction is calculated to obtain the physical horizontal deviation.

[0153] This step is based on the mapping relationship between the image pixel coordinate system and the physical world coordinate system established in step S6;

[0154] The specific process for calculating the physical lateral deviation is as follows:

[0155] Obtain the pixel coordinates of the real-time seedling strip position identified in step S7 in the image, denoted as (u real ,v real );

[0156] Convert the pixel coordinates to distortion-free image coordinates, and calculate the distortion-inclusive image coordinates x. d1 and y d1 :

[0157]

[0158] Where, d x1 and d y1 These represent the physical dimensions of each pixel in the camera, both horizontally and vertically, also known as the pixel size. x1 and I y1 Image size;

[0159] Next, using the pre-calibrated image distortion parameters to generate the correction parameter γ, the distortion-free image coordinates x are calculated. u1 and y u1 :

[0160]

[0161] Calculate the physical world horizontal coordinate x corresponding to the real-time seedling location. wreal :

[0162]

[0163] Where z is the distance parameter along the camera optical axis obtained from the calibration, φ is the angle between the camera optical axis and the ground, and f is the camera focal length. These parameters are all obtained during the calibration process in step S4.

[0164] The formula for parameter γ is:

[0165]

[0166] k1 and k2 are pre-calibrated parameters. There will be slight differences in γ at different positions. γ at any position in the image can be calculated in advance based on the image size and used in real time.

[0167] The horizontal coordinate x of the target seedling in the physical world coordinate system wtargetx is a known constant, determined by system presets or calibration procedures. For example, the projection line of the camera's optical axis onto the ground can be set as the center line of the target seedling belt. wtarget =0; or a fixed lateral offset can be set according to agronomic requirements;

[0168] Finally, calculate the physical lateral deviation Δx. w That is, the difference between the world horizontal coordinates of the real-time seedling strip and the world horizontal coordinates of the target seedling strip at the same vertical position:

[0169]

[0170] The physical lateral deviation Δx w It directly reflects the actual deviation distance of the blade centerline on the weeding machine from the ideal seedling strip centerline on the horizontal ground. This deviation is used as the input of the control system to generate a row correction control signal.

[0171] S9 generates a control signal based on the physical lateral deviation, which drives the lateral movement mechanism of the weeding machine to correct the row deviation.

[0172] Based on the physical lateral deviation calculated in step S8, a corresponding control signal is generated according to the preset control algorithm. The control algorithm can be proportional control, proportional-integral control or other more advanced control strategies. The generated control signal is sent to the lateral movement mechanism of the weeding implement. This mechanism is usually a hydraulically driven or electric screw-driven lateral movement platform.

[0173] After receiving the control signal, the lateral movement mechanism drives the entire implement, including the weeding blades, to move laterally relative to the tractor traction point, correcting physical lateral deviations and realigning the blades with the rows of seedlings, thus completing the row correction.

[0174] Specifically, through steps S7 to S9, the static camera parameters determined by interactive point selection are combined with real-time machine vision recognition, coordinate mapping transformation, and closed-loop motion control technology to form a complete, automatic, and intelligent vision-assisted row weeding control method, which simplifies the initial calibration and enables continuous and automatic correction operations in complex field environments.

[0175] Furthermore, in step S7, when identifying the real-time location of the seedling strip, an image classification judgment step is added while performing the original identification process to simultaneously distinguish between the seedling strip area and the non-seedling strip area in the image.

[0176] After the system initially extracts the connected regions of green vegetation through color space conversion, threshold segmentation and morphological processing, it does not determine all regions as seedling strips. The system will further analyze the morphological features, texture features or distribution patterns of each connected region in the entire image.

[0177] True corn seedling zones typically exhibit a continuous, strip-like distribution with a specific orientation, and the tops of the plants have a relatively uniform shape; while non-seedling zones may refer to weeds growing between or within the seedling zones, which are more randomly distributed and have morphological characteristics that differ from those of the crops.

[0178] By using preset feature rules or lightweight machine learning classification models, each vegetation area is identified and classified as a seedling zone or a non-seedling zone area. The system not only outputs the center coordinates of the seedling zone area, but also records the image location information of the non-seedling zone area, such as the pixel coordinates of its bounding rectangle.

[0179] In step S9, in addition to generating the main control signal for line correction, auxiliary control signals are also generated in parallel.

[0180] The generation logic of the auxiliary control signal is based on the non-seedling zone area identification result obtained in step S7. According to the pixel position information of the non-seedling zone area in the image, combined with the mapping relationship between the currently calibrated image pixel coordinate system and the physical world coordinate system established in step S6, the specific physical position coordinates of these non-seedling zone targets on the field ground in front of the machinery are calculated.

[0181] By combining these physical coordinates with the real-time position and forward speed of the weeding equipment, the timing of operations can be precisely planned.

[0182] When the machine moves into the target non-seedling area and enters the preset working range, a trigger command is sent to the special working component on the weeding machine.

[0183] These working components can be high-precision inline mechanical weeding shovels, directional micro-spraying devices, or laser weeding modules, etc.

[0184] The auxiliary control signal drives these operating components to start at specific times and in specific spatial locations to perform fixed-point and quantitative weed removal operations, accurately handling weeds in non-seedling areas while avoiding impact on nearby seedling crops;

[0185] Specifically, the scope of visual perception is expanded from simple seedling location to semantic understanding of field scenes, and multi-level control commands are generated accordingly to drive different actuators to work together, improve the overall effect and intelligence level of row weeding, achieve true selective weeding, reduce pesticide use and crop damage, and enable a vision system to serve the two core links of navigation and plant protection management at the same time, thereby improving the integration and operational efficiency of agricultural equipment.

[0186] Another embodiment of the present invention provides a camera parameter self-calibration system for row weeding based on interactive point selection, for implementing the above method.

[0187] Finally, it should be noted that the device models used in this embodiment are only for verification and do not limit the scope of this application. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit the scope of protection of this application. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the substance and scope of the technical solutions of this application.

Claims

1. A method for row weeding based on interactive point selection and camera parameter self-calibration, characterized in that, Includes the following steps: S1, acquire farmland images containing at least two parallel seedling strips using a camera; S2, display the image on the interactive interface, receive the two points selected by the user for each seedling strip on the image, and receive the row spacing of the seedling strip input by the user; S3, based on the position of each selected point in the image, determine the seedling strip to which each selected point belongs and its longitudinal position on the seedling strip; S4. Using the row spacing of the seedling strip as a geometric constraint in the physical world coordinate system, the image pixel coordinates of the selected point are subjected to distortion correction processing. Based on the distortion-corrected coordinates and their attribution relationships, the camera's installation parameters are calculated, including at least the camera's pitch angle. The geometric constraint condition is that the ground row spacing calculated from the coordinates of the upper selected points of the two different seedling strips is equal to the ground row spacing calculated from the coordinates of the lower selected points, and both are equal to the input seedling strip row spacing.

2. The method according to claim 1, characterized in that, Step S3 specifically includes: Using the central axis of the image as a boundary, the selected points are divided into upper and lower selected points; The selected points at the top and bottom are sorted according to their horizontal coordinate values ​​to establish the correspondence between each selected point and the adjacent seedling strips on the left and right.

3. The method according to claim 1, characterized in that, In step S4, the installation parameters also include the camera's height above the seedling relative to the seedling strip plane, calculated using the following formula: Wherein, cam_h is the height of the camera above the seedling, z is the distance from the camera center along the optical axis to the ground, and φ is the angle between the camera optical axis and the ground, which is obtained by solving the selected point coordinates and the geometric constraints.

4. The method according to claim 3, characterized in that, In step S4, the formula for calculating the camera pitch angle is: Where pitch is the camera tilt angle.

5. The method according to claim 1, characterized in that, In step S2, the system receives multiple point selections from the user for the same seedling location, optimizes the set of image pixel coordinates of the multiple selected points to obtain optimized coordinates, and performs the calculation in step S4 using the optimized coordinates. The optimization process includes: Based on the image pixel coordinate set obtained from multiple point selections, abnormal coordinate points are identified and eliminated. The arithmetic mean of the x and y coordinates of the remaining valid coordinate points is calculated, and the average value is used as the optimized coordinate.

6. The method according to claim 1, characterized in that, Also includes: S5. After the camera installation status changes or the preset operation time is reached, steps S1 to S4 are re-executed to update the installation parameters.

7. The method according to claim 1, characterized in that, Also includes: S6. Using the calculated installation parameters, establish a mapping relationship between the image pixel coordinate system and the physical world coordinate system, and generate and overlay a distorted curve on the farmland image.

8. The method according to claim 7, characterized in that, It also includes row control steps: S7 acquires real-time farmland images and identifies the location of seedling strips in real time; S8. Based on the mapping relationship established in step S6, the image pixel coordinates of the real-time seedling strip position are converted into the coordinates of the real-time seedling strip position in the physical world coordinate system, and the coordinate difference between the coordinates and the target physical world coordinates corresponding to the distorted curve in the horizontal direction is calculated to obtain the physical horizontal deviation. S9, a control signal is generated based on the physical lateral deviation to drive the lateral movement mechanism of the weeding machine to correct the row deviation.

9. The method according to claim 8, characterized in that, In step S7, while identifying the real-time location of the seedling strip, the seedling strip and non-seedling strip areas are simultaneously distinguished. In step S9, based on the identification result of the non-seedling zone area, an auxiliary control signal is generated to drive the working parts installed on the weeding machine to perform a fixed-point cleaning operation on the non-seedling zone area.

10. A camera parameter self-calibration system for row weeding based on interactive point selection, characterized in that, Used to implement the method as described in any one of claims 1 to 9.

Images