Visual trajectory tracking control method and system for rice weeding robot

Abstract

The invention provides a visual trajectory tracking control method and system for a rice weeding robot, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the real-time shooting of a rice field, and processing a shot picture, and obtaining candidate line information; screening the candidate lines according to the candidate line information, and taking a straight line closest to the center line of the image as a guide line to obtain the guide line; constructing a control model according to the position relationship between the guide line and the image center line, and calculating the control model to obtain the steering control quantity of the current frame; according to the steering control quantity, the steering control quantity of the two continuous frames of images is processed, and the steering angle of the rice weeding robot is obtained. By providing an accurate navigation path, accidental injury of a non-target area is reduced, the environment adaptability and stability are enhanced, the operation efficiency and the response speed are improved, and visual trajectory tracking control of the rice weeding robot is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly to a visual trajectory tracking control method and system for a rice weeding robot. Background Art

[0002] In the traditional rice planting mode, the weeding operation has always been an indispensable but extremely cumbersome link. For a long time, farmers mainly rely on manual weeding, which is not only extremely inefficient but also brings huge labor intensity to farmers. In recent years, the visual-based trajectory tracking technology has gradually become a research hotspot in the field of robot navigation. This technology uses a camera mounted on the robot to capture real-time images of the surrounding environment, and uses image processing algorithms to analyze and process the captured images, so as to identify and track a predetermined trajectory. The emergence of this technology provides strong support for the robot to achieve autonomous navigation.

[0003] However, in practical applications, the environmental characteristics of paddy fields pose quite a challenge to the visual-based trajectory tracking technology. In paddy fields, the rows of seedlings are usually irregularly arranged, and as the growth cycle changes, the morphology and color of the seedlings will also change accordingly. In addition, there may be complex factors such as weeds, mud, and water stains in the field, which will interfere with the robot's visual recognition. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a visual trajectory tracking control method and system for a rice weeding robot, which realizes the visual trajectory tracking control of the rice weeding robot by providing an accurate navigation path, reducing the accidental injury of non-target areas, enhancing the environmental adaptability and stability, and improving the operation efficiency and response speed.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In the first aspect, a visual trajectory tracking control method for a rice weeding robot, the method includes:

[0007] Taking a real-time photo of the paddy field and processing the taken picture to obtain candidate line information;

[0008] According to the candidate line information, screening the candidate lines, and taking the straight line closest to the center line of the image as the guiding line to obtain the guiding line;

[0009] Constructing a control model according to the position relationship between the guiding line and the center line of the image, and calculating the control model to obtain the steering control amount of the current frame;

[0010] According to the steering control amount, processing the steering control amounts of two consecutive frames of images to obtain the steering angle of the rice weeding robot;

[0011] Based on the steering angle, calculate the torque output by the steering motor of the rice weeding robot to achieve the tracking control of the robot's visual trajectory.

[0012] Furthermore, conduct real-time shooting of the paddy field, process the captured pictures, and obtain candidate line information, including:

[0013] Fix the RGB camera at the central position in front of the robot, conduct real-time shooting of the paddy field, obtain the paddy field image, and preprocess the original image captured by the camera to obtain the preprocessed image;

[0014] Detect the preprocessed image to obtain the coordinate points of the rice seedlings in the paddy field;

[0015] Perform coordinate system conversion on the seedling coordinate points, obtain the corresponding positions of the seedling rows from the bird's-eye view, and obtain the seedling coordinate points after coordinate system conversion;

[0016] Classify the seedling coordinate points after coordinate system conversion, and divide them into different clusters according to the crop density around the seedling coordinate points to obtain the coordinate points of each cluster of seedlings;

[0017] Fit the coordinate points of each cluster of seedlings to obtain candidate line information.

[0018] Furthermore, according to the candidate line information, screen the candidate lines, and take the straight line closest to the center line of the image as the guiding line to obtain the guiding line, including:

[0019] According to the candidate line information, obtain the candidate line parameters, and calculate the average value of the x-axis coordinates of the starting point and the ending point of each straight line to obtain the x-axis coordinate of the midpoint coordinate of the candidate line;

[0020] According to the x-axis coordinate of the midpoint coordinate of the candidate line, through Calculate the distance between the average value and the x-axis coordinate of the center point of the image, where x max is the coordinate of the starting point of each candidate line on the x-axis, x min is the coordinate of the ending point of each candidate line on the x-axis, W is the width of the image, and b1 is the distance between the average value and the x-axis coordinate of the center point of the image;

[0021] Take the straight line closest to the center line of the image among the average values of each candidate line as the guiding line to obtain the guiding line.

[0022] Furthermore, construct a control model according to the positional relationship between the guiding line and the center line of the image, and calculate the control model to obtain the steering control amount of the current frame, including:

[0023] According to the position where the RGB camera is fixed on the robot, obtain the position of the center line of the image;

[0024] Calculate the direction angle deviation and the lateral distance deviation between the guiding line and the image center line according to the angle between the guiding line and the negative direction of the image coordinate axis;

[0025] Through Calculate the lateral distance deviation between the guiding line and the image center line. Wherein, b is the lateral distance deviation, and f x Is the focal length of the camera, and u c And v c Are the u and v coordinates of the image center line in the image coordinate system respectively, u0 and v0 are the u and v coordinates of the intersection point of the guiding line and the lower edge of the image, D is the vertical distance from the camera to the seedling row, Z is the Z coordinate of the seedling row plane in the camera coordinate system, Δu is the basic offset of the u coordinate, and u m Is the change rate of the u coordinate of the image center line, k1 and k2 are distortion coefficients, and k3 is the dynamic correction coefficient.

[0026] According to the lateral distance deviation, through Calculate the lateral error compensation and construct a control model. Wherein, β1 is the lateral error compensation angle, and its value range is -90° < β1 < 90°; b is the lateral distance deviation, and c is the compensation parameter;

[0027] According to the angle between the guiding line and the negative direction of the image coordinate axis, through Calculate the direction angle deviation. Wherein, β2 is the direction angle deviation, and v x And v y Are the x and y components of the unit direction vector of the current guiding line in the image coordinate system respectively, v x,p And v y,p Are the x and y components of the unit direction vector of the previous frame of guiding line in the image coordinate system respectively, α is the fixed rotation angle of the camera coordinate system relative to the robot coordinate system, v is the current speed of the robot, and v max Is the maximum speed of the robot;

[0028] According to the control model, obtain the steering control amount of the current frame through ψ = β1 + β2. Wherein, β1 is the lateral error compensation angle, β2 is the direction angle deviation, and ψ is the steering control amount of the current frame.

[0029] Furthermore, according to the steering control amount, process the steering control amounts of two consecutive frames of images to obtain the steering angle of the rice weeding robot, including:

[0030] According to the steering control amount, calculate the error between the steering control amount of the current frame and the steering control amount of the previous frame to obtain an error value;

[0031] According to the error value, through Determine the adaptive filtering coefficient, where e is the error value and α is the adaptive filtering coefficient;

[0032] By performing a guide line jump detection on the error between two frames of steering control amounts, obtain the original steering control amount of the current frame and the filtered steering control amount of the previous frame;

[0033] According to the adaptive filtering coefficient, combine the original steering control amount of the current frame and the filtered steering control amount of the previous frame, and calculate the filtered steering control amount of the current frame through ω[k] = α×ψ+(1 - α)×ω[k - 1], where ω[k] is the image output after filtering for the k-th frame, ω[k - 1] is the image obtained by filtering for the (k - 1)-th frame, α is the adaptive filtering coefficient, and ψ is the steering control amount of the current frame, to obtain the control amount of the steering angle of the robot after filtering;

[0034] By continuously iterating the control amount of the steering angle of the robot after filtering, obtain a smoothed steering control amount;

[0035] Convert the filtered steering control amount into an actual steering angle to obtain the steering angle of the rice weeding robot.

[0036] Further, according to the steering angle, calculate the torque output by the steering motor of the rice weeding robot to achieve the tracking control of the robot's visual trajectory, including:

[0037] Design a PD controller;

[0038] According to the PD controller, through u1 = K p e (k) +K d Δe (k) Convert the steering angle into a motor torque value, where u1 is the control signal or torque value output by the PD controller, K p is the proportional term coefficient), e (k) is the difference between the desired steering angle and the actual steering angle, k d is the differential term coefficient, and Δe (k) is the difference between the current error and the previous error;

[0039] According to the motor torque value, control the rotation of the steering motor to form a closed-loop control of the motor to achieve the tracking control of the robot's visual trajectory.

[0040] In a second aspect, a visual trajectory tracking control system for a rice weeding robot includes:

[0041] An acquisition module, configured to perform real-time shooting on a rice field, process the taken pictures to obtain candidate line information; according to the candidate line information, screen the candidate lines, and use the straight line closest to the center line of the image as the guide line to obtain the guide line;

[0042] A processing module, configured to construct a control model according to the positional relationship between the guiding line and the image center line, calculate the control model to obtain the steering control amount of the current frame; and process the steering control amounts of two consecutive frames of images according to the steering control amount to obtain the steering angle of the rice weeding robot.

[0043] A control module, configured to calculate the torque output by the steering motor of the rice weeding robot according to the steering angle, and implement the tracking control of the robot's visual trajectory.

[0044] In a third aspect, a computing device includes:

[0045] One or more processors;

[0046] A storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, enable the one or more processors to implement the method described above.

[0047] In a fourth aspect, a computer-readable storage medium stores a program, and when the program is executed by a processor, the method described above is implemented.

[0048] The above solution of the present invention has at least the following beneficial effects:

[0049] By taking real-time pictures of the rice field and processing the images to obtain candidate line information, and then screening out the guiding line, an accurate navigation path is provided for the rice weeding robot. The robot can travel along a predetermined trajectory, ensuring the accuracy and efficiency of the weeding operation and reducing the accidental injury to non-target areas. Using an adaptive correction coefficient filter controlled by error to process the steering control amounts of two consecutive frames of images enables the robot to flexibly adjust the steering angle according to real-time situations, enhancing the adaptability and stability of the robot in the complex and changeable rice field environment and improving the success rate of the operation. Inputting the steering angle into a PD controller and calculating the output torque to implement the tracking control of the robot's visual trajectory can ensure that the robot quickly and accurately tracks the predetermined trajectory and improve the operation efficiency. Using an RGB camera and advanced image processing technology can work stably in the complex and changeable rice field environment, effectively identify and track the trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic flowchart of a method for visual trajectory tracking control of a rice weeding robot provided by an embodiment of the present invention.

[0051] Figure 2 is a schematic diagram of a visual trajectory tracking control system of a rice weeding robot provided by an embodiment of the present invention.

[0052] Figure 3 It is the positional relationship between the guiding line and the camera center line of a visual trajectory tracking control method for a rice weeding robot provided by an embodiment of the present invention.

[0053] Figure 4 It is a process diagram of the visual trajectory tracking control of a rice weeding robot provided by an embodiment of the present invention. Detailed implementation manners

[0054] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0055] As Figure 1 shown, an embodiment of the present invention provides a visual trajectory tracking control method for a rice weeding robot, and the method includes the following steps:

[0056] Step 11: Take a real-time photo of the rice field and process the taken picture to obtain candidate line information;

[0057] Step 12: Screen the candidate lines according to the candidate line information, and use the straight line closest to the image center line as the guiding line to obtain the guiding line;

[0058] Step 13: Construct a control model according to the positional relationship between the guiding line and the image center line, and calculate the control model to obtain the steering control amount of the current frame;

[0059] Step 14: Process the steering control amounts of two consecutive frames of images according to the steering control amount to obtain the steering angle of the rice weeding robot;

[0060] Step 15: Calculate the torque output by the steering motor of the rice weeding robot according to the steering angle to achieve the tracking control of the robot visual trajectory.

[0061] In an embodiment of the present invention, the paddy field is photographed in real time, and candidate line information is obtained through image processing, and then the guiding line is screened out, providing an accurate navigation path for the rice weeding robot, ensuring that the robot travels along a predetermined trajectory, and improving the accuracy and efficiency of weeding. The steering control amount of two consecutive frames of images is processed by an adaptive correction coefficient filter controlled by error, enabling the robot to flexibly adjust the steering angle according to real-time conditions, enhancing the robot's environmental adaptability and stability. The steering angle is input into the PD controller, and the output torque is calculated to achieve the tracking control of the robot's visual trajectory, ensuring that the robot can quickly and accurately track the predetermined trajectory. By using an RGB camera and advanced image processing technology, it can work stably in the complex and changeable paddy field environment, effectively identify and track the trajectory, and improve the robustness of the system.

[0062] In a preferred embodiment of the present invention, step 11 may include:

[0063] Step 111, fix the RGB camera at the front center position of the robot, photograph the paddy field in real time, obtain the paddy field image, and preprocess the original image captured by the camera to obtain the preprocessed image;

[0064] Step 112, detect the preprocessed image to obtain the coordinate points of the rice seedlings in the paddy field;

[0065] Step 113, perform coordinate system conversion on the seedling coordinate points, obtain the corresponding positions of the seedling rows from an aerial perspective, and obtain the seedling coordinate points after coordinate system conversion;

[0066] Step 114, classify the seedling coordinate points after coordinate system conversion, and divide them into different clusters according to the crop density around the seedling coordinate points to obtain the coordinate points of each cluster of seedlings;

[0067] Step 115, fit the coordinate points of each cluster of seedlings to obtain candidate line information.

[0068] In an embodiment of the present invention, by installing the RGB camera at the front center of the robot and tilting it downward, it is ensured that the camera can capture images of the paddy field in the traveling direction of the robot, and real-time shooting guarantees the immediacy of information. Necessary preprocessing of the images can effectively remove noise in the images, enhance image features, and improve image quality. Through the detection of rice seedlings, the coordinate points of the seedlings in the image can be accurately obtained. Transforming the coordinate points of the seedlings in the image coordinate system to the bird's-eye view can better reflect the distribution of the seedlings in the actual field, which helps the robot to comprehensively understand the operating environment. Grouping the seedling coordinate points through a clustering algorithm simplifies the data processing process and at the same time enables the robot to more easily identify different seedling rows or regions. Fitting a straight line to the clustered seedling coordinate points, the obtained candidate line information can be directly used as the basis for the robot's navigation, enabling the robot to travel along a predetermined path in the paddy field, improving the accuracy and efficiency of the operation.

[0069] In a specific embodiment of the present invention, the specific steps include:

[0070] Step 111: Install the RGB camera at the front center position of the robot to ensure that the camera's view can cover the paddy field area in front of the robot. Adjust the camera's angle to tilt it downward, connect the camera to the robot's control system, and configure relevant software for real-time shooting and image transmission. Start the camera to perform real-time shooting, capture images of the paddy field, and preprocess the images.

[0071] Step 112: Extract features from the preprocessed paddy field images to obtain a feature map. Divide the feature map into multiple grids and predict the bounding boxes for each grid to obtain the bounding boxes predicted for each grid. Obtain the overlap degree and confidence score of the bounding boxes. By comparing the confidence scores and overlap degrees of the bounding boxes, filter each retained bounding box. Classify each retained bounding box according to the category with the highest confidence score, and obtain the coordinate points of the rice seedlings in the paddy field based on the coordinates and sizes of the bounding boxes.

[0072] Step 113: Before performing the coordinate system transformation, calibrate the RGB camera by taking images of a calibration board at multiple known spatial positions to obtain the internal and external parameters of the camera. The internal parameters include the focal length and principal point coordinates, and the external parameters include the position and orientation of the camera. Use the camera parameters obtained from the calibration to perform a perspective transformation on the detected seedling coordinate points, mapping the points in the image coordinate system to the actual three-dimensional space, thereby obtaining the corresponding positions of the seedlings in the bird's-eye view.

[0073] Step 114: Cluster the transformed seedling coordinate points using a clustering algorithm. First, there is already a set of seedling coordinate points after coordinate system transformation, and the coordinate points reflect the actual positions of the seedlings in the field. Considering the characteristics of the seedling coordinate points, the DBSCAN clustering algorithm is selected to cluster the transformed seedling coordinate points. After clustering, a set of clusters is obtained, and each cluster represents a row of seedlings.

[0074] Step 115: Line fitting For the seedling coordinate points in each cluster, use the line fitting algorithm for fitting. The least squares method finds the best fitting line by minimizing the sum of the squares of the distances between the data points and the fitting line. For the seedling coordinate points in each cluster, calculate their mean value, which is used as the starting point reference for the fitting line, and use the least squares method to fit these points. After obtaining the fitting line for each cluster, these lines can be used as the navigation path for the robot. The robot can travel along these lines to ensure accurate coverage of each row of seedlings during operations such as weeding and fertilizing.

[0075] In a preferred embodiment of the present invention, the above step 12 may include:

[0076] Step 121: Obtain the candidate line parameters according to the candidate line information, and calculate the average value of the x-axis coordinates of the starting point and the ending point of each line to obtain the x-axis coordinate of the midpoint of the candidate line;

[0077] Step 122: According to the x-axis coordinate of the midpoint of the candidate line, calculate the distance between the average value and the x-axis coordinate of the center point of the image, where x is the x-axis coordinate of the starting point of each candidate line, x is the x-axis coordinate of the ending point of each candidate line, W is the width of the image, and b1 is the distance between the average value and the x-axis coordinate of the center point of the image; Calculate the distance between the average value and the x-axis coordinate of the center point of the image, where, x max is the coordinate of the starting point of each candidate line on the x-axis, x min is the coordinate of the ending point of each candidate line on the x-axis, W is the width of the image, and b1 is the distance between the average value and the x-axis coordinate of the center point of the image;

[0078] Step 123: Take the line with the closest distance between the average value of each candidate line and the center line of the image as the guiding line to obtain the guiding line.

[0079] In the embodiments of the present invention, by calculating the average value of the x-axis coordinates of the starting point and the ending point of each straight line, the midpoint of the candidate line can be more accurately located. Using the midpoint coordinates for calculation, compared with using all the points of the entire line, the amount of calculation is greatly reduced, and the processing speed is improved. By calculating the distance between the average value and the x-axis coordinate of the center point of the image, it is possible to effectively identify which candidate lines are closer to the center of the image, thereby making the selection of the guiding line more robust. By using the straight line with the closest distance between the average value of each candidate line and the center line of the image as the guiding line, the path for the robot to travel can be determined quickly and accurately, improving the efficiency and accuracy of navigation. During the movement of the robot, the guiding line can be updated in real time according to the current image information to ensure that the robot always moves along the correct path.

[0080] In a specific embodiment of the present invention, the specific steps include:

[0081] Step 121, according to the candidate line information, each candidate is represented as an object or data structure, which contains the starting point and ending point coordinates of the line. Read the x-axis coordinates of the starting point and the ending point of each candidate line, which represent the positions in the image. For each candidate line in the list, extract the x coordinate of the starting point and the x coordinate of the ending point respectively, and calculate the average value of these two coordinates to obtain the x-axis coordinate of the midpoint, which is the x-axis position of the center of the line segment. Repeat this process for each candidate line in the list to calculate the x-axis coordinate of the midpoint of each candidate line.

[0082] Step 122, obtain the x-axis coordinate of the center point of the image, which is obtained by dividing the width of the image by 2. For the x-axis coordinate of the midpoint of each candidate line calculated previously, calculate the difference between it and the x-axis coordinate of the center of the image through a formula. This difference represents the horizontal distance by which the midpoint deviates from the center of the image. Perform such calculations for each candidate line to obtain a distance list, which contains the horizontal distance from the midpoint of each candidate line to the center of the image.

[0083] Step 123, in the distance list obtained in the previous step, traverse this list to find the minimum value. The candidate line corresponding to this minimum value is the line closest to the center of the image. Select the candidate line with the minimum distance as the guiding line. This line will be used as the benchmark for subsequent robot navigation or path planning.

[0084] In a preferred embodiment of the present invention, the above step 13 may include:

[0085] Step 131, according to the position where the RGB camera is fixed on the robot, obtain the position of the center line of the image;

[0086] Step 132, according to the angle between the guiding line and the negative direction of the image coordinate axis, calculate the direction angle deviation and calculate the lateral distance deviation between the guiding line and the center line of the image;

[0087] Step 133, according to the lateral distance deviation, through calculate the lateral error compensation, where β1 is the lateral error compensation angle, and its value range is -90° < β1 < 90°; b is the lateral distance deviation, and c is the compensation parameter;

[0088] Step 134, according to the control model, through ψ = β1 + β2, obtain the steering control amount of the current frame, where β1 is the lateral error compensation angle, β2 is the direction angle deviation, and ψ is the steering control amount of the current frame.

[0089] In the embodiment of the present invention, taking the image center line as the reference benchmark helps to accurately calculate the relative position and deviation between the guiding line and the center line. Directly using the image center line without complex calculations improves the processing speed. By calculating the angle between the guiding line and the image coordinate axis and the lateral distance between the guiding line and the image center line, the direction deviation and position deviation of the robot can be accurately obtained, and the direction error and position offset of the robot during the traveling process can be discovered and corrected in time to ensure that the robot travels along the predetermined path. Through parameters such as the lateral traveling error compensation angle and the direction angle deviation, the constructed control model can adaptively adjust the state of the robot according to the real-time situation, improving the adaptability and flexibility of the robot. The steering control amount of the current frame calculated according to the control model can realize the real-time control of the robot's steering, ensuring that the robot responds quickly and executes the steering action accurately. By continuously calculating the steering control amount of each frame, the traveling of the robot can be made smoother, reducing the bumps and pauses caused by improper steering.

[0090] In a specific embodiment of the present invention, the specific steps include:

[0091] Step 131, since the RGB camera is fixed on the robot, first confirm the installation position and orientation of the camera, and the image center line is defined as the middle of the image width; assuming the width of the image is W, the x coordinate of the image center line is W / 2.

[0092] Step 132, the guiding line represents the path that the robot should follow. Calculate the direction angle deviation and the lateral distance deviation through formulas, and the lateral distance deviation refers to the horizontal distance from the guiding line to the image center line.

[0093] Step 133: Determine a compensation angle based on the direction angle deviation for adjusting the traveling direction of the robot to keep it consistent with the guiding line. The lateral error compensation is calculated based on the lateral distance deviation, indicating the adjustment amount that the robot needs to make laterally to eliminate this deviation. Through a proportional control strategy, multiply the lateral distance deviation by a compensation parameter, which is an empirical value or a constant determined through experiments, to obtain the lateral error compensation. Combine the direction angle compensation and the lateral error compensation to construct a control model for generating the steering control amount of the robot.

[0094] Step 134: Automatically input the calculated direction angle compensation and lateral error compensation into the control model. The control model calculates the steering control amount of the current frame through a formula based on these inputs. This control amount is a specific value indicating the angle or speed by which the robot needs to adjust its traveling direction. Output the calculated steering control amount to the control system of the robot to adjust the traveling direction of the robot.

[0095] In a preferred embodiment of the present invention, the above step 132 may include:

[0096] Step 1321: Calculate the direction angle deviation according to the angle between the guiding line and the negative direction of the image coordinate axis through where β2 is the direction angle deviation, v x and v y are the x and y components of the unit direction vector of the current guiding line in the image coordinate system respectively, v x,p and v y,p are the x and y components of the unit direction vector of the previous frame's guiding line in the image coordinate system respectively, γ is the fixed rotation angle of the camera coordinate system relative to the robot coordinate system, v is the current speed of the robot, and v max is the maximum speed of the robot;

[0097] Step 1322: Calculate the lateral distance deviation between the guiding line and the image center line through where b is the lateral distance deviation, f x is the focal length of the camera, u c and v c are the u and v coordinates of the image center line in the image coordinate system respectively, u0 and v0 are the u and v coordinates of the intersection point of the guiding line and the lower edge of the image, D is the vertical distance from the camera to the seedling row, Z is the Z coordinate of the seedling row plane in the camera coordinate system, Δu is the u coordinate base offset, u m is the change rate of the u coordinate of the image center line, and k1 and k2 are distortion coefficients, and k3 is the dynamic correction coefficient.

[0098] In an embodiment of the present invention, by calculating the direction angle deviation and the lateral distance deviation, the robot can travel more precisely along a predetermined path. This improvement in accuracy helps to reduce the situation of deviating from the path, thereby improving the operation efficiency and quality. Since the angle between the guiding line and the image coordinate axis and the direction change between multiple frames are considered, it can better adapt to the bending and change of the path. In a complex farmland environment, by introducing a dynamic correction coefficient and other real-time parameters, such as the current speed and the maximum speed of the robot, the system can perform real-time adjustment according to the actual situation, so that the robot can maintain an efficient and stable working state when facing different conditions. The introduction of the distortion coefficient helps to correct image distortion, thereby improving the robustness of the visual navigation system. Through precise navigation and adjustment capabilities, the robot can perform tasks more accurately, reducing the need for manual intervention, thereby reducing labor costs and increasing production efficiency.

[0099] In a preferred embodiment of the present invention, the above step 14 may include:

[0100] Step 141, according to the steering control amount, calculate the error between the steering control amount of the current frame and the steering control amount of the previous frame to obtain an error value;

[0101] Step 142, according to the error value, by Determine the adaptive filtering coefficient, where e is the error value and α is the adaptive filtering coefficient;

[0102] Step 143, through the guiding line jump detection of the error between the steering control amounts of two frames, obtain the original steering control amount of the current frame and the filtered steering control amount of the previous frame;

[0103] Step 144, according to the adaptive filtering coefficient, combining the original steering control amount of the current frame and the filtered steering control amount of the previous frame, calculate the filtered steering control amount of the current frame through ω[k] = α×ψ+(1-α)×ω[k-1], where ω[k] is the image output by filtering the kth frame, ω[k-1] is the image obtained by filtering the (k-1)th frame, α is the adaptive filtering coefficient, and ψ is the steering control amount of the current frame, to obtain the control amount of the steering angle of the robot after filtering;

[0104] Step 145, by continuously iterating the control amount of the steering angle of the robot after filtering, obtain the smoothed steering control amount;

[0105] Step 146, convert the filtered steering control amount into an actual steering angle to obtain the steering angle of the rice weeding robot.

[0106] In the embodiment of the present invention, by calculating the error between the current frame and the steering control amount of the previous frame, the change of the steering state of the robot can be accurately understood. The adaptive filtering coefficient is dynamically determined according to the error value, making the filtering process more flexible and capable of adjusting the filtering intensity according to the actual situation. By performing leading line jump detection on the error between the steering control amounts of two frames, abnormal situations can be detected and processed in a timely manner to ensure the stability of the robot's driving. Combining the original steering control amount of the current frame, the filtered steering control amount of the previous frame, and the adaptive filtering coefficient, a smoother filtered steering control amount of the current frame can be calculated to reduce the jitter and mutation of the robot. By continuously iterating the filtered steering control amount, the steering control of the robot can be made more continuous and stable, improving the smoothness of driving. Converting the filtered steering control amount into an actual steering angle makes the steering control of the robot more intuitive and practical.

[0107] In a specific embodiment of the present invention, the specific steps include:

[0108] Step 141: Obtain the steering control amount of the previous frame, perform a subtraction operation on the numerical value of the steering control amount of the current frame and the numerical value of the steering control amount of the previous frame to obtain the difference between the two. This difference, that is, the error value, reflects the change of the steering control amount from the previous frame to the current frame. Save the calculated error value in a variable.

[0109] Step 142: Observe the magnitude and sign of the error value calculated in Step 141, analyze the statistical characteristics of the error value, such as the mean, variance, etc., to understand the fluctuation of the steering control amount. According to the application scenario and requirements, select the formula for calculating the adaptive filtering coefficient, and dynamically calculate the adaptive filtering coefficient in combination with the error value. The adaptive filtering coefficient is a value between 0 and 1, which is used to balance the weights of new data and old data during the filtering process.

[0110] Step 143: Obtain the original steering control amount of the current frame and the filtered steering control amount of the previous frame. Calculate the error value by taking the difference from the steering control amount of the previous frame. During actual testing, if the difference from the error of the previous frame is less than 5, then according to the relationship of a linear function, the weight of the proportion of the current frame decreases with the increase of the error, making the control smoother. When the error between two frames is greater than 5 and less than 10, assign the weight of the current frame as 0.9. Since the plane of the paddy field is not completely flat, it causes the camera not to be parallel to the ground when shooting. At this time, after steps such as the above-mentioned candidate line recognition and candidate line screening, there is a large deviation between the obtained guiding line and the guiding line detected in the previous frame, which is hereinafter simply referred to as the guiding line jump. Therefore, the error between the steering control amounts of two frames is used to detect whether there is a guiding line jump. After multiple tests, it is obtained that when the error is greater than 10, the phenomenon of guiding line jump occurs. Since the guiding line screened in the previous frame is appropriate, the calculated steering control amount is also the correct value and the control frequency is high. The steering control amount of the current frame can completely adopt the steering control amount detected in the previous frame.

[0111] Step 144: Use the adaptive filtering coefficient, combine the original steering control amount of the current frame and the filtered steering control amount of the previous frame, and calculate the filtered steering control amount of the current frame according to the filtering formula. Save the calculated filtered steering control amount of the current frame as the new filtering state for use in the next frame calculation.

[0112] Step 145: As time goes by and new image frames arrive, continuously repeat the process of steps 141 to 144. Each iteration will update the steering control amount according to the new image data and the filtering state. During the iteration process, continuously monitor the filtering effect and adjust the adaptive filtering coefficient as needed to ensure that the filtered steering control amount can smoothly follow the change of the guiding line.

[0113] Step 146: According to the mechanical structure and control system design of the robot, determine the mapping relationship between the steering control amount and the actual steering angle, and convert the filtered steering control amount into the actual steering angle value.

[0114] In a preferred embodiment of the present invention, the above step 15 may include:

[0115] Step 151: Design a PD controller;

[0116] Step 152: According to the PD controller, convert the steering angle into the motor torque value through u1 = K p e (k) +K d Δe (k) where u1 is the control signal or torque value output by the PD controller, and K pis the proportional term coefficient), e (k) is the difference between the desired steering angle and the actual steering angle, K d is the derivative term coefficient, Δe (k) is the difference between the current error and the previous error;

[0117] Step 153: According to the motor torque value, control the steering motor to rotate to form a closed-loop control of the motor, so as to realize the tracking control of the robot's visual trajectory.

[0118] In the embodiment of the present invention, the design of the PD controller can effectively improve the dynamic response performance and stability of the system, ensuring the accuracy and rapidity of the robot during the trajectory tracking process. By adopting the PD controller, the design complexity of the control system can be simplified, the implementation difficulty can be reduced, and the reliability and usability of the system can be improved. Converting the steering angle into the motor torque value realizes the precise control of the robot's steering, ensuring that the robot can accurately travel along the predetermined trajectory. By adjusting the proportional term coefficient and the derivative term coefficient, the performance of the controller can be flexibly adjusted to adapt to different working environments and task requirements. The closed-loop control system can monitor the actual steering angle of the robot in real time and compare it with the desired steering angle, so as to timely adjust the motor torque value to ensure that the robot always travels along the predetermined trajectory. The closed-loop control enhances the robustness of the system, enabling the robot to maintain stable trajectory tracking performance in the face of external disturbances or uncertainties.

[0119] In a specific embodiment of the present invention, the specific steps include:

[0120] Step 151: The PD controller, that is, the proportional-derivative controller, is a commonly used control algorithm. It adjusts the output of the system through the proportional term and the derivative term to achieve the desired control effect. In this embodiment, the control objective is to make the actual steering angle of the robot follow the desired steering angle to realize trajectory tracking. The proportional term coefficient determines the reaction speed of the system to the current error. The larger the coefficient, the faster the system reacts to the error, but too large will cause the system to be unstable. The derivative term coefficient is used to predict the change trend of the error and make a reaction in advance, which helps to reduce overshoot and improve the stability of the system. Write code or logic circuits to output control signals according to the calculation formulas of the proportional term and the derivative term.

[0121] Step 152: The desired steering angle is obtained from the trajectory planning or visual navigation system, indicating the angle that the robot should turn. The actual steering angle is obtained from the steering motor of the robot through a sensor (such as an encoder), indicating the current actual steering angle of the robot. The error is the difference between the desired steering angle and the actual steering angle. The error change rate is the difference between the current error and the previous error divided by the time interval, indicating the change speed of the error. Use the formula to calculate the output of the PD controller.

[0122] Step 153: Send the control signal or torque value calculated in Step 152 to the driver of the steering motor. After receiving the control signal, the motor driver drives the motor to rotate according to the magnitude and direction of the signal. The rotation of the motor drives the steering mechanism of the robot, causing the robot to turn at the desired steering angle. During the turning of the robot, the actual steering angle is continuously detected by sensors and fed back to the control system. The control system compares the fed-back actual steering angle with the desired steering angle, calculates the error and the rate of change of the error again, and then updates the control signal to form a closed-loop control. As the robot moves and turns, the control system continuously adjusts the control signal according to the feedback signal to ensure that the actual steering angle of the robot always follows the desired steering angle. Through this closed-loop control method, precise tracking control of the robot's visual trajectory can be achieved.

[0123] As Figure 2 shown, an embodiment of the present invention also provides a visual trajectory tracking control system 20 for a rice weeding robot, including:

[0124] An acquisition module 21, configured to perform real-time shooting on a paddy field, process the taken pictures to obtain candidate line information; according to the candidate line information, screen the candidate lines, and use the straight line closest to the center line of the image as the guiding line to obtain the guiding line;

[0125] A processing module 22, configured to construct a control model according to the positional relationship between the guiding line and the center line of the image, calculate the steering control amount of the current frame for the control model; according to the steering control amount, process the steering control amounts of two consecutive frames of images to obtain the steering angle of the rice weeding robot.

[0126] A control module 23, configured to calculate the torque output by the steering motor of the rice weeding robot according to the steering angle, and achieve tracking control of the robot's visual trajectory.

[0127] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle described in the present invention, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A visual trajectory tracking control method for a rice weeding robot, characterized in that, The method includes: Taking real-time pictures of the paddy field and processing the taken pictures to obtain candidate line information; According to the candidate line information, screening the candidate lines, and taking the straight line closest to the center line of the image as the guiding line to obtain the guiding line; Constructing a control model based on the positional relationship between the guiding line and the center line of the image, and calculating the control model to obtain the steering control amount of the current frame; According to the steering control amount, processing the steering control amounts of two consecutive frames of images to obtain the steering angle of the rice weeding robot; According to the steering angle, calculating the torque output by the steering motor of the rice weeding robot to achieve the tracking control of the robot's visual trajectory.

2. The visual trajectory tracking control method of the rice weeding robot according to claim 1, characterized in that Taking real-time pictures of the paddy field and processing the taken pictures to obtain candidate line information, including: Fixing the RGB camera at the front center position of the robot, taking real-time pictures of the paddy field, obtaining the paddy field image, and preprocessing the original image taken by the camera to obtain the preprocessed image; Detecting the preprocessed image to obtain the coordinate points of the rice seedlings in the paddy field; Converting the coordinate system of the seedling coordinate points to obtain the corresponding positions of the seedling rows from an aerial perspective to obtain the coordinate points of the seedlings after coordinate system conversion; Classifying the coordinate points of the seedlings after coordinate system conversion and dividing them into different clusters according to the crop density around the seedling coordinate points to obtain the coordinate points of each cluster of seedlings; Fitting the coordinate points of each cluster of seedlings to obtain candidate line information.

3. The visual trajectory tracking control method of the rice weeding robot according to claim 1, characterized in that According to the candidate line information, screening the candidate lines, and taking the straight line closest to the center line of the image as the guiding line to obtain the guiding line, including: According to the candidate line information, obtaining candidate line parameters, and calculating the average value of the x-axis coordinates of the starting point and the ending point of each straight line to obtain the x-axis coordinate of the midpoint coordinate of the candidate line; According to the x-axis coordinate of the midpoint of the candidate line, by calculating the distance between the average value and the x-axis coordinate of the center point of the image, where x max is the coordinate of the starting point of each candidate line on the x-axis, x min is the coordinate of the ending point of each candidate line on the x-axis, W is the width of the image, and b1 is the distance between the average value and the x-axis coordinate of the center point of the image; Taking the straight line closest to the average value of each candidate line and the center line of the image as the guiding line to obtain the guiding line.

4. The visual trajectory tracking control method of the rice weeding robot according to claim 1, wherein, Constructing a control model based on the positional relationship between the guiding line and the center line of the image, and calculating the control model to obtain the steering control amount of the current frame, including: According to the position where the RGB camera is fixed on the robot, obtaining the position of the center line of the image; Calculating the direction angle deviation according to the angle between the guiding line and the negative direction of the image coordinate axis, and calculating the lateral distance deviation between the guiding line and the center line of the image; According to the lateral distance deviation, by calculating the lateral error compensation, where β1 is the lateral error compensation angle, and its value range is -90° < β1 < 90°; b is the lateral distance deviation, and c is the compensation parameter; According to the control model, through ψ = β1 + β2, obtaining the steering control amount of the current frame, where β1 is the lateral error compensation angle, β2 is the direction angle deviation, and ψ is the steering control amount of the current frame.

5. The visual trajectory tracking control method of the rice weeding robot according to claim 1, wherein, Calculating the direction angle deviation according to the angle between the guiding line and the negative direction of the image coordinate axis, and calculating the lateral distance deviation between the guiding line and the center line of the image, including: According to the angle between the guiding line and the negative direction of the image coordinate axis, calculate the direction angle deviation through where β2 is the direction angle deviation, v x and v y are the x and y components of the unit direction vector of the current guiding line in the image coordinate system respectively, v x,p and v y,p are the x and y components of the unit direction vector of the previous frame guiding line in the image coordinate system respectively, γ is the fixed rotation angle of the camera coordinate system relative to the robot coordinate system, v is the current speed of the robot, and v max is the maximum speed of the robot; By calculating the lateral distance deviation between the guiding line and the image center line, where b is the lateral distance deviation, f x is the focal length of the camera, u c and v c are the u and v coordinates of the image center line in the image coordinate system respectively, u0 and v0 are the u and v coordinates of the intersection point of the guiding line and the lower edge of the image, D is the vertical distance from the camera to the seedling row, Z is the Z coordinate of the seedling row plane in the camera coordinate system, Δu is the basic offset of the u coordinate, u m is the change rate of the u coordinate of the image center line, k1 and k2 are distortion coefficients, and k3 is the dynamic correction coefficient.

6. The visual trajectory tracking control method of the rice weeding robot according to claim 1, characterized in that According to the steering control amount, processing the steering control amounts of two consecutive frames of images to obtain the steering angle of the rice weeding robot, including: According to the steering control amount, calculating the error between the steering control amount of the current frame and the steering control amount of the previous frame to obtain the error value; According to the error value, through determine the adaptive filtering coefficient, where e is the error value and α is the adaptive filtering coefficient; Detecting the jump of the guiding line through the error between the steering control amounts of two frames to obtain the original steering control amount of the current frame and the filtered steering control amount of the previous frame; According to the adaptive filtering coefficient, combining the original steering control amount of the current frame and the filtered steering control amount of the previous frame, calculate the filtered steering control amount of the current frame through ω[k] = α×ψ+(1 - α)×ω[k - 1], where ω[k] is the image output by filtering in the k-th frame, ω[k - 1] is the image obtained by filtering in the (k - 1)-th frame, α is the adaptive filtering coefficient, and ψ is the steering control amount of the current frame, to obtain the control amount of the steering angle of the robot after filtering; By continuously iterating the control amount of the steering angle of the robot after filtering, obtain the smoothed steering control amount; Convert the filtered steering control amount into the actual steering angle to obtain the steering angle of the rice weeding robot.

7. The visual trajectory tracking control method of the rice weeding robot according to claim 1, wherein According to the steering angle, calculate the torque output by the steering motor of the rice weeding robot to achieve the tracking control of the robot's visual trajectory, including: Design a PD controller; According to the PD controller, through u1 = K p e (k) +K d Δr (k) the steering angle is converted into a motor torque value, where u1 is the control signal or torque value output by the PD controller, K p is the proportional term coefficient), e (k) is the difference between the desired steering angle and the actual steering angle, K d is the differential term coefficient, Δe (k) is the difference between the current error and the previous error; According to the motor torque value, control the rotation of the steering motor to form a closed-loop control of the motor, so as to achieve the tracking control of the robot's visual trajectory.

8. A visual trajectory tracking control system for a rice weeding robot, characterized in that, Including: An acquisition module, used for taking real-time pictures of the rice field and processing the taken pictures to obtain candidate line information; According to the candidate line information, screen the candidate lines, and take the straight line closest to the center line of the image as the guiding line to obtain the guiding line; A processing module, used for constructing a control model according to the position relationship between the guiding line and the center line of the image, and calculating the control model to obtain the steering control amount of the current frame; According to the steering control amount, process the steering control amounts of two consecutive frames of images to obtain the steering angle of the rice weeding robot. A control module, used for calculating the torque output by the steering motor of the rice weeding robot according to the steering angle to achieve the tracking control of the robot's visual trajectory.

9. A computing device, characterized in that, Including: One or more processors; A storage device, used for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored in the computer-readable storage medium, and when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images