Automatic navigation method and device for agricultural robot

Abstract

The invention relates to the technical field of agricultural robots, and particularly provides an automatic navigation method and device for an agricultural robot, and the method comprises the following steps: S1, carrying out the filtering processing of a grayscale image, so as to obtain a filtered image; s2, calculating a horizontal direction gradient component and a vertical direction gradient component of each pixel point in the filtered image to obtain a gradient direction and a gradient amplitude of each pixel point in the filtered image; s3, constructing a gradient direction point set based on all the pixel points of which the gradient amplitudes are greater than an amplitude threshold, wherein the gradient direction point set comprises a plurality of pixel point coordinates and gradient directions corresponding to the plurality of pixel point coordinates; s4, straight line fitting is carried out on the gradient direction point set based on an RANSAC algorithm to obtain a straight line used for representing the crop row direction, and the weighted weight used in the iteration process of the RANSAC algorithm is determined by the gradient amplitude corresponding to the pixel points; s5, controlling the agricultural robot to run according to the straight line; according to the method, accurate identification of crop rows can be realized in the presence of trapezoidal distortion.

Description

Technical Field

[0001] This application relates to the technical field of agricultural robots, and more specifically, to an automatic navigation method and device for agricultural robots. Background Art

[0002] In agricultural production in complex terrains such as hilly and mountainous areas, it has become an inevitable trend to use agricultural robots to replace manual labor for fine field operations. To achieve autonomous operation of agricultural robots, vision sensors are widely used to acquire images of field crops, and image processing technology is combined to identify crop rows and guide agricultural robots to drive along the crop rows. However, different from flat fields, the terrain of hilly and mountainous areas is rough and uneven, and there is usually a certain slope in the fields. When an agricultural robot operates on an inclined field surface, the vision sensor fixedly installed on the robot body will shift due to the inclination of the body posture, resulting in obvious trapezoidal distortion in the collected crop images. This trapezoidal distortion makes the originally parallel crop rows converge or diverge in the image, seriously damaging the geometric structure of the image. Since the existing agricultural robot navigation methods usually rely on the straight line features or specific shape features of crop rows in the image for crop row recognition, and under the influence of trapezoidal distortion, the straight line features of crop rows are no longer obvious and the shape features are also distorted, there is a problem in the existing technology that the recognition accuracy of crop rows is significantly reduced due to trapezoidal distortion, thus seriously affecting the accuracy of the agricultural robot driving along the crop rows and the navigation accuracy of the agricultural robot navigation method, and further affecting the operation quality of the agricultural robot.

[0003] In view of the above problems, there is currently no effective technical solution. It should be noted that the above information disclosed in this part is only used to understand the background of the inventive concept of the present invention, and therefore may include information that does not constitute prior art. Summary of the Invention

[0004] The purpose of this application is to provide an automatic navigation method and device for agricultural robots, which can achieve accurate recognition of crop rows in the presence of trapezoidal distortion.

[0005] In a first aspect, this application provides an automatic navigation method for an agricultural robot. The agricultural robot is applied in hilly and mountainous areas, and a vision sensor for collecting grayscale images is installed on the agricultural robot. The automatic navigation method for the agricultural robot includes the following steps: S1. Perform filtering processing on the grayscale image to obtain a filtered image; S2. Calculate the horizontal direction gradient component and vertical direction gradient component of each pixel point in the filtered image to obtain the gradient direction and gradient amplitude of each pixel point in the filtered image; S3. Construct a gradient direction point set based on all pixel points whose gradient magnitude is greater than the magnitude threshold. The gradient direction point set includes multiple pixel point coordinates and their corresponding gradient directions; S4. Perform line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a line for characterizing the crop row direction. The weighted weight used by the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point; S5. Control the agricultural robot to travel according to the line.

[0006] An automatic navigation method for an agricultural robot provided by this application first obtains the gradient direction and gradient magnitude of each pixel point in the filtered image, then constructs a gradient direction point set based on all pixel points whose gradient magnitude is greater than the magnitude threshold, and finally performs line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a line for characterizing the crop row direction. Since the gradient direction can effectively indicate the extension direction of the crop row, the gradient magnitude can accurately reflect the clarity of the crop row edge area, and the weighted weight used by the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point, that is, this application enables pixel points with a large gradient magnitude to have a greater impact on the line fitting result, so as to improve the fitting accuracy and the accuracy of the finally formed line. Therefore, even in the presence of trapezoidal distortion, the line obtained by this application can accurately reflect the crop row direction, that is, this application can achieve accurate recognition of crop rows in the presence of trapezoidal distortion, thereby effectively solving the problem that the recognition accuracy of crop rows is significantly reduced due to trapezoidal distortion, and further effectively improving the accuracy of the agricultural robot traveling along the crop row, the navigation accuracy of the agricultural robot navigation method, and the operation quality of the agricultural robot.

[0007] Optionally, step S4 includes: S41. Count the number of pixel points corresponding to each gradient direction in the gradient direction point set, and eliminate the gradient directions with the number of pixel points less than the number threshold to obtain a preprocessed gradient direction point set; S42. Randomly select at least two pixel points from the preprocessed gradient direction point set, and construct an initial line model based on the selected pixel points; S43. For each pixel point in the preprocessed gradient direction point set, obtain a weighted weight according to its corresponding gradient magnitude and the first preset conversion relationship, and calculate the weighted distance according to the distance from the pixel point to the initial line model and the weighted weight; S44. Divide all pixel points with a weighted distance less than the distance threshold into the same data set to obtain an inlier set, and calculate the sum of the gradient magnitudes corresponding to each pixel point in the inlier set to obtain the total inlier set gradient magnitude; S45. Analyze whether the number of iteration times reaches the preset upper limit of the number of iteration times. If so, select the initial straight-line model with the largest sum of gradient magnitudes of the inlier set as the straight line used to represent the crop row direction. If not, return to step S42.

[0008] Due to the influence of various factors, the crop edge information in the grayscale image collected by the vision sensor is unclear and there is a lot of noise. And this technical solution can eliminate the noise gradient directions with fewer pixel points and highlight the main gradient directions of the crop row direction by eliminating the gradient directions with the number of pixel points less than the number threshold. This can reduce the possibility of interference with the subsequent RANSAC algorithm. This technical solution can also make the RANSAC algorithm pay more attention to the pixels with large gradient magnitudes by calculating the weighted distance. And this technical solution can also ensure the robustness and accuracy of the RANSAC algorithm by calculating the sum of gradient magnitudes of the inlier set to evaluate the quality of the straight-line model and selecting the straight-line model with the largest sum of gradient magnitudes of the inlier set as the final result. Therefore, this technical solution can further improve the accuracy of the finally formed straight line, so that the straight line can more accurately reflect the crop row direction, thereby further improving the accuracy of the agricultural robot moving along the crop row and the navigation accuracy of the agricultural robot navigation method.

[0009] Optionally, the vision sensor is also used to collect color image information. Step S41 includes: S411. Construct a gradient direction histogram. The horizontal axis of the gradient direction histogram is the quantized gradient direction, and the vertical axis of the gradient direction histogram is the number of pixel points corresponding to the gradient direction. S412. Perform Gaussian filtering and smoothing processing on the gradient direction histogram to reduce noise interference. S413. Calculate the mean value of the number of pixel points in each gradient direction in the gradient direction histogram after Gaussian filtering and smoothing processing to obtain the mean value of the number of pixel points. S414. Obtain the crop growth stage according to the color image information, determine the quantity adjustment coefficient according to the crop growth stage, and then multiply the quantity adjustment coefficient by the mean value of the number of pixel points to obtain the number threshold. S415. Eliminate the gradient directions with the number of pixel points less than the number threshold to obtain the preprocessed gradient direction point set.

[0010] Since the edge features of the same crop are different at different growth stages, and this technical solution is equivalent to flexibly adjusting the number threshold according to the crop growth stage to remove more noise points. Therefore, this technical solution can effectively improve the fitting accuracy and accuracy of the straight-line fitting, thereby effectively enhancing the stability and reliability of crop row recognition in complex field environments.

[0011] Optionally, a light intensity acquisition component is also installed on the agricultural robot, and step S414 includes: S4141. Obtain the crop growth stage according to the color image information, and obtain the light intensity when the vision sensor acquires the grayscale image; S4142. Determine the quantity adjustment coefficient according to the crop growth stage and the light intensity; S4143. Multiply the quantity adjustment coefficient by the average value of the number of pixel points to obtain the quantity threshold.

[0012] Since the change of field illumination will affect the clarity of crop edges, and this technical solution comprehensively considers the light intensity and the crop growth stage when calculating the quantity threshold, this technical solution can effectively improve the accuracy of the quantity threshold, thereby effectively improving the fitting accuracy and accuracy of line fitting.

[0013] Optionally, step S4141 includes: S41411. Obtain the crop growth stage and crop type information according to the color image information, and obtain the light intensity when the vision sensor acquires the grayscale image; Step S4142 includes: S41421. Obtain the quantity adjustment coefficient according to the crop growth stage, crop type information, light intensity and the second preset conversion relationship. The second preset conversion relationship is a mapping relationship table pre-constructed to include the corresponding relationship between the crop growth stage, light intensity, crop type and the direction adjustment coefficient.

[0014] Since there are differences in the edge characteristics of different types of crops, this technical solution comprehensively considers the influences of the crop growth stage, crop type and light intensity when determining the quantity adjustment coefficient. Therefore, this technical solution can make the setting of the quantity threshold more reasonable, so as to more effectively eliminate the noise gradient direction and retain the effective gradient direction representing the crop row direction, thereby further improving the fitting accuracy and accuracy of line fitting.

[0015] Optionally, step S41411 includes: S414111. Obtain the crop growth stage, crop type information and weed occlusion ratio according to the color image information, and obtain the light intensity when the vision sensor acquires the grayscale image; S414112. Correct the light intensity based on the weed occlusion ratio.

[0016] Since this technical solution can obtain the weed occlusion ratio and correct the light intensity based on the weed occlusion ratio, it can effectively avoid the situation where the actual light intensity received by the crop changes due to weed occlusion, and there is a deviation between the light intensity collected by the light intensity acquisition component and the actual light intensity received by the crop. Thus, it can effectively avoid the situation where the accuracy of the quantity adjustment coefficient determined based on the light intensity collected by the light intensity acquisition component decreases due to the deviation between the light intensity collected by the light intensity acquisition component and the actual light intensity received by the crop.

[0017] Optionally, step S2 includes: S21. Calculate the average gray value of the filtered image, and obtain the gradient amplitude adjustment coefficient according to the average gray value and the third preset conversion relationship; S22. Multiply the gradient amplitude adjustment coefficient by the preset gradient amplitude threshold to obtain the adaptive gradient amplitude threshold; S23. Construct a gradient direction point set based on all pixel points whose gradient amplitude is greater than the adaptive gradient amplitude threshold.

[0018] Optionally, step S1 includes: S11. Perform Gaussian filtering and smoothing on the gray image to obtain a filtered image.

[0019] Optionally, step S5 includes: S51. Generate a moving path according to the straight line based on the path planning method, and control the movement of the agricultural robot according to the moving path.

[0020] In a second aspect, the present application further provides an automatic navigation device for an agricultural robot. The agricultural robot is applied in hilly and mountainous areas. A vision sensor for collecting gray images is installed on the agricultural robot. The automatic navigation device for the agricultural robot includes: A filtering module for filtering the gray image to obtain a filtered image; A gradient calculation module for calculating the horizontal direction gradient component and the vertical direction gradient component of each pixel point in the filtered image to obtain the gradient direction and gradient amplitude of each pixel point in the filtered image; A gradient direction point set construction module for constructing a gradient direction point set based on all pixel points whose gradient amplitude is greater than the amplitude threshold. The gradient direction point set includes the coordinates of multiple pixel points and their corresponding gradient directions; A straight line generation module for performing straight line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a straight line for characterizing the crop row direction. The weighted weight used in the iteration process of the RANSAC algorithm is determined by the gradient amplitude corresponding to the pixel point; A navigation module for controlling the agricultural robot to travel according to the straight line.

[0021] An automatic navigation device for an agricultural robot provided by the present application first obtains the gradient direction and gradient magnitude of each pixel point in the filtered image, then constructs a gradient direction point set based on all pixel points with a gradient magnitude greater than the magnitude threshold, and finally performs line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a line for characterizing the crop row direction. Since the gradient direction can effectively indicate the extension direction of the crop row, the gradient magnitude can accurately reflect the clarity of the crop row edge region, and the weighted weight used in the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point, that is, the present application enables pixel points with a large gradient magnitude to have a greater impact on the line fitting result, so as to improve the fitting accuracy and the accuracy of the finally formed line. Therefore, even in the presence of trapezoidal distortion, the line obtained by the present application can accurately reflect the crop row direction, that is, the present application can achieve accurate identification of the crop row in the presence of trapezoidal distortion, thereby effectively solving the problem that the recognition accuracy of the crop row is significantly reduced due to trapezoidal distortion, and further effectively improving the accuracy of the agricultural robot traveling along the crop row, the navigation accuracy of the agricultural robot navigation method, and the operation quality of the agricultural robot.

[0022] As can be seen from the above, an automatic navigation method and device for an agricultural robot provided by the present application first obtain the gradient direction and gradient magnitude of each pixel point in the filtered image, then construct a gradient direction point set based on all pixel points with a gradient magnitude greater than the magnitude threshold, and finally perform line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a line for characterizing the crop row direction. Since the gradient direction can effectively indicate the extension direction of the crop row, the gradient magnitude can accurately reflect the clarity of the crop row edge region, and the weighted weight used in the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point, that is, the present application enables pixel points with a large gradient magnitude to have a greater impact on the line fitting result, so as to improve the fitting accuracy and the accuracy of the finally formed line. Therefore, even in the presence of trapezoidal distortion, the line obtained by the present application can accurately reflect the crop row direction, that is, the present application can achieve accurate identification of the crop row in the presence of trapezoidal distortion, thereby effectively solving the problem that the recognition accuracy of the crop row is significantly reduced due to trapezoidal distortion, and further effectively improving the accuracy of the agricultural robot traveling along the crop row, the navigation accuracy of the agricultural robot navigation method, and the operation quality of the agricultural robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flowchart of an automatic navigation method for an agricultural robot provided by an embodiment of the present application.

[0024] Figure 2Schematic structural diagram of an automatic navigation device for an agricultural robot provided by an embodiment of the present application.

[0025] Reference numerals: 1, filtering module; 2, gradient calculation module; 3, gradient direction point set construction module; 4, straight line generation module; 5, navigation module. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Components of the embodiments of the present application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0027] It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, terms such as "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0028] In the first aspect, as Figure 1 shown, the present application provides an automatic navigation method for an agricultural robot. The agricultural robot is applied in hilly and mountainous areas. A visual sensor for collecting grayscale images is installed on the agricultural robot. The automatic navigation method for the agricultural robot includes the following steps: S1. Perform filtering processing on the grayscale image to obtain a filtered image; S2. Calculate the horizontal direction gradient component and vertical direction gradient component of each pixel point in the filtered image to obtain the gradient direction and gradient amplitude of each pixel point in the filtered image; S3. Based on all pixel points with a gradient amplitude greater than the amplitude threshold, construct a gradient direction point set. The gradient direction point set includes multiple pixel point coordinates and their corresponding gradient directions; S4. Based on the RANSAC algorithm, perform linear fitting on the gradient direction point set to obtain a straight line for characterizing the crop row direction. The weighted weight used by the RANSAC algorithm in the iterative process is determined by the gradient amplitude corresponding to the pixel point; S5. Control the agricultural robot to drive according to the straight line.

[0029] In step S1, the existing median filtering method can be used to filter the grayscale image to reduce the noise in the grayscale image and improve the image quality of the grayscale image.

[0030] Since the edge of the crop usually presents a gradient change, step S2 is equivalent to extracting the edge information with obvious grayscale value changes in the grayscale image by obtaining the gradient direction and gradient amplitude of each pixel in the filtered image, so as to extract the edge information of the crop row. Step S2 can use the existing Sobel operator to calculate the horizontal gradient component and the vertical gradient component of each pixel in the filtered image, and then obtain the gradient direction and gradient amplitude. It should be understood that there is an obvious grayscale difference between the crop row and the ridge. This grayscale difference is manifested as a gradient in the grayscale image. The gradient direction is always perpendicular to the equal grayscale line and points to the direction where the grayscale value increases fastest. The gradient amplitude characterizes the severity of the grayscale change. Even if the crop row image undergoes trapezoidal distortion, the gradient direction of the edge area of the crop row can still effectively indicate the extension direction of the crop row, and the gradient amplitude reflects the clarity of the edge area of the crop row.

[0031] The gradient amplitude of step S3 may be a value predetermined by those skilled in the art based on experience. Step S3 first uses the amplitude threshold to filter out pixels with large gradient amplitudes from the grayscale image, and then integrates all the filtered pixels into a gradient direction point set. It should be understood that pixels with gradient amplitudes greater than the amplitude threshold correspond to significant edges in the grayscale image, so compared with pixels with smaller gradient amplitudes, the pixels filtered out in step S3 are more likely to be components of the crop row.

[0032] The RANSAC algorithm of step S4 is used for straight line fitting. The RANSAC algorithm is used to robustly estimate the straight line representing the direction of the crop row from the gradient direction point set. Since the weighted weight used by the RANSAC algorithm in the iteration process is determined by the pixel gradient amplitude, this embodiment can make the pixel points with large gradient amplitude play a greater role in the RANSAC straight line fitting, so as to improve the fitting accuracy and the accuracy of the final straight line. Since the straight line obtained in step S4 can represent the direction of the crop row, step S5 can make the agricultural robot travel along the crop row by controlling the agricultural robot to travel according to the straight line, thereby realizing the automatic navigation of the agricultural robot.

[0033] An automatic navigation method for an agricultural robot provided by this application first obtains the gradient direction and gradient magnitude of each pixel point in the filtered image, then constructs a gradient direction point set based on all pixel points with a gradient magnitude greater than the magnitude threshold, and finally performs a line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a line for characterizing the crop row direction. Since the gradient direction can effectively indicate the extension direction of the crop row, the gradient magnitude can accurately reflect the clarity of the crop row edge region, and the weighted weight used in the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point, that is, this application can make pixel points with a large gradient magnitude have a greater impact on the line fitting result, so as to improve the fitting accuracy and the accuracy of the finally formed line. Therefore, even in the presence of trapezoidal distortion, the line obtained by this application can accurately reflect the crop row direction, that is, this application can achieve accurate recognition of the crop row in the presence of trapezoidal distortion, thereby effectively solving the problem that the recognition accuracy of the crop row is significantly reduced due to trapezoidal distortion, and further effectively improving the accuracy of the agricultural robot moving along the crop row, the navigation accuracy of the agricultural robot navigation method, and the operation quality of the agricultural robot.

[0034] In some preferred embodiments, step S4 includes: S41. Count the number of pixel points corresponding to each gradient direction in the gradient direction point set, and eliminate the gradient directions with the number of pixel points less than the number threshold to obtain a preprocessed gradient direction point set; S42. Randomly select at least two pixel points from the preprocessed gradient direction point set, and construct an initial line model based on the selected pixel points; S43. For each pixel point in the preprocessed gradient direction point set, obtain a weighted weight according to its corresponding gradient magnitude and the first preset conversion relationship, and calculate a weighted distance according to the distance from the pixel point to the initial line model and the weighted weight; S44. Divide all pixel points with a weighted distance less than the distance threshold into the same data set to obtain an inlier set, and calculate the sum of the gradient magnitudes corresponding to each pixel point in the inlier set to obtain the total inlier set gradient magnitude; S45. Analyze whether the number of iterations has reached the preset upper limit of the number of iterations. If so, select the initial line model with the largest total inlier set gradient magnitude as the line for characterizing the crop row direction. If not, return to step S42.

[0035] Step S41 of counting the number of pixel points corresponding to each gradient direction in the gradient direction point set is equivalent to constructing a gradient direction histogram. The horizontal axis of the gradient direction histogram represents the quantized gradient direction, and the vertical axis of the gradient direction histogram represents the number of pixel points corresponding to the gradient direction. The quantity threshold in step S41 can be determined according to the mean value of the number of pixel points in each gradient direction in the gradient direction histogram. The first preset conversion relationship in step S43 is preferably a pre-constructed mapping relationship between the gradient amplitude and the weighting weight. Specifically, pixel points with a large gradient amplitude are given a higher weighting weight, and pixel points with a small gradient amplitude are given a lower weighting weight, so that pixel points with a large gradient amplitude (equivalent to pixel points with a high edge intensity in the grayscale image) play a greater role in line fitting. The distance threshold in step S44 is used to determine whether a pixel point is an inlier. It should be understood that if the distance threshold is set too small, true inliers are likely to be misjudged as outliers, and if the distance threshold is set too large, outliers are likely to be misjudged as inliers. Therefore, the setting of the distance threshold needs to balance the accuracy and robustness of line fitting. Preferably, the distance threshold can be preset according to factors such as image resolution and sensor parameters. The gradient amplitudes of the inlier set obtained in step S44 are comprehensively used to evaluate the quality of the current line model. Specifically, the larger the sum of the gradient amplitudes of the inlier set, the better the current line model fits the pixel points with a large gradient amplitude in the gradient direction point set. Due to the influence of various factors, the crop edge information in the grayscale image collected by the vision sensor is unclear and there is a lot of noise. And this embodiment can eliminate the noise gradient directions with a small number of pixel points and highlight the main gradient directions of the crop row direction by eliminating the gradient directions with the number of pixel points less than the quantity threshold, so as to reduce the possibility of interference in the subsequent RANSAC algorithm. This embodiment can also make the RANSAC algorithm pay more attention to the pixels with a large gradient amplitude by calculating the weighted distance, and this embodiment can also ensure the robustness and accuracy of the RANSAC algorithm by calculating the sum of the gradient amplitudes of the inlier set to evaluate the line model quality and selecting the line model with the largest sum of the gradient amplitudes of the inlier set as the final result. Therefore, this embodiment can further improve the accuracy of the finally formed line, so that the line can more accurately reflect the crop row direction, thereby further improving the accuracy of the agricultural robot moving along the crop row and the navigation accuracy of the agricultural robot navigation method.

[0036] In some preferred embodiments, the vision sensor is further configured to collect color image information, and step S41 includes: S411. Construct a gradient direction histogram, where the horizontal axis of the gradient direction histogram is the quantized gradient direction, and the vertical axis of the gradient direction histogram is the number of pixel points corresponding to the gradient direction; S412. Perform Gaussian filtering and smoothing processing on the gradient direction histogram to reduce noise interference; S413. Calculate the mean value of the number of pixel points in each gradient direction in the gradient direction histogram after Gaussian filtering and smoothing to obtain the mean value of the number of pixel points. S414. Obtain the crop growth stage based on the color image information, determine the quantity adjustment coefficient according to the crop growth stage, and then multiply the quantity adjustment coefficient by the mean value of the number of pixel points to obtain the quantity threshold. S415. Eliminate the gradient directions with the number of pixel points less than the quantity threshold to obtain the preprocessed gradient direction point set.

[0037] Among them, the process of constructing the gradient direction histogram in step S411 can be: determine the quantized gradient direction range, such as 0 to 180 degrees, and divide this range into several intervals, each interval representing a gradient direction; traverse each pixel point in the gradient direction point set, determine the gradient direction interval to which the pixel point belongs according to the gradient direction of the pixel point, and accumulate the number of pixel points in the corresponding interval in the histogram. Step S414 can use existing image recognition technologies or image recognition models to obtain the crop growth stage based on the color image information. Specifically, the crop growth stage includes the seedling stage, the growth stage, and the maturity stage. Step S414 can obtain the quantity adjustment coefficient according to the crop growth stage and the pre-constructed mapping relationship between the crop growth stage and the adjustment coefficient. The quantity adjustment coefficient in step S414 can be preset as a value related to the crop growth stage. For example, a smaller quantity adjustment coefficient corresponds to the seedling stage, and a larger quantity adjustment coefficient corresponds to the maturity stage. Since the edge features of the same crop are different in different growth stages, and this embodiment is equivalent to flexibly adjusting the quantity threshold according to the crop growth stage to remove more noise points, this embodiment can effectively improve the fitting accuracy and accuracy of line fitting, thereby effectively improving the stability and reliability of crop row recognition in complex field environments.

[0038] In some preferred embodiments, a light intensity acquisition component is also installed on the agricultural robot, and step S414 includes: S4141. Obtain the crop growth stage based on the color image information and obtain the light intensity when the vision sensor acquires the grayscale image. S4142. Determine the quantity adjustment coefficient according to the crop growth stage and the light intensity. S4143. Multiply the quantity adjustment coefficient by the mean value of the number of pixel points to obtain the quantity threshold.

[0039] Step S4142 can obtain the clinker adjustment coefficient according to the crop growth stage, light intensity, and the pre-constructed mapping relationship between the crop growth stage, light intensity, and the adjustment coefficient. Since the change of field illumination will affect the clarity of the crop edge, and this embodiment is equivalent to comprehensively considering the light intensity and the crop growth stage when calculating the quantity threshold, this embodiment can effectively improve the accuracy of the quantity threshold, thereby effectively improving the fitting accuracy and accuracy of the straight line fitting.

[0040] In some preferred embodiments, step S4141 includes: S41411. Obtain the crop growth stage and crop type information according to the color image information, and obtain the light intensity when the visual sensor collects the grayscale image; Step S4142 includes: S41421. Obtain the quantity adjustment coefficient according to the crop growth stage, crop type information, light intensity, and the second preset conversion relationship. The second preset conversion relationship is a mapping relationship table pre-constructed to include the corresponding relationship between the crop growth stage, light intensity, crop type, and the direction adjustment coefficient.

[0041] Step S41411 can obtain the crop type information by using the existing image recognition technology to identify the crop type in the color image information. Since the edge features of different types of crops are different, and this embodiment comprehensively considers the influence of the crop growth stage, crop type, and light intensity when determining the quantity adjustment coefficient, this embodiment can make the setting of the quantity threshold more reasonable, so as to more effectively eliminate the noise gradient direction and retain the effective gradient direction representing the crop row direction, thereby further improving the fitting accuracy and accuracy of the straight line fitting.

[0042] In some preferred embodiments, step S41411 includes: S414111. Obtain the crop growth stage, crop type information, and weed occlusion ratio according to the color image information, and obtain the light intensity when the visual sensor collects the grayscale image; S414112. Correct the light intensity based on the weed occlusion ratio.

[0043] The weed occlusion ratio of this embodiment can be determined by an image segmentation algorithm. Specifically, this embodiment can first use the image segmentation algorithm to segment the crop and weed regions in the color image information, and then calculate the ratio of the weed region to the total crop region to obtain the weed occlusion ratio. The process of correcting the light intensity in step S414112 can be as follows: obtaining a light intensity adjustment coefficient according to the weed occlusion ratio and a pre-constructed mapping relationship between the occlusion ratio and the correction coefficient; correcting the light intensity by multiplying the light intensity by the light intensity adjustment coefficient. Since this embodiment can obtain the weed occlusion ratio and correct the light intensity based on the weed occlusion ratio, this embodiment can effectively avoid the situation where the actual light intensity received by the crop changes due to weed occlusion, and there is a deviation between the light intensity collected by the light intensity acquisition component and the actual light intensity received by the crop. Therefore, it can effectively avoid the situation where the accuracy of the quantity adjustment coefficient determined based on the light intensity collected by the light intensity acquisition component decreases due to the deviation between the light intensity collected by the light intensity acquisition component and the actual light intensity received by the crop.

[0044] In some preferred embodiments, step S2 includes: S21. Calculate the average gray value of the filtered image, and obtain a gradient amplitude adjustment coefficient according to the average gray value and a third preset conversion relationship; S22. Multiply the gradient amplitude adjustment coefficient by a preset gradient amplitude threshold to obtain an adaptive gradient amplitude threshold; S23. Construct a gradient direction point set based on all pixel points whose gradient amplitude is greater than the adaptive gradient amplitude threshold.

[0045] Step S21 can calculate the average gray value of the filtered image by first adding the gray values of all pixel points in the filtered image and then dividing the sum value by the total number of pixel points. This average gray value can reflect the overall brightness of the filtered image. The third preset conversion relationship in step S21 is preferably a pre-constructed mapping relationship between the gray value and the amplitude adjustment coefficient. Specifically, when the overall brightness of the image is low, the gradient amplitude adjustment coefficient decreases, so that the adaptive gradient amplitude threshold decreases and more potential crop row gradient points are retained. When the overall brightness of the image is high, the gradient amplitude adjustment coefficient increases, so that the adaptive gradient amplitude threshold increases, thereby suppressing noise interference and reducing false detection. This embodiment is equivalent to dynamically adjusting the gradient amplitude threshold according to the overall brightness of the filtered image. Therefore, this embodiment can effectively extract the gradient features of crop rows under different lighting conditions, thereby effectively improving the accuracy and robustness of line fitting.

[0046] In some preferred embodiments, step S1 includes: S11. Perform Gaussian filtering and smoothing on the grayscale image to obtain a filtered image.

[0047] This embodiment can effectively reduce the noise in the grayscale image by performing Gaussian filtering and smoothing on the grayscale image, making the grayscale image smoother.

[0048] In some preferred embodiments, step S5 includes: S51. Generate a movement path based on the straight line according to the path planning method, and control the agricultural robot to move according to the movement path.

[0049] In step S51, a series of path points can be first calculated based on the straight line according to the path planning method, and these path points together constitute the movement path of the agricultural robot, and then the agricultural robot is controlled to move along the movement path.

[0050] As can be seen from the above, an automatic navigation method for an agricultural robot provided by this application first obtains the gradient direction and gradient amplitude of each pixel point in the filtered image, then constructs a gradient direction point set based on all pixel points whose gradient amplitude is greater than the amplitude threshold, and finally performs linear fitting on the gradient direction point set based on the RANSAC algorithm to obtain a straight line for characterizing the crop row direction. Since the gradient direction can effectively indicate the extension direction of the crop row, the gradient amplitude can accurately reflect the clarity of the crop row edge area, and the weighted weight used in the iteration process of the RANSAC algorithm is determined by the gradient amplitude corresponding to the pixel point, that is, this application can make the pixel points with a large gradient amplitude have a greater impact on the linear fitting result to improve the fitting accuracy and the accuracy of the finally formed straight line. Therefore, even if there is trapezoidal distortion, the straight line obtained by this application can accurately reflect the crop row direction, that is, this application can achieve accurate recognition of the crop row in the case of trapezoidal distortion, thereby effectively solving the problem that the recognition accuracy of the crop row is significantly reduced due to trapezoidal distortion, and further effectively improving the accuracy of the agricultural robot moving along the crop row, the navigation accuracy of the agricultural robot navigation method, and the operation quality of the agricultural robot.

[0051] In a second aspect, as Figure 2 shown, this application also provides an automatic navigation device for an agricultural robot. The agricultural robot is applied in hilly and mountainous areas. A vision sensor for collecting grayscale images is installed on the agricultural robot. The automatic navigation device for the agricultural robot includes: A filtering module 1 for filtering the grayscale image to obtain a filtered image; A gradient calculation module 2 for calculating the horizontal direction gradient component and the vertical direction gradient component of each pixel point in the filtered image to obtain the gradient direction and gradient amplitude of each pixel point in the filtered image; The gradient direction point set construction module 3 is configured to construct a gradient direction point set based on all pixel points whose gradient magnitude is greater than the magnitude threshold. The gradient direction point set includes multiple pixel point coordinates and their corresponding gradient directions; The straight line generation module 4 is configured to perform straight line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a straight line for characterizing the crop row direction. The weighted weight used by the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point; The navigation module 5 is configured to control the agricultural robot to travel according to the straight line.

[0052] An automatic navigation device for an agricultural robot provided by this application includes a filtering module 1, a gradient calculation module 2, a gradient direction point set construction module 3, a straight line generation module 4, and a navigation module 5. The automatic navigation device for an agricultural robot provided by this embodiment is configured to execute an automatic navigation method for an agricultural robot provided in the first aspect above. The principle of the automatic navigation device for an agricultural robot provided by this embodiment is the same as the principle of the automatic navigation method for an agricultural robot provided in the first aspect above, and will not be elaborated in detail here.

[0053] As can be seen from the above, for an automatic navigation method and device for an agricultural robot provided by this application, first, the gradient direction and gradient magnitude of each pixel point in the filtered image are obtained, then a gradient direction point set is constructed based on all pixel points whose gradient magnitude is greater than the magnitude threshold, and finally, straight line fitting is performed on the gradient direction point set based on the RANSAC algorithm to obtain a straight line for characterizing the crop row direction. Since the gradient direction can effectively indicate the extension direction of the crop row, the gradient magnitude can accurately reflect the clarity of the crop row edge region, and the weighted weight used by the RANSAC algorithm during the iteration process is determined by the gradient magnitude corresponding to the pixel point, that is, this application can make pixel points with a large gradient magnitude have a greater impact on the straight line fitting result to improve the fitting accuracy and the accuracy of the finally formed straight line. Therefore, even in the presence of trapezoidal distortion, the straight line obtained by this application can accurately reflect the crop row direction, that is, this application can achieve accurate recognition of the crop row in the presence of trapezoidal distortion, thereby effectively solving the problem that the recognition accuracy of the crop row is significantly reduced due to trapezoidal distortion, and further effectively improving the accuracy of the agricultural robot traveling along the crop row, the navigation accuracy of the agricultural robot navigation method, and the operation quality of the agricultural robot.

[0054] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another robot, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0055] In addition, in each embodiment of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0056] In this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0057] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An automatic navigation method for an agricultural robot, the agricultural robot is applied in hilly and mountainous areas, and a vision sensor for collecting grayscale images is installed on the agricultural robot, characterized in that, The automatic navigation method for an agricultural robot comprises the following steps: S1, filtering the grayscale image to obtain a filtered image; S2, calculating the horizontal gradient component and the vertical gradient component of each pixel in the filtered image to obtain the gradient direction and gradient amplitude of each pixel in the filtered image; S3, constructing a gradient direction point set based on all pixel points whose gradient amplitude is greater than the amplitude threshold, wherein the gradient direction point set includes multiple pixel point coordinates and their corresponding gradient directions; S4, performing straight line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a straight line used to characterize the direction of the crop row, wherein the weight used by the RANSAC algorithm in the iteration process is determined by the gradient amplitude corresponding to the pixel point; S5. Control the agricultural robot to move according to the straight line.

2. The automatic navigation method for an agricultural robot according to claim 1, characterized in that, Step S4 includes: S41, counting the number of pixel points corresponding to each gradient direction in the gradient direction point set, and eliminating the gradient direction whose number of pixel points is less than the number threshold, so as to obtain a preprocessed gradient direction point set; S42, randomly selecting at least two pixel points from the preprocessed gradient direction point set, and constructing an initial straight line model based on the selected pixel points; S43, for each pixel point in the preprocessing gradient direction point set, obtaining a weighted weight according to its corresponding gradient amplitude and the first preset conversion relationship, and calculating a weighted distance according to the distance from the pixel point to the initial straight line model and the weighted weight; S44, dividing all pixel points whose weighted distance is less than the distance threshold into the same data set to obtain an inlier set, and calculating the sum of the gradient amplitudes corresponding to each pixel point in the inlier set to obtain the sum of the gradient amplitudes of the inlier set; S45, analyzing whether the number of iterations reaches the preset upper limit of the number of iterations. If so, selecting the initial straight line model with the largest sum of the gradient amplitudes of the internal point set as the straight line for characterizing the direction of the crop row. If not, returning to step S42.

3. The automatic navigation method for an agricultural robot according to claim 1, characterized in that The visual sensor is also used to collect color image information. Step S41 includes: S411, constructing a gradient direction histogram, wherein the horizontal axis of the gradient direction histogram is the quantized gradient direction, and the vertical axis of the gradient direction histogram is the number of pixels corresponding to the gradient direction; S412, performing Gaussian filtering and smoothing processing on the gradient direction histogram to reduce noise interference; S413, calculating the mean value of the number of pixels in each gradient direction in the gradient direction histogram after Gaussian filtering and smoothing, to obtain the mean value of the number of pixels; S414, obtaining the crop growth stage according to the color image information, and determining the quantity adjustment coefficient according to the crop growth stage, and then multiplying the quantity adjustment coefficient by the mean value of the number of pixel points to obtain a quantity threshold; S415 , eliminating the gradient directions whose number of pixel points is less than the number threshold, to obtain a preprocessed gradient direction point set.

4. The automatic navigation method for an agricultural robot according to claim 3, characterized in that, The agricultural robot is also equipped with a light intensity collection component. Step S414 includes: S4141. Acquire the crop growth stage according to the color image information, and acquire the light intensity when the visual sensor acquires the grayscale image; S4142. Determine the quantity adjustment coefficient according to the crop growth stage and light intensity; S4143. Multiply the quantity adjustment coefficient by the mean value of the number of pixels to obtain a quantity threshold.

5. The automatic navigation method for an agricultural robot according to claim 4, wherein Step S4141 includes: S41411. Obtain crop growth stage and crop type information according to the color image information, and obtain the light intensity when the vision sensor collects the grayscale image; Step S4142 includes: S41421. Obtain the quantity adjustment coefficient according to the crop growth stage, crop type information, light intensity and the second preset conversion relationship. The second preset conversion relationship is a mapping relationship table pre-constructed to include the corresponding relationship between crop growth stage, light intensity, crop type and direction adjustment coefficient.

6. The automatic navigation method for an agricultural robot according to claim 5, wherein, Step S41411 includes: S414111. Obtain crop growth stage, crop type information and weed occlusion ratio according to the color image information, and obtain the light intensity when the vision sensor collects the grayscale image; S414112. Correct the light intensity based on the weed occlusion ratio.

7. The automatic navigation method for an agricultural robot according to claim 1, characterized in that, Step S2 includes: S21. Calculate the average grayscale value of the filtered image, and obtain the gradient amplitude adjustment coefficient according to the average grayscale value and the third preset conversion relationship; S22. Multiply the gradient amplitude adjustment coefficient by the preset gradient amplitude threshold to obtain the adaptive gradient amplitude threshold; S23. Construct a gradient direction point set based on all pixel points whose gradient amplitude is greater than the adaptive gradient amplitude threshold.

8. The automatic navigation method for an agricultural robot according to claim 1, wherein Step S1 includes: S11. Perform Gaussian filtering and smoothing on the grayscale image to obtain a filtered image.

9. The automatic navigation method for an agricultural robot according to claim 1, characterized in that, Step S5 includes: S51. Generate a moving path based on the straight line according to the path planning method, and control the movement of the agricultural robot according to the moving path.

10. An automatic navigation device for an agricultural robot, the agricultural robot is applied in hilly and mountainous areas, and a vision sensor for collecting grayscale images is installed on the agricultural robot, characterized in that, An automatic navigation device for an agricultural robot includes: A filtering module for filtering the grayscale image to obtain a filtered image; A gradient calculation module for calculating the horizontal direction gradient component and the vertical direction gradient component of each pixel point in the filtered image to obtain the gradient direction and gradient amplitude of each pixel point in the filtered image; A gradient direction point set construction module for constructing a gradient direction point set based on all pixel points whose gradient amplitude is greater than the amplitude threshold. The gradient direction point set includes multiple pixel point coordinates and their corresponding gradient directions; A straight line generation module for performing straight line fitting on the gradient direction point set based on the RANSAC algorithm to obtain a straight line for characterizing the crop row direction. The weighted weight used by the RANSAC algorithm in the iteration process is determined by the gradient amplitude corresponding to the pixel point; A navigation module for controlling the movement of the agricultural robot according to the straight line.

Images