Crop navigation line extraction method in high ridge cultivation environment based on machine vision

Abstract

A crop navigation line extraction method for a high-ridge cultivation environment of plants based on machine vision, the camera acquires an image in RGB color space, and converts the image color space into CIE‑Lab color space. The a component of the CIE‑Lab color space is extracted to achieve preliminary extraction of crop row features, and OTSU and morphological processing are combined to achieve complete separation of crop rows and background. The equidistant segmented vertical projection method is used to extract crop row feature points. The adaptive clustering method is used to cluster adjacent feature points, and the dynamic segmentation point clustering method is used to determine the final clustered feature point set, and the lateral distance optimization method and the point-line distance optimization method are used to optimize the feature point set. During the optimization process, a linear regression method based on the Huber loss function is used for fitting, and the navigation line is calculated according to the crop row line. Finally, a dynamic segmentation horizontal strip number feedback mechanism is introduced to improve the multi-period adaptability of the algorithm. The image processing process of the present invention takes an average of 38.53ms per frame, which can meet the efficiency requirements of visual navigation.

Description

Technical Field

[0001] The present invention relates to the field of agricultural engineering, in particular to the extraction of navigation lines for farmland machinery in automated intensive farming. Background Art

[0002] The concept of precision agriculture has gained widespread international recognition and has become a research hotspot for researchers worldwide. Automatic navigation of agricultural machinery is a key technology in precision agriculture applications and forms the foundation for its implementation. Machine vision (MV) navigation is a key technology for automatic navigation of agricultural machinery. MV navigation accurately detects guidance lines within images and uses these lines to automatically guide the machine. Image segmentation, feature point extraction, and guidance line calculation are three key steps in crop row centerline extraction. Scholars both domestically and internationally are conducting in-depth research in this area. et al. divided a grayscale image into 15 horizontal strips and then calculated the vertical sum of the grayscale values of each strip, with the maximum value representing the center of each crop row. Chen Ziwen et al. segmented the image into multiple horizontal strips and used a dual-threshold vertical projection method to extract feature points from lettuce and cabbage crop rows. By setting a target pixel ratio threshold and a noise threshold in the brightness projection view, they determined the location of feature points and filtered out weed noise. Zhai Zhiqiang et al. used a similarity function to calculate the similarity between the center pixel of a mask and all pixels within the mask area to obtain initial corner points. They then performed non-maximum suppression on these initial corner points to obtain a cluster of target cotton crop row feature points. Pla et al. applied a vanishing point detection algorithm to skeleton segment images to obtain points defining all lines that define crop and row edges. Diao Zhihua marked candidate skeleton points that met the maximum square criterion in binary images of leeks and corn. Then, using a scanning operation, they searched for the target skeleton points closest to the center of the traversed area, using these marked skeleton points as target features. Fontaine and Crowe identified and described adjacent pixel regions of the same value within a binary wheat image. With an appropriately sized window, they were able to effectively locate feature points in low-lying crop rows. Jiang et al. used a moving window to scan early wheat images row by row, obtaining crop row feature points at the center of the image.

[0003] Currently, domestic crop row or navigation line extraction algorithms are primarily targeted at environments with no or low ridges, such as wheat and corn. Broadleaf crops cultivated in high-ridge environments have unique characteristics. For convenience, this high-ridge environment will be referred to as the high-ridge environment below. Broadleaf crops experience significant plant size variations corresponding to different field growth days after transplanting. This is primarily reflected in images as plants within a row and between rows gradually connect as the crop grows. High-ridge environments are susceptible to lighting conditions, producing numerous shadows. Most researchers have only studied and analyzed cases where plants within a row are connected or not connected within the same period. Currently, no unified detection algorithm for high-ridge broadleaf crop rows across multiple periods has been developed. Furthermore, in practice, the machine can only rely on a single navigation line. Some researchers aim to detect multiple crop rows within an image. Such algorithms subsequently require the calculation of navigation lines based on these multiple crop rows. Although some researchers have proposed improving algorithm timeliness by detecting only two crop rows before calculating navigation lines, the algorithm's adaptability to diverse environments requires further enhancement. Finally, the number and height values of horizontal strips divided by the vertical projection method proposed by most scholars are fixed values, which generally cannot meet the requirements of multi-period detection. At present, no scholar has proposed a solution to realize automatic adjustment of this value. Summary of the Invention

[0004] While expanding the advantages of the above-mentioned existing methods, the present invention proposes a new crop row and navigation line extraction method. While ensuring timeliness, this method focuses on improving the adaptability of the algorithm to the high ridge environment of broad-leaved crops at different times.

[0005] This method comprises four sequential modules: image segmentation, feature point extraction, navigation line calculation, and a dynamic segmentation horizontal stripe number feedback mechanism. For image segmentation, the illumination-independent α component of the CIE-Lab color space is first used to extract green crop features, and crop-background separation is achieved by combining OTSU and morphological processing methods. For feature point extraction, an improved equidistant segmented vertical projection method is used to extract crop row feature points. Then, an adaptive clustering method and a dynamic segmentation point clustering method are used to determine the final feature point set, which is then optimized using lateral distance optimization and point-line distance optimization. During the point-line distance optimization process, a linear regression method based on the Huber loss function is used to fit the crop rows. Finally, a dynamic segmentation horizontal stripe number feedback mechanism is introduced to improve the algorithm's adaptability across multiple crop growth periods. Therefore, the main contribution of this invention is to propose a unified method for extracting navigation lines for broad-leaved crops across multiple growth periods, combining the adaptive clustering method with the dynamic segmentation point clustering method. Furthermore, a feedback mechanism is designed and applied to automate the calculation of the number of segmentation horizontal stripes. Furthermore, a linear regression method using the Huber loss function is used to further enhance the robustness of the method.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A crop navigation line extraction method for a high ridge cultivation environment based on machine vision includes the following steps:

[0008] S1: Use a color camera to obtain RGB images of farmland;

[0009] S2: Convert the image color space from RGB color space to CIE-Lab color space;

[0010] S3: extracting the a component of the CIE-Lab color space to preliminarily extract crop row features and grayscale the acquired color image;

[0011] S4: performing binarization processing on the grayscale image to obtain a binary image;

[0012] S5: filtering out noise from the binary image;

[0013] S6: Extract crop row feature points using the improved equidistant segmented vertical projection method;

[0014] S7: using the adaptive clustering method to cluster adjacent feature points;

[0015] S8: Using the dynamic segmentation point clustering method to determine the final clustering feature point set;

[0016] S9: The feature point set is optimized using the lateral distance optimization method and the point-line distance optimization method. During the optimization process, the linear regression method based on the Huber loss function is used to fit the optimized feature point set.

[0017] S10: Calculating a navigation line based on crop rows;

[0018] S11: Introduce a dynamic segmentation level strip number feedback mechanism to improve the algorithm's multi-period adaptability.

[0019] In the step S5 , a closing operation is first performed on the binarized image, and then an opening operation is performed on the processed image, so as to filter out noise.

[0020] Step S6 includes the following steps:

[0021] (1) Calculate the number of horizontal stripes N based on the dynamic segmentation horizontal stripe number feedback mechanism D ;

[0022] (2) According to the number of horizontal strips N D Calculate the horizontal strip height D;

[0023] (3) Divide N into binary images DHorizontal strips, number all the divided horizontal strips in order from bottom to top, initialize x = N D -1, in the x-th horizontal strip, find the point where f(i, j) = 255, and project the points where f(i, j) = 255 in the same column onto the column coordinate axis u to obtain a vertical projection statistical graph;

[0024] (4) Calculate the vertical projection statistics of the x-th horizontal strip, which is recorded as the x-th vertical projection statistics. Let p(i) be the number of pixels with a pixel value of 255 in the i-th column of the binary horizontal strip. The calculation formula of p(i) is shown in formula (8), where i∈{0,1,2,…,u}; x∈{N D -1,…,2,1,0}.

[0025]

[0026] (5) Calculate the average projection value avgp(x) of p(i) in all columns of the x-th vertical projection statistical graph; identify the larger and higher peak area as the crop characteristic area, and simplify the crop characteristic area into a point, which is the crop characteristic point; the calculation formula of avgp(x) is:

[0027]

[0028] (6) Using the method of determining the vertical coordinate value of the feature point based on the area threshold, the vertical coordinate of the horizontal line of the feature point is determined as m(x).

[0029]

[0030] Yx=UppB+m(x) (4)

[0031] (7) Compare the magnitude relationship between p(i) and m(x);

[0032]

[0033] (8) The calculation formula of the horizontal coordinate of the feature point is as follows:

[0034]

[0035] where p_mid c represents the horizontal coordinate of the feature point numbered c in the xth horizontal strip; u1 is the starting point of the continuous data set; u2 is the ending point of the continuous data set; n u is the number of data in the continuous data set; n k =u2-u1;

[0036] (9) Output feature point set SET_FP = {(p_mid0, Y x),(p_mid1,Y x ),(p_mid2,Y x ),…,(p_mid Q-1 ,Y x )}, Q is the number of feature points in the horizontal strip;

[0037] (10) Let x = x-1, and repeat steps (4) to (9) until x < 0, at which point the module program stops running.

[0038] In the step S7, the adaptive clustering method is used to cluster multiple feature points of the same crop into one feature point. The number of feature points of the Kth horizontal strip is Q. K , P K,m (m=0,1,2,…n K ) represents the mth feature point in the Kth horizontal strip, then the distance between the mth feature point and the m+1th feature point in the Kth horizontal strip can be expressed as d K,m,(m+1) , where m=0,1,2,…n K -1, the average distance between all adjacent feature points in the same horizontal strip can be expressed as d K,avg ; Set a clustering threshold th div The distance d between each adjacent feature point K,m,(m+1) In comparison, for the feature points in the Kth horizontal strip, when d K,m,(m+1) >th div When , record the m value at this time, and divide the feature points into several feature point sets according to all the recorded m values; div and d K,avg The calculation formula is as follows:

[0039]

[0040] Among them F K is the F value of the Kth horizontal strip, F K+1 is the F value of the K+1th horizontal strip, and the F value setting value is a dynamic changing value.

[0041] In said S8, a dynamic segmentation point clustering method is used to assign the cluster feature points to corresponding feature point sets; the segmentation midpoint p used is div,K The calculation needs to comprehensively consider the distribution of existing segmentation midpoints.

[0042] In the above S8 , only two cluster feature points on the left and right sides of the segmentation midpoint are extracted.

[0043] In the step S9, the lateral distance optimization is performed by comparing the horizontal coordinate difference A with the distance threshold th hori The size relationship is used to determine whether to remove it.

[0044] In the S9, the point-line distance optimization based on the Huber loss function is performed by comparing the distance from the point to the first fitted straight line and the distance threshold th hori The size relationship is used to determine whether to remove it.

[0045] In the above-mentioned S9, the point-line distance optimization of the Huber loss function adopts iterative weighted least squares to estimate the regression coefficient, and the weight of each sample is determined according to the size of the regression residual.

[0046] In S10, the navigation line is defined as the angle bisector of the angle formed by two crop row straight lines at two center positions. The angle bisector is determined based on the relationship between the straight lines and the axes of the coordinate system.

[0047] The present invention has the following beneficial effects: it proposes a unified method for extracting multi-period navigation lines for broad-leaved trees by combining adaptive clustering with dynamic segmentation point clustering. It also designs and applies a feedback mechanism to automate the calculation of the number of segmented horizontal stripes. Furthermore, a linear regression method using the Huber loss function is employed to further enhance the robustness of the method. The image processing process of the present invention takes an average of 38.53ms per frame, meeting the efficiency requirements of visual navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is the original image of the cloudy environment of the present invention;

[0049] Figure 2(a) is a grayscale image of the a component in a cloudy environment, Figure 2(b) is a histogram of the a component in a cloudy environment, Figure 2(c) is a binary image in a cloudy environment, and Figure 2(d) is an image obtained by morphological processing in a cloudy environment.

[0050] Figure 3 This is the feature point extraction process in the horizontal strip, where (a) is the strip extracted from the original image, (b) is the strip strip from the binary image, (c) is the vertical projection curve, (d) is the vertical projection curve after thresholding, and (e) is the result of locating the feature points. Note: W is the image width, pixel; H is the image height, pixel; ΔH is the image strip height, pixel;

[0051] Figure 4 This is a description of the feature point clustering process;

[0052] Figure 5(a) shows the horizontal strip division and feature point extraction, and Figure 5(b) shows the feature point clustering;

[0053] Figure 6 It is the algorithm flow chart;

[0054] Figures 7(a) to 7(d) are the results of field image navigation line detection in three growth periods and two lighting environments: Figure 7(a) is the 3rd day after crop transplanting, in a cloudy environment (sample A), Figure 7(b) is the 18th day after crop transplanting, in a cloudy environment (sample B), Figure 7(c) is the 30th day after crop transplanting, in a cloudy environment (sample C), and Figure 7(d) is the 19th day after crop transplanting, in a sunny environment (sample D). DETAILED DESCRIPTION

[0055] like Figure 1 As shown, the navigation line extraction algorithm of the present invention has the following steps:

[0056] S1: Use a color camera to obtain RGB images of farmland;

[0057] Videos of four different field environments of green broadleaf plants were filmed. To improve the efficiency of obtaining image samples, image samples were obtained by extracting frames from the video, extracting an image every 30 frames. The extracted images were processed using a resampling method. The final image resolution was 960 pixels × 544 pixels, and the image information was stored in the RGB color space in the JPG file format with a bit depth of 24. An image was randomly extracted from the video and used to illustrate the processing method. The original image of the cloudy environment extracted is shown in the figure below. Figure 1 shown.

[0058] S2: Convert the image color space from RGB color space to CIE-Lab color space;

[0059] Of the three components in the CIE-Lab color space, the a component is sensitive to green. Convert the image color space from RGB to CIE-Lab. Refer to the calculation method for COLOR_BGR2Lab in the OpenCV library documentation for the conversion method.

[0060] S3: extracting the a component of the CIE-Lab color space to preliminarily extract crop row features and grayscale the acquired color image;

[0061] The a component of the CIE-Lab color space is separated. The a component value range is quantized to [0, 255]. The extracted a component image is a grayscale image, as shown in Figure 2(a). As shown in the histogram in Figure 2(b), the a component histogram of the CIE-Lab color space has two obvious peaks and a narrow valley. The histogram is bimodal, and the histogram distribution conditions are conducive to binarization.

[0062] S4: performing binarization processing on the grayscale image to obtain a binary image;

[0063] The grayscale image is binarized using the OTSU (maximum between-class variance) algorithm, and the resulting binary image is shown in Figure 2(c);

[0064] S5: First, perform a closing operation on the binarized image, and then perform an opening operation on the processed image to filter out noise;

[0065] The morphological opening operation can eliminate isolated, fine noise in an image, while the morphological closing operation can effectively fill holes. Leveraging their respective strengths, we first perform the closing operation on the image, followed by the opening operation on the processed image. These operations effectively remove noise and fill holes. Compared to the binary image in Figure 2(c), the morphological image in Figure 2(d) shows a significant reduction in noise and holes in the white areas.

[0066] S6: Extract crop row feature points using the improved equidistant segmented vertical projection method;

[0067] (1) Calculate the number of horizontal stripes N based on the dynamic segmentation horizontal stripe number feedback mechanism D . After equidistant segmentation from bottom to top, we get N D horizontal strips, N D The value of is calculated based on the dynamic segmentation horizontal strip quantity feedback mechanism, and the mechanism rules are shown in step S11.

[0068] (2) Remove the top area of 100pix height and calculate the number of horizontal strips N. D Calculate the horizontal strip height D.

[0069] (3) Divide N into binary images D Horizontal strips are numbered in order from bottom to top, for example, the Nth D -1 segment, ..., segment 2, segment 1, segment 0. Initialize x = N D -1, in the xth horizontal strip, find the point where f(i,j)=255, and project the points where f(i,j)=255 in the same column onto the column coordinate axis u to obtain a vertical projection statistical graph, such as Figure 3 (c) shown.

[0070] (4) Calculate the vertical projection statistics of the x-th horizontal strip, and record it as the x-th vertical projection statistics. Initialize x = N D -1, in the xth horizontal strip, find the point where f(i,j)=255, and project the points where f(i,j)=255 in the same column onto the column coordinate axis u to obtain a vertical projection statistical graph, such as Figure 3(c) Assume that p(i) is the number of pixels with a pixel value of 255 in the binary horizontal strip located in the i-th column. The calculation formula of p(i) is shown in formula (8), where i∈{0,1,2,…,u}; x∈{N D -1,…,2,1,0}.

[0071]

[0072] (5) Calculate the average projection value avgp(x) of p(i) in all columns of the x-th vertical projection statistical graph. Identify the larger and higher peak area as the crop characteristic area, and simplify the crop characteristic area into a point, which is the crop characteristic point. The calculation formula of avgp(x) is:

[0073]

[0074] (6) Calculate P according to the following definition S and th s After calculating m(x) according to formula (10), the vertical coordinate value Y of each feature point in the x-th horizontal strip can be calculated according to formula (11): x Image area threshold th s , this value is determined by experiment, the target pixel ratio P S It is the area ratio of the number of pixels with grayscale value of 255 in the binary horizontal strip to the total number of pixels in the binary horizontal strip. In the x-th vertical projection statistical chart, set the horizontal line through the vertical projection, and the vertical coordinate of the horizontal line of the feature point is m(x), such as Figure 3 (c) is shown. When crops are in the seedling stage, the crop leaves are small and there are relatively few weeds. Therefore, taking an appropriate m(x) can effectively avoid the influence of multimodal peaks on feature point extraction. When the crops are in the mature stage, taking a larger m(x) can filter out the interference caused by crop branches and weeds. Step (6) is called the method for determining the vertical coordinate value of the feature point based on the area threshold. The calculation method of m(x) is as follows:

[0075]

[0076] Yx=UppB+m(x) (11)

[0077] (7) Compare the size relationship between p(i) and m(x), simplify the x-th vertical projection statistical graph according to formula (12), and obtain a binary statistical graph, as shown in Figure 3 (d).

[0078]

[0079] (8) Calculate the horizontal coordinate value X of each feature point in the xth horizontal strip xExtract each data set with a continuous B(i) value of 1, and take the average value of the horizontal coordinate u corresponding to the continuous data set as the feature point position. Use the noise judgment threshold th n As the minimum width judgment index for identifying continuous data sets, it is used to filter out weeds and branches and leaves noise interference. Noise judgment threshold th n Determined by experiment. When the continuous data set is less than the noise judgment threshold th n When , this continuous data set is determined to be noise interference and discarded. The formula for calculating the horizontal coordinate of the feature point is as follows:

[0080]

[0081] where p_mid c represents the horizontal coordinate of the feature point numbered c in the xth horizontal strip; u1 is the starting point of the continuous data set; u2 is the ending point of the continuous data set; n u is the number of data in the continuous data set; n k =u2-u1. The final feature point is Figure 3 (d) The feature points in the original image are as follows Figure 3 (e) The distribution is shown in Figure 5. Steps (6) to (8) are actually the process of determining the horizontal coordinates of the feature points. This method is called the feature point extraction method based on the noise judgment threshold.

[0082] (9) Output feature point set SET_FP = {(p_mid0, Y x ),(p_mid1,Y x ),(p_mid2,Y x ),…,(p_mid Q-1 ,Y x )}, Q is the number of feature points in this horizontal strip.

[0083] (10) Let x = x-1, and repeat steps (4) to (9) until x < 0, at which point the module program stops running.

[0084] According to a large number of experiments, the image area threshold th s =35%, noise judgment threshold th n =5, a better feature point extraction effect can be obtained.

[0085] S7: using the adaptive clustering method to cluster adjacent feature points;

[0086] An adaptive clustering method is used to cluster multiple feature points of the same crop into one feature point. The following is a description of this method.

[0087] During the clustering process, the horizontal strips formed in the feature point extraction step are used as units for traversal. Traverse the horizontal strips from bottom to top, and traverse each feature point from left to right in the horizontal strips. Let the number of feature points in the Kth horizontal strip be Q K , P K,m (m=0,1,2,…n K ) represents the mth feature point in the Kth horizontal strip, then the distance between the mth feature point and the m+1th feature point in the Kth horizontal strip can be expressed as d K,m,(m+1) , where m=0,1,2,…n K -1. The average distance between all adjacent feature points in the same horizontal strip can be expressed as d K,avg , as shown in formula (14). Figure 4 As shown, set a clustering threshold th div The distance d between each adjacent feature point K,m,(m+1) In comparison, for the feature points in the Kth horizontal strip, when d K,m,(m+1) >th div When , record the m value at this time. According to all the recorded m values, the feature points can be divided into several feature point sets. div and d K,avg The calculation formula is as follows:

[0088]

[0089] Because the image crop rows are arranged with vanishing points, the average distance between adjacent feature points in the horizontal strip at the top of the image is smaller than the average distance between adjacent feature points in the horizontal strip at the bottom of the image. A fixed F value cannot adapt to every horizontal strip. Therefore, it is necessary to set the F value to a dynamic value. After a lot of experiments, the following empirical calculation formula for the F value was obtained:

[0090] F K =F K+1 -0.04 (15)

[0091] Among them F K is the F value of the Kth horizontal strip, F K+1 is the F value of the K+1th horizontal strip.

[0092] S8: Using the dynamic segmentation point clustering method to determine the final clustering feature point set;

[0093] The dynamic segmentation point clustering method is used to assign the cluster feature points to the corresponding feature point set. Introducing a segmentation midpoint p div , according to the split midpoint p div The adjacent feature point sets on the left and right sides can be obtained ( Figure 4The left feature point set is recorded as Cleft, and the right feature point set is recorded as Cright. In the 0th horizontal strip, p div,0 The value is the integer of W / 2. From the first horizontal strip to the last horizontal strip, the p of the current horizontal strip is div is the p of all previous horizontal strips div The point average of the established set, the midpoint p of the Kth horizontal strip div,K The calculation formula (16) is as follows:

[0094]

[0095] Calculate the feature point set ( Figure 4 The number of feature points Nc in A, B, C, D) is Ncleft, the number of points in the left feature point set Cleft, and Ncright the number of points in the right feature point set Cright. The feature points of the set Cleft and the set Cright are clustered according to the following rules:

[0096] When Nc=1, there is only one feature point set, and the cluster point of this set is the feature point in the set.

[0097] When Nc = 2, calculate the distances between all adjacent points within the feature point set. If the distance between two points within the feature point set is less than or equal to W / 5, the abscissa of the cluster point is the average of the abscissas of the two points within the set. If the distance between two points within the feature point set is greater than W / 5, the two points may belong to two different crop rows. For the left feature point set, the abscissa of the cluster point is the larger of the two points within the set; for the right feature point set, the abscissa of the cluster point is the smaller of the two points within the set.

[0098] When Nc>2, the horizontal coordinate of the cluster point is the average of the horizontal coordinates of all points in the set.

[0099] The vertical coordinates of the above cluster points are consistent with the vertical coordinates of the original feature points. Finally, the cluster feature points of the two crop rows near the center position in the horizontal strip can be obtained, such as Figure 4 Points II and III. Each horizontal strip is extracted according to the above rules, and the original feature point extraction results are shown in Figure 5. The left and right cluster points are respectively assigned to the cluster feature point sets SetFit_Left and SetFit_Right. The clustering results and classification results are shown in Figure 5(b).

[0100] S9: Then use the horizontal distance optimization method and the point-line distance optimization method to optimize the feature point set. During the optimization process, the linear regression method based on the Huber loss function is used to fit the optimized feature point set.

[0101] After determining the clustered feature points for the left and right crop rows, the problem is now simplified to finding the corresponding straight line equation for each crop row. Although the three steps described above—feature point extraction, morphological processing, and clustering—can reduce various types of interference to a certain extent, their ability to resist interference needs to be further improved. At this point, the clustered feature point set may still contain interfering points. To further enhance the robustness of the algorithm, the clustered feature points need to be further optimized. This method completes the straight line fitting step during the optimization of the clustered feature points.

[0102] (1) Horizontal distance optimization

[0103] In the pixel coordinate system, the main feature of the interference point is that its horizontal coordinate deviates from the horizontal coordinate of most feature point groups, so the interference points can be eliminated based on this feature. Since the two left and right cluster feature point sets (SetFit_Left, SetFit_Right) are generated from bottom to top based on the vertical projection method, the vertical coordinates of each point in the cluster feature point set are always larger than the next point in order. Calculate the two adjacent points P in the cluster feature point set in sequence. m and P m+1 The horizontal coordinate difference A of the distance threshold th can be calculated based on the average value avg(A) of all differences hori Compare all differences A and the distance threshold th hori The size relationship, when A>th hori When point P is eliminated m+1 ; Otherwise, click P m+1 Continue to retain the cluster feature point set, P m+1 Assign to P m Then continue to iterate and calculate in the above way. After optimizing the left and right cluster feature point sets according to the above rules, we get the left and right horizontal distance optimized point sets (Set_optimized_Left, Set_optimized_Right). Distance threshold th hori The calculation method is shown in Formula 12.

[0104] th hori =1.1*avg(A) (17)

[0105] (2) Point-line distance optimization based on Huber loss function

[0106] The steps are as follows: (1) Perform linear fitting on the left horizontal distance optimization point set Set_optimized_Left to obtain the first fitting straight line L1 = k1x + b1; (2) Introduce the distance threshold th dis, traverse and calculate the distance between all points in the set Set_optimized_Left and the straight line L1, when the distance is greater than the distance threshold th dis When , the point is removed; otherwise, the point continues to be retained in the set Set_optimized_Left. dis The calculation is shown in formula (13), and the processing method of the right horizontal distance optimization point set Set_optimized_Right is the same.

[0107]

[0108] m is the integer index of the point in Set_optimized_Left, ranging from [0, m]. i is the abscissa of the point, and j is the ordinate of the point. avg(G) is the arithmetic average of the distances from all points P(i, j) in Set_optimized_Left to the first fitted line L1. A second linear fit is performed on the data, excluding outliers, to obtain the optimized line L2. Line L2 is the centerline of the crop row. Both fits use the same linear fitting method.

[0109] Many studies have chosen LSM as a method for linear fitting. However, LSM has limitations: First, since LSM gives the same weight to each point in the fitting process, it is only applicable to situations with high correlation. LSM is not applicable to situations where the data is discrete and there are singular values. Second, since LSM solves the residual sum of squares to achieve the minimum regression coefficient, it is easy to exaggerate the influence of singular values in the experimental data and increase statistical errors. Therefore, LSM lacks robustness. The basic idea of M-estimation robust regression is to use iterative weighted least squares to estimate the regression coefficient and determine the weight of each sample according to the size of the regression residual to achieve robustness. ρ(r m ) is the deviation function, and the cost function cost(L) of Huber-M estimation is defined as:

[0110]

[0111] The k and b calculated under the condition that the straight line L is a fitting straight line make cost(L) take the minimum value. ρ(r m ) is the Huber function, and its expression is:

[0112]

[0113] C is a tuning parameter. In OpenCV, the fitLine function can be used to fit a point set to a straight line, which can automatically adjust to the optimal C value. Take the distance type as Cv.DIST_HUBER, the distance parameter as 0, the radial accuracy as 0.01, and the angular accuracy as 0.01. The output is four parameters: sinα, cosα, x0, and y0. Among them, sinα and cosα are the sine and cosine values of the inclination angle α of the fitted line, respectively, and x0 and y0 are the horizontal and vertical coordinate values of the point (x0, y0) through which the fitted line passes. Therefore, the equation of the fitted line L can be expressed as:

[0114]

[0115] S10: Calculating a navigation line based on crop rows;

[0116] After obtaining the equations of the fitting lines of the left and right ridges, the angle bisector equation of the angle between the fitting lines can be obtained based on the equations of the two straight lines, which is the navigation reference line. The formula for the slope of the angle bisector is:

[0117]

[0118] k is the slope of the navigation line; k1 is the slope of the left ridge fitting line; k2 is the slope of the right ridge fitting line.

[0119] Then find the intersection of the left and right ridge fitting line equations to get the angle bisector equation. Simplifying formula (22) we can get:

[0120] (k1+k2)k 2 +2(1-k1k2)k-(k1+k2)=0 (23)

[0121] When equation (18) has a solution, the solution is the slope of the guide line. According to the relationship between roots and coefficients in a quadratic equation, when the equation has two solutions S1 and S2:

[0122]

[0123] S1 and S2 are negative reciprocals of each other, and the relationship between the two corresponding straight lines is a plane perpendicular relationship. According to the two straight lines L F1 and L F2 The equation can be used to obtain the intersection point P of the two ridge fitting lines in , according to the intersection point P in The two solutions of the slope can be used to obtain the corresponding straight line equations L of the two navigation lines to be selected NA1 and L NA2 .

[0124] In L NA1 and L NA2Select the straight line between the two ridge fitting lines, and the selected straight line is the navigation line. The specific method is to calculate L F1 , L F2 , L NA1 and L NA2 In the pixel coordinate system uv, the intersection point with the u axis is F1 The intersection with the u-axis and L F2 The straight line in the middle of the intersection with the u-axis is the navigation line.

[0125] S11: Introduce a dynamic segmentation level strip number feedback mechanism to improve the algorithm's multi-period adaptability.

[0126] The number of horizontal strips set by existing research institutes is fixed, and the number is obtained based on the crop growth period or weed condition test. The height of the horizontal strips used to divide the image can be determined based on the number of horizontal strips, so the height is also a fixed value. If the height is fixed, when the plants are small and the spacing between plants is large, too few feature points will be extracted from a single crop, affecting the fitting accuracy; when the plants are large and the spacing between plants is small, too many feature points will be extracted from a single crop, affecting the timeliness of the overall method. In response to the above problems, the present invention introduces a self-created dynamic segmentation horizontal strip number feedback mechanism, which can automatically determine the number and height of horizontal strips suitable for each environment. The main idea is to form a closed-loop feedback mechanism between the result output process of the previous frame and the calculation process of the number of segmentation strips of the next frame. The specific process of this mechanism is as follows:

[0127] (1) Initialize a variable NUMBER to represent the number of horizontal stripes. The algorithm starts with the first frame and sets NUMBER to 30.

[0128] (2) After the image processing process of the present invention, the number of left feature points num_left and the number of right feature points num_right of the frame image can be obtained.

[0129] (3) Calculate the average of num_left and num_right, expressed as avg_feature.

[0130] (4) Calculate the number of horizontal strips required for the next frame image according to the following feedback mechanism formula, expressed as num_next, set NUMBER = num_next, start processing the next frame image, and repeat this cycle until the program termination command is issued.

[0131]

[0132] The flowchart of the above process is as follows:

[0133] Figure 7 shows the results of navigation line detection in field images for three growth periods and two lighting environments. This demonstrates that our method can effectively extract navigation lines in the corresponding environments. The solid lines represent the detection lines obtained by our method, the dashed lines represent the manually marked reference lines, and the solid line in the middle represents the navigation lines. It can be seen that the detection lines and reference lines align well, and the navigation lines are generally in the ideal position, demonstrating the adaptability of our method to various environments.

[0134] The present invention can be widely used in extracting navigation lines of other different types of crop rows.

[0135] The above embodiments are only used to illustrate the present invention, and are not intended to limit the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention. The scope of patent protection of the present invention should be defined by the claims.

Claims

1. A method for extracting crop navigation lines in a high-ridge cultivation environment based on machine vision, characterized by: It includes the following steps: S1: Use a color camera to obtain RGB images of farmland; S2: Convert the image color space from RGB color space to CIE-Lab color space; S3: extracting the a component of the CIE-Lab color space to preliminarily extract crop row features and grayscale the acquired color image; S4: performing binarization processing on the grayscale image to obtain a binary image; S5: filtering out noise from the binary image; S6: Extract crop row feature points using the improved equidistant segmented vertical projection method; S7: using the adaptive clustering method to cluster adjacent feature points; S8: Using the dynamic segmentation point clustering method to determine the final clustering feature point set; S9: The feature point set is optimized using the lateral distance optimization method and the point-line distance optimization method. During the optimization process, the linear regression method based on the Huber loss function is used to fit the optimized feature point set. S10: Calculating a navigation line based on crop rows; S11: Introducing a dynamic segmentation level strip number feedback mechanism to improve the algorithm's multi-period adaptability; Step S6 includes the following steps: (1) Calculate the number of horizontal stripes N based on the dynamic segmentation horizontal stripe number feedback mechanism D ; (2) According to the number of horizontal strips N D Calculate the horizontal strip height D; (3) Divide N into binary images D Horizontal strips, number all the divided horizontal strips in order from bottom to top, initialize x = N D -1, in the x-th horizontal strip, find the point where f(i, j) = 255, and project the points where f(i, j) = 255 in the same column onto the column coordinate axis to obtain a vertical projection statistical graph; (4) Calculate the vertical projection statistics of the x-th horizontal strip, which is recorded as the x-th vertical projection statistics. Let p(i) be the number of pixels with a pixel value of 255 in the i-th column of the binary horizontal strip. The calculation formula of p(i) is shown in formula (1), where i∈{0,1,2,…,u}; x∈{N D -1,…,2,1,0}; (5) Calculate the average projection value avgp(x) of p(i) in all columns of the x-th vertical projection statistical graph; identify the larger and higher peak area as the crop characteristic area, and simplify the crop characteristic area into a point, which is the crop characteristic point; the calculation formula of avgp(x) is: Where W is the image width; (6) Using the method of determining the vertical coordinate value of the feature point based on the area threshold, the vertical coordinate of the horizontal line of the feature point is determined as m(x). Yx=UppB+m(x) (4) Among them, s is the image area threshold, Y x is the vertical coordinate value of each feature point in the x-th horizontal strip; (7) Compare the magnitude relationship between p(i) and m(x); B(i) is the binary statistic of the x-th vertical projection; (8) The calculation formula of the horizontal coordinate of the feature point is as follows: where p_mid c represents the horizontal coordinate of the feature point numbered c in the xth horizontal strip; u1 is the starting point of the continuous data set; u2 is the ending point of the continuous data set; n k is the number of data in the continuous data set; n k =u2-u1;th n is the noise judgment threshold; (9) Output feature point set SET_FP = {(p_mid0, Y x ),(p_mid1,Y x ),(p_mid2,Y x ),…,(p_mid Q-1 ,Y x )}, Q is the number of feature points in the horizontal strip; (10) Let x = x - 1, and repeat steps (4) to (9) until x < 0, at which point the program stops.

2. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 1, characterized in that: In the step S5 , a closing operation is first performed on the binarized image, and then an opening operation is performed on the processed image, so as to filter out noise.

3. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 1, characterized in that: In the step S7, the adaptive clustering method is used to cluster multiple feature points of the same crop into one feature point. The number of feature points of the Kth horizontal strip is Q. K , P K,m ,m=0,1,2,…n K represents the mth feature point in the Kth horizontal strip, then the distance between the mth feature point and the m+1th feature point in the Kth horizontal strip can be expressed as d K,m,(m+1) , where m=0,1,2,…n K -1, the average distance between all adjacent feature points in the same horizontal strip can be expressed as d K,avg ; Set a clustering threshold th div The distance d between each adjacent feature point K,m,(m+1) In comparison, for the feature points in the Kth horizontal strip, when d K,m,(m+1) >th div When , record the m value at this time, and divide the feature points into several feature point sets according to all the recorded m values; div and d K,avg The calculation formula is as follows: Because the image crop rows are arranged with vanishing points, the average distance between adjacent feature points in the horizontal strip at the top of the image is smaller than the average distance between adjacent feature points in the horizontal strip at the bottom of the image. A fixed F value cannot adapt to every horizontal strip. Therefore, it is necessary to set the F value to a dynamic value. After a lot of experiments, the following empirical calculation formula for the F value was obtained: F K =F K+1 -0.04 Among them, F K is the F value of the Kth horizontal strip, F K+1 is the F value of the K+1th horizontal strip, and the F value setting value is a dynamic changing value.

4. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 1, characterized in that: In said S8, a dynamic segmentation point clustering method is used to assign the cluster feature points to corresponding feature point sets; the segmentation midpoint p used is div,K The calculation needs to comprehensively consider the distribution of existing segmentation midpoints.

5. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 4, characterized in that: It also includes extracting only two cluster feature points on the left and right sides of the segmentation midpoint in the S8.

6. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 1, characterized in that: In the step S9, the lateral distance optimization is performed by comparing the horizontal coordinate difference A with the distance threshold th hori The size relationship is used to determine whether to remove it.

7. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 1, characterized in that: In the S9, the point-line distance optimization based on the Huber loss function is performed by comparing the distance from the point to the first fitted straight line and the distance threshold th hori The size relationship is used to determine whether to remove it.

8. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 7, characterized in that: In the above-mentioned S9, the point-line distance optimization of the Huber loss function adopts iterative weighted least squares to estimate the regression coefficient, and the weight of each sample is determined according to the size of the regression residual.

9. The method for extracting crop navigation lines in a high ridge cultivation environment based on machine vision according to claim 1, characterized in that: In S10, the navigation line is defined as the angle bisector of the angle formed by two crop row straight lines at two center positions. The angle bisector is determined based on the relationship between the straight lines and the axes of the coordinate system.

Images