A method for furrow identification and navigation line extraction in a self-propelled transport platform for ridge-grown vegetables

Abstract

A method for furrow identification and navigation line extraction using a self-propelled transport platform for ridge-grown vegetables includes image acquisition, image boundary smoothing, color conversion, threshold segmentation, denoising, edge detection, midpoint extraction, and navigation line fitting. Gaussian blur filtering is used to blur the image boundaries, and the traditional BGR image is converted to HSV to reduce the impact of lighting conditions on furrow identification. Low-impact components are separated for further processing. A masking algorithm is used to enhance grayscale information, combined with the Otsu's method to effectively achieve threshold segmentation and successfully identify furrow locations. Morphological denoising methods are then used to remove most noise interference from the image, and depth denoising is achieved based on the contour function (FindContours). Simultaneously, furrow boundaries are extracted. Then, using the x-coordinates of the two boundary points of the furrow, the midpoint coordinates of each row in the image pixels are obtained as a point cloud. Finally, these point cloud coordinates are used to improve the RANSAC algorithm to achieve navigation line fitting for the furrows.

Description

Technical Field

[0001] This invention relates to visual recognition technology for ridge-growing agriculture, specifically to furrow recognition and navigation line extraction technology for field self-propelled auxiliary transportation platforms suitable for ridge-growing vegetables. Background Technology

[0002] Ridge-cultivated vegetables are one of the main vegetable crops cultivated in my country, involving many varieties, a wide planting area, and high yields. Especially in the rainy climate of southern regions, a large number of vegetable crops are grown using ridge cultivation, with the main equipment including single or double ridge ridging machines, double or four-row transplanters, and self-propelled auxiliary harvesting platforms. With the continuous improvement of agricultural mechanization in my country, the demand for agricultural machinery operations has become increasingly strong, moving from "organic and usable" to "organic and easy to use." This requires agricultural machinery operations to develop towards automation and unmanned operation, and visual recognition technology is one of the fundamental technologies supporting this development direction. Based on machine vision technology, feature recognition of agricultural work scenes is performed. Through image information preprocessing, target detection, target tracking, and other steps, the required information is finally extracted from the image, thus replacing the human eye for measurement and judgment. Combined with tracking control and other technologies, autonomous high-precision navigation of ridge-cultivated vegetable agricultural machinery equipment has become an urgent problem to be solved in current visual recognition technology for ridge-cultivated vegetables. Summary of the Invention

[0003] The technical problem solved by this invention is to provide a method for furrow identification and navigation line extraction of a self-propelled transportation platform for ridge-grown vegetables, so as to solve the problems in the background art mentioned above.

[0004] The technical problem solved by this invention is achieved by the following technical solution, and the specific steps are as follows:

[0005] Step 1): Image acquisition of ridge-grown vegetables

[0006] Data is collected from video segments of the operation scene using a self-propelled auxiliary transportation platform. The collected video images are decomposed frame by frame and a coordinate system is established. The top left corner of the image is taken as the origin (0,0), the top right corner of the image is taken as (width,0), and the bottom left corner of the image is taken as (0,height).

[0007] Step 2): Perform boundary smoothing processing on the acquired images of ridge-grown vegetables.

[0008] Because the boundaries of ridge-grown vegetables are extremely uneven, they will cause significant interference to subsequent straight-line fitting. Therefore, it is necessary to perform Gaussian blur filtering on the images acquired in step 1) to smooth them according to the growth characteristics of the vegetables, so as to obtain the boundaries between the furrows and the vegetables.

[0009] Step 3): Perform color conversion on the image of the ridged vegetables after boundary smoothing.

[0010] Based on the color difference between furrows and vegetables, and taking into account the differences in light conditions at different times in the field, the image obtained after processing in step 2) is converted into hue (H), saturation (S), and brightness (V) according to the blue (B), green (G), and red (R) color model.

[0011] Step 4): Perform color separation on the color-converted image of the cultivated vegetables to obtain a grayscale image.

[0012] After HSV three-color conversion, the three-color channels are separated, and the H and S components are observed separately. The H component, which is not affected by lighting conditions and has clear furrow boundaries, is obtained through observation for further processing.

[0013] Step 5): Perform threshold segmentation on the color-separated image of the cultivated vegetables to obtain a binary image.

[0014] Because vegetables grow differently at different stages and their leaves become larger after reaching maturity, the image contrast decreases after the Gaussian blur filtering smoothing process in step 2). In addition, the uneven lighting on different parts of the vegetable leaves affects the threshold segmentation effect. Therefore, it is necessary to create a mask and combine it with the Otsu's method to perform threshold segmentation on the components that are not affected by lighting conditions observed in step 4), thereby obtaining a binary image.

[0015] Step 6): Denoise the binary image.

[0016] Since the binary image obtained in step 5) contains a lot of noise, in order to eliminate this noise, the image needs to be fine-tuned using the dilation function in morphology. Then, the small dots, burrs and small bridges in the image are removed by using the closing operation of dilation followed by erosion and the opening operation of erosion followed by dilation, so as to remove noise while compensating for lost image information.

[0017] Step 7): Perform edge detection on the denoised image of the ridged vegetables.

[0018] The outermost contour of all the ridged vegetable images after the denoising process in step 6) is detected using the contour function (findContours), and then the contour with the largest area in the contour detection is selected as the ridge boundary.

[0019] Step 8): Extract the midpoints of the furrows based on the detected furrow boundaries.

[0020] The image information of the ridge-grown vegetables has been extracted through steps 1) to 7). Then, the two boundaries of the furrow between the two rows of vegetables in step 7) are extracted. The white point with a pixel value of 255 in each row is found by scanning line by line. The point cloud of the pixel coordinates of the white point in each row is calculated as a reference point and then used as the object for line fitting.

[0021] Step 9): Generate navigation lines by fitting the extracted midpoints of the furrows.

[0022] Using the point cloud extracted in step 8), the navigation line is obtained by improving the RANSAC algorithm based on the distribution characteristics of the point cloud.

[0023] Beneficial effects:

[0024] This invention converts the traditional RGB color model to the HSV color model, reducing the impact of lighting conditions on furrow recognition. It enhances image feature information using the Masking algorithm, improving threshold segmentation under uneven lighting conditions. The findContours function extracts furrow boundaries, and finally, an improved RANSAC algorithm is used to fit the navigation line. Compared to the Least Square Method (LSM), the RANSAC algorithm effectively reduces the impact of error points on the fitting result. The traditional RANSAC algorithm uses random sample sampling to fit a straight line, which can lead to a relatively large discrepancy between the fitted line and the actual line when processing similar point clouds in close frames in step 8), resulting in fitted lines with significant jitter. The improved RANSAC algorithm effectively removes and neutralizes these fitted lines with significant jitter, improving the accuracy of the fitted navigation line. This invention is highly adaptable to the environment and has low computational complexity. Verification shows an average processing time of 52.3ms per frame, effectively meeting the operational requirements of visual navigation for ridge-grown vegetable harvesting. It can also be applied in the field of visual navigation and automated operations for ridge-grown broadleaf vegetables. Attached Figure Description

[0025] Figure 1 A color image with a coordinate system established after being captured by a color camera;

[0026] Figure 2 This is an image of the result after smoothing using Gaussian blur filtering;

[0027] Figure 3 The image is a result of converting an RGB color image to an HSV color image.

[0028] Figure 4 The grayscale image after component extraction from an HSV color image;

[0029] Figure 5 This is a diagram showing the result of image thresholding.

[0030] Figure 6 This is a preliminary image showing the noise reduction effect;

[0031] Figure 7 The result image shows the effect of further edge detection and extraction of furrow boundaries;

[0032] Figure 8 The image shows the result after extracting the point cloud from the midpoint of the furrow boundary.

[0033] Figure 9 The image shows the effect of using the improved RANSAC algorithm to fit and generate a straight line. Detailed Implementation

[0034] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0035] See Figures 1-9 The method for furrow identification and navigation line extraction using a self-propelled transportation platform for ridge-grown cabbage is described in the following steps:

[0036] Step 1): Image acquisition of ridge-grown cabbage

[0037] Using double-row cabbage planted on a single ridge as the subject, a self-propelled auxiliary transport platform was used to collect data by shooting video segments of the operation scene. The camera was 20cm to 100cm off the ground, with an angle of 10° to 90° downwards. The focus was adjusted so that the camera's field of view covered both ridges. Video images of the cabbage planted on ridges in the field were captured at a resolution of 1920×1080 pixels and a frame rate of 30fps. The captured video images were decomposed frame by frame and a coordinate system was established. The top left corner of the image was set as the origin (0,0), the top right corner of the image was set as (width,0), and the bottom left corner of the image was set as (0,height).

[0038] Step 2): Perform boundary smoothing processing on the acquired images of ridge-grown cabbage.

[0039] Based on the growth characteristics of cabbage, the image obtained in step 1) is smoothed by Gaussian blur filtering to separate the furrows from the cabbage boundary. A second-order Gaussian function equation (1) is set with a Gaussian kernel size of 21×21 and a standard deviation of 21.

[0040]

[0041] Step 3): Perform color conversion on the smoothed cabbage image.

[0042] Based on the color difference between furrows and cabbage, and taking into account the different light conditions in the field at different times, the blue (B), green (G), and red (R) color models of the image are converted into hue (H), saturation (S), and brightness (V) according to equations (2) to (8).

[0043]

[0044] V = Cmax (4)

[0045] (R,G,B)′=(R,G,B) / 255 (5)

[0046] Cmax=max(R′,G′,B′) (6)

[0047] Cmin=min(R′,G′,B′) (7)

[0048] Δ=Cmax-Cmin (8)

[0049] Step 4): Perform color separation on the color-converted image of the cabbage to obtain a grayscale image.

[0050] After the HSV three-color conversion in step 3), since the H component is the basic attribute of color, determined by different wavelengths of light in the spectrum, while the S component reflects the purity or vividness of the color, which is related to how much white or gray is mixed in the color, these two components are not directly affected by the light intensity. Therefore, the three color channels are separated and the H and S components are selected for comparative observation. Through image comparison, it is found that the furrows and cabbage are easier to distinguish in the H component, and the furrows with clear boundaries can be obtained. Finally, the grayscale image of the H component is selected for the next step of processing.

[0051] Step 5): Perform thresholding on the color-separated cabbage image to obtain a binary image.

[0052] Threshold segmentation is performed on the H component grayscale image obtained after comparison in step 4) to distinguish between cabbage and furrows in the image. After Gaussian blur filtering and smoothing in step 2), the image contrast decreases, and the threshold segmentation effect is poor due to uneven lighting in different parts of the cabbage leaves. To solve this problem, a mask is first created and the grayscale range of interest in the mask is defined as 50 to 150. Then, the grayscale information of the H component of the grayscale image obtained in step 4) is enhanced by the mask. Finally, the Otsu's method is used to perform threshold segmentation to obtain a binary image.

[0053] Step 6): Denoise the binary image.

[0054] Although the binary image obtained in step 5) can effectively extract furrow information, the extracted furrow information is accompanied by a lot of noise due to interference from factors such as cabbage leaves. To eliminate this noise, the image is first fine-tuned using the dilation function in morphology, with the kernel size set to 5×5 and the number of iterations to 1. Then, the small dots, burrs and small bridges in the image are removed by using the closing operation of dilation followed by erosion, and then the opening operation of erosion followed by dilation. The kernel size is 40×40, so as to remove noise while compensating for lost image information.

[0055] Step 7): Perform edge detection on the denoised cabbage image.

[0056] Edge detection can detect the true furrow boundaries and remove large blank noise blocks. The contour function (findContours) is used to detect all the outermost contours of the image obtained in step 6), and the elements in the horizontal, vertical and diagonal directions are compressed to obtain contour-like curves. The area contained in the specified contour is used as the comparison value, and the contour with the largest area in the contour detection is selected as the furrow boundary. Then, an array of all zeros is created, that is, a completely black image. The area calculation result is then plotted on the array of all zeros, and the contour line index is set to -1, that is, the entire contour is drawn. Finally, the contour line color is set to white, that is, B,G,R=255, and the contour line thickness is specified to be 1.

[0057] Step 8): Extract the midpoints of the furrows based on the detected furrow boundaries.

[0058] Having extracted the image information of the ridged cabbage through steps 1) to 7), to successfully fit the center navigation line of the furrow, the midpoint point cloud of the pixels at the two boundaries of the furrow is selected as the reference point for straight line fitting. The image height (number of rows) is calculated as height, and the width (number of columns) is calculated as width. An empty list is initialized to store the coordinate information. A for loop is established to find the n pixels (y1, x1), (y1, x2), ..., (y1, x...) with a pixel value of 255 in each row. n ), (y2, x1)..., (y height x n ), 2≤n≤6, calculate the average coordinates of the pixels in each row. The integer part is then stored in an empty list as a fitting reference point. Skip this line until the loop exits after `height` times.

[0059] Step 9): Generate navigation lines by fitting the extracted midpoints of the furrows.

[0060] Initialize a sliding list to store the slope and intercept of lines, and set its size to 18 cells, based on the point cloud stored in the list in step 8). Determine if the number of stored point clouds is greater than 1, i.e., at least two points are required for line fitting;

[0061] Use list comprehension to convert all x and y coordinates into NumPy arrays to meet the input requirements of the RANSAC regression model. Create a RANSAC regression model with 100 iterations. Fit a straight line using the RANSAC algorithm, denoted as (slope, intercept). Store (slope, intercept) in a cell of a sliding list. When the sliding list is full, sort all the lines in the list by slope in ascending order to obtain (slope1, intercept1), (slope2, intercept2), ..., (slope... 18 intercept 18 Then, the cut-off mean method is used to cut off (slope9, intercept9), (slope) 10 intercept 10 ), (slope 11 intercept 11 ), calculate the new line as Let (avg_slope, avg_intercept) be the line fitted by the improved RANSAC algorithm. Then, remove the line with the smallest slope (slope1, intercept1) from the original list. Then, add the fitted line of the next frame image to the sliding list and reorder it to get a new list. Repeat the above process to output a new line until the new navigation line of the last frame image is output.

[0062] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for furrow identification and navigation line extraction in a self-propelled transport platform for ridge-cultivated vegetables, characterized in that, The specific steps are as follows: Step 1): Acquire images of the ridge-grown vegetables; Step 2): Perform boundary smoothing processing on the acquired images of ridge-grown vegetables; Step 3): Perform color conversion on the image of the ridged vegetables after boundary smoothing; Step 4): Perform color separation on the color-converted image of the ridge-grown vegetables to obtain a grayscale image; Step 5): Threshold segmentation is performed on the color-separated image of the ridge-grown vegetables to obtain a binary image; Step 6): Denoise the binary image; Step 7): Perform edge detection on the denoised image of the ridged vegetables; Step 8): Extract the midpoint of the furrow based on the detected furrow boundaries; Step 9): Generate navigation lines based on the extracted midpoints of the furrows; Step 8) involves extracting the midpoints of furrows by scanning row by row to find the white points (pixels with a value of 255) in each row. The midpoint cloud of the white points in each row is then calculated as a reference point for line fitting. The image height (number of rows) is calculated as `height`, and the width (number of columns) is calculated as `width`. An empty list is initialized to store the coordinate information. A for loop is then established to find the n pixels with a value of 255 in each row. y 1, x 1), y 1, x 2), ..., ( y 1, x n (), y 2, x 1) ..., ( y height , x n ), 2 n 6. Calculate the average coordinates of each row of pixels, i.e. ( y 1, (), y 2, ), ..., ( y height , The integer part is then rounded and stored in an empty list as a fitting reference point. When n... Skip this line when [2:6] until the loop exits after looping height times; The midpoint fitting in step 9) generates the navigation line using the point cloud extracted in step 8). Based on the distribution characteristics of the point cloud, the RANSAC algorithm is improved to fit the navigation line. A sliding list is initialized to store the slope and intercept of the line, and its size is set to 18 cells. Based on the point cloud stored in the list in step 8) y 1, (), y 2, ), ..., ( y height , Determine whether the number of stored point clouds is greater than 1, that is, at least two points are required to perform line fitting.

2. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The image acquisition mentioned above is based on the self-propelled auxiliary transportation platform to collect video segments of the operation scene. The acquired video images are decomposed frame by frame and a coordinate system is established. The top left corner of the image is taken as the origin (0,0), the top right corner of the image is taken as (width,0), and the bottom left corner of the image is taken as (0,height). The camera should be positioned 20 to 100 cm off the ground, tilted horizontally downwards at an angle of 10 to 90 degrees, and the focus adjusted so that the camera's field of view covers both rows. The video image of vegetables grown on ridges in the field should be captured at a resolution of 1920×1080 pixels and a frame rate of 30fps.

3. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The boundary smoothing process described above requires Gaussian blur filtering to smooth the image acquired in step 1) based on the growth characteristics of vegetables, thereby obtaining the boundary between the furrows and the vegetables. Set up the second-order Gaussian function equation (1): The Gaussian kernel size is 21×21, and the standard deviation is 21.

4. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The color conversion described above requires considering the color difference between furrows and vegetables, as well as the differences in light conditions at different times in the field. The image obtained after processing in step 2) is converted from blue (B), green (G), and red (R) color models to hue (H), saturation (S), and brightness (V). The conversion equation is as follows: 。 5. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The aforementioned color separation to obtain a grayscale image involves separating the three color channels after HSV three-color conversion, and then comparing and observing the H and S components separately. By observation, the H component, which is not affected by lighting conditions and has clear furrow boundaries, is obtained. Finally, the grayscale image of the H component is selected for the next step of processing.

6. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The threshold segmentation to obtain a binary image involves performing threshold segmentation on the H component grayscale image obtained after comparison in step 4) to distinguish between vegetables and furrows in the image. First, a mask is created and the grayscale range of interest for the mask is defined as 50 to 150. Then, the mask is used to enhance the grayscale information of the H component of the grayscale image obtained in step 4). Finally, the Otsu's method is used to perform threshold segmentation to obtain a binary image.

7. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The aforementioned denoising process for binary images involves fine-tuning the image using a dilation function in morphology, followed by a closing operation (dilation followed by erosion) and an opening operation (erosion followed by dilation) to remove small dots, burrs, and bridges in the image. This achieves noise removal while compensating for lost image information. The kernel size for the dilation function is set to 5×5, with one iteration, and the kernel size for the closing and opening operations is set to 40×40.

8. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, The edge detection described above uses the contour function (findContours) to detect the outermost contour of all the denoised vegetable images after step 6). At the same time, it compresses elements in the horizontal, vertical and diagonal directions to obtain a contour curve. Furthermore, the area contained in the specified contour is used as a comparison value, and the contour with the largest area in the contour detection is selected as the furrow boundary. Then, an array of all zeros is created, that is, a completely black image. Subsequently, the area calculation result is plotted on the array of all zeros, and the contour line index is set to -1, that is, the entire contour is drawn. Finally, the contour line color is set to white, that is, B, G, R = 255, and the contour line thickness is specified as 1.

9. The method for furrow identification and navigation line extraction of a self-propelled transport platform for ridge-grown vegetables according to claim 1, characterized in that, Use list comprehension to convert all x and y coordinates into NumPy arrays to meet the input requirements of the RANSAC regression model. Create a RANSAC regression model with 100 iterations. Fit a straight line using the RANSAC algorithm, denoted as (slope, intercept). Store (slope, intercept) in a cell of a sliding list. When the sliding list is full, sort all the lines in the list by slope in ascending order to obtain (slope1, intercept1), (slope2, intercept2), ..., (slope... 18 intercept 18 Then, using the truncation mean method, (slope9, intercept9) are extracted. 10 intercept 10 (slope) 11 intercept 11 ), calculate the new line as ( , The line is denoted as (avg_slope, avg_intercept), and the line is output as the navigation line fitted by the improved RANSAC algorithm. Then, the line with the smallest slope (slope1, intercept1) in the original list is removed. The line fitted by the next frame image is added to the sliding list and reordered to obtain a new list. The above process is repeated to output a new line until the new navigation line of the last frame image is output.

Images