Furrow identification and navigation line extraction method of self-propelled transportation platform for ridge planting vegetables

Abstract

The invention discloses a furrow identification and navigation line extraction method of a self-propelled transportation platform for ridge planting vegetables. The furrow identification and navigation line extraction method comprises the steps of image acquisition, image boundary smoothing processing, color conversion, threshold segmentation, denoising processing, edge detection, midpoint extraction and navigation line fitting. Gaussian blur filtering is used to carry out boundary blur processing on the image, then a traditional BGR image is converted into an HSV image to reduce the influence of an illumination condition on furrow identification, and a low-influence component is separated to carry out next processing; the method comprises the following steps of: enhancing gray information by adopting a Masking algorithm, effectively realizing threshold segmentation and successfully identifying a furrow position in combination with an OTSU method, further removing most noise interference in an image by utilizing a morphological denoising method, realizing deep denoising of the image on the basis of a FindContours, extracting a furrow boundary, and finally, extracting the furrow boundary. Then, the horizontal coordinates of the two boundary points of the furrow are utilized to solve the midpoint coordinate point cloud of each row in the image pixels, and finally, the point cloud coordinates are utilized to improve the RANSAC algorithm to realize the navigation line fitting of the furrow.

Description

Technical Field

[0001] The invention relates to a visual recognition technology for ridge farming, and in particular to a ridge ditch recognition and navigation line extraction technology for a field self-propelled auxiliary transport platform suitable for ridge farming vegetables. Background Art

[0002] Ridge-grown vegetables are one of the main vegetables cultivated in my country, involving many varieties, a wide planting area, and high yields. Especially in the southern region with a rainy climate, a large number of vegetables are planted in ridge cultivation mode, and the main machines used are single-ridge or double-ridge ridgers, double-row or four-row transplanters, and self-propelled auxiliary harvesting platforms. With the continuous improvement of my country's agricultural mechanization level, the demand for agricultural machinery operations from "organic availability" to "organic ease of use" is becoming increasingly strong, which requires agricultural machinery operations to develop in the direction of automation and unmanned operation, and visual recognition technology is one of the basic guarantee technologies for this development direction. Based on machine vision technology, feature recognition of agricultural work scenes is performed, and through steps such as image information preprocessing, target detection, and target tracking, the required information is finally extracted from the image, and then the human eye is replaced for measurement and judgment, and combined with tracking control and other technologies, autonomous high-precision navigation of ridge-grown vegetable agricultural machinery equipment is realized, which has become a difficult problem that needs to be solved in the current ridge-grown vegetable visual recognition technology. Summary of the invention

[0003] The technical problem solved by the present invention is to provide a method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-cultivated vegetables, so as to solve the problems in the above-mentioned background technology.

[0004] The technical problem solved by the present invention is achieved by adopting the following technical solution, and the specific steps are as follows:

[0005] Step 1): Image acquisition of ridge-grown vegetables

[0006] Based on the self-propelled auxiliary transportation platform, the video segments shot in the working scene are used for data collection. The collected video images are decomposed frame by frame and a coordinate system is established. The top left corner of the image is used as the coordinate origin (0, 0), the top right corner coordinates of the image are (width, 0), and the bottom left corner coordinates of the image are (0, height).

[0007] Step 2): Perform boundary smoothing on the collected ridge vegetable images

[0008] Since the boundaries of ridge-grown vegetables are extremely uneven, they will cause great interference to the subsequent straight-line fitting. Therefore, it is necessary to perform Gaussian blur filtering and smoothing processing on the image collected in step 1) according to the growth characteristics of the vegetables, so as to obtain the boundaries of the ridges and vegetables.

[0009] Step 3): Perform color conversion on the ridge vegetable image after boundary smoothing

[0010] According to the color difference between the ridges and vegetables, and considering the differences in lighting conditions at different times in the field, the image obtained after processing in step 2) is converted into hue (H), saturation (S), and brightness (V) according to the blue (B), green (G), and red (R) color model.

[0011] Step 4): Perform color separation on the color-converted ridge vegetable image to obtain a grayscale image

[0012] After HSV three-color conversion, the three color channels are separated, and then the H and S components are observed respectively. The H component that is not affected by lighting conditions and has clear ridge boundaries is obtained through observation for the next step of processing.

[0013] Step 5): Threshold segmentation is performed on the color-separated ridge vegetable image to obtain a binary image

[0014] Since vegetables grow differently at different growth stages and their leaves become wider after entering the mature stage, the image contrast decreases after the Gaussian blur filter smoothing process in step 2). In addition, the illumination of different parts of the vegetable leaves is uneven, which affects the threshold segmentation effect. Therefore, it is necessary to create a mask and combine the maximum between-class variance method (OTSU) to perform threshold segmentation on the components not affected by the illumination conditions observed in step 4) to obtain a binary image.

[0015] Step 6): De-noise the binary image

[0016] Since the binary image obtained in step 5) contains a lot of noise, in order to eliminate the noise, it is necessary to use the dilation function in morphology to fine-tune the image, and then use the closing operation of first dilation and then erosion, and the opening operation of first erosion and then dilation to remove small dots, burrs and small bridges in the image, so as to remove the noise and compensate for the lost image information at the same time.

[0017] Step 7): Perform edge detection on the denoised ridge vegetable image

[0018] The contour function (findContours) is used to detect the outermost contours of all ridge vegetable images after denoising in step 6), and then the contour with the largest area in the contour detection is selected as the ridge boundary.

[0019] Step 8): Extract the ridge midpoint based on the detected ridge boundary

[0020] Through steps 1) to 7), the image information of ridge-grown vegetables has been extracted, and then the two boundaries of the ridge between the two rows of vegetables in step 7) are extracted. The pixel of each row is 255, that is, the white point, and the point cloud of the pixel coordinates of each row of white points is calculated as a reference point and then as a straight line fitting object.

[0021] Step 9): Generate a navigation line based on the extracted ridge midpoint fitting

[0022] Using the point cloud extracted in step 8), the navigation line is obtained by improving the RANSAC algorithm according to the distribution characteristics of the point cloud.

[0023] Beneficial effects:

[0024] The present invention converts the traditional RGB color model into the HSV color model, thereby reducing the influence of the illumination conditions on ridge and furrow recognition; uses the Masking algorithm to enhance the image feature information, and improves the threshold segmentation under the condition of uneven illumination; uses the findContours function to extract the ridge and furrow boundary, and finally improves the RANSAC algorithm to fit the navigation line. Compared with the least square method (Least Square Method, LSM), the RANSAC algorithm can effectively reduce the influence of the error point on the fitting result, while the traditional RANSAC algorithm uses the random sample sampling method to fit the straight line. When processing the similar point cloud of the similar frames in step 8), there is a relatively large gap between the fitted straight line and the actual straight line, that is, the fitted straight line with a large jitter amplitude appears, and the improved RANSAC algorithm can effectively remove and neutralize these fitted straight lines with a large jitter amplitude, thereby improving the accuracy of the fitted navigation line; the present invention has strong adaptability to the environment, and the algorithm has a small amount of calculation. It has been verified that the average processing time of each frame image is 52.3ms, which can effectively meet the operation requirements of visual navigation of ridge vegetable harvesting, and can be applied in the field of visual navigation and automated operation of ridge broad-leaved vegetable operation equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A color image of the coordinate system established after taking pictures with a color camera;

[0026] Figure 2 This is the image effect obtained after smoothing using Gaussian blur filtering;

[0027] Figure 3 This is the image effect after converting the RGB color image to the HSV color image;

[0028] Figure 4 This is the grayscale image effect after extracting components from the HSV color image;

[0029] Figure 5 This is the effect diagram after image threshold segmentation;

[0030] Figure 6 This is the effect picture of preliminary denoising;

[0031] Figure 7 To further detect edges and extract the effect of ridge and ditch boundaries;

[0032] Figure 8 This is the effect picture after the point cloud of the ridge boundary is extracted;

[0033] Fig. 9 This is the effect diagram of using the improved RANSAC algorithm to fit and generate a straight line. DETAILED DESCRIPTION

[0034] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below with reference to specific diagrams.

[0035] See also Figures 1 to 9 The method for ridge ditch recognition and navigation line extraction of a self-propelled transport platform for ridge-planted cabbage is described in detail, and the specific steps are as follows:

[0036] Step 1): Image acquisition of ridge-grown cabbage

[0037] The ridge-planted cabbage with one ridge width and two rows was taken as the object, and a self-propelled auxiliary transport platform was used to capture video segments of the operation scene for data collection. The camera was 20 cm to 100 cm above the ground and tilted 10° to 90° horizontally downward. The focal length was adjusted to make the camera field of view cover two ridges, and video images of ridge-planted cabbage in the field were captured with a pixel of 1920×1080 and a frame rate of 30fps. The collected video images were decomposed frame by frame and a coordinate system was established. The top left corner of the image was taken as the coordinate origin (0,0), the top right corner of the image was taken as (width,0), and the bottom left corner of the image was taken as (0,height).

[0038] Step 2): Perform boundary smoothing on the collected ridge-grown cabbage image

[0039] According to the growth characteristics of cabbage, the image obtained in step 1) is smoothed by using Gaussian blur filtering to smooth the ridge and cabbage boundaries, and a second-order Gaussian function equation (1) is set, with a Gaussian kernel size of 21×21 and a standard deviation of 21.

[0040]

[0041] Step 3): Perform color conversion on the ridged cabbage image after boundary smoothing

[0042] According to the color difference between ridges and cabbages, and taking into account the different lighting conditions in the field at different times, the blue (B), green (G), and red (R) color models of the image are converted to hue (H), saturation (S), and brightness (V) according to equations (2) to (8).

[0043]

[0044] V=Cmax (4)

[0045] (R,G,B)′=(R,G,B) / 255 (5)

[0046] Cmax=max(R′,G′,B′) (6)

[0047] Cmin=min(R′,G′,B′) (7)

[0048] Δ=Cmax-Cmin (8)

[0049] Step 4): Perform color separation on the color-converted ridge-grown cabbage image to obtain a grayscale image

[0050] After the HSV three-color conversion in step 3), since the H component is the basic attribute of color and is determined by the different wavelengths of light in the spectrum, and the S component reflects the purity or brightness of the color and is related to how much white or gray is mixed in the color, these two components are not directly affected by the light intensity, so the three-color channels are separated and the H and S components are selected for comparative observation. Through image comparison observation, it is found that the ridges and cabbages in the H component are easier to distinguish, and the ridges with clear boundaries can be obtained. Finally, the grayscale image of the H component is selected for the next step of processing.

[0051] Step 5): Threshold segmentation is performed on the color-separated ridged cabbage image to obtain a binary image

[0052] The H component grayscale image obtained after comparison in step 4) is threshold segmented to distinguish the cabbage and the furrow in the image; the image contrast is reduced after the Gaussian blur filter smoothing process in step 2), and the threshold segmentation effect is not good due to the uneven illumination of different parts of the cabbage leaves; to solve this problem, a mask is first created and the grayscale range of interest of the mask is defined as 50 to 150, and then the mask is used to enhance the grayscale information of the H component of the grayscale image obtained in step 4), and finally the maximum between-class variance method (OTSU) is used for threshold segmentation to obtain a binary image.

[0053] Step 6): De-noise the binary image

[0054] Although the binary image obtained in step 5) can effectively extract the ridge information, the extracted ridge information will be accompanied by a large amount of noise due to the interference of factors such as cabbage leaves. In order to eliminate these noises, the image is first fine-tuned using the dilation function in morphology, and the processing kernel size is set to 5×5 and the number of iterations is 1. Then, the small dots, burrs and bridges in the image are removed by using a closing operation of first dilation and then corrosion, and then an opening operation of first corrosion and then dilation. The processing kernel size is 40×40, so as to achieve noise removal and compensate for the lost image information at the same time.

[0055] Step 7): Perform edge detection on the denoised ridged cabbage image

[0056] Edge detection can detect the real ridge boundary and remove large blank noise blocks. The contour function (findContours) is used to detect all the outermost contours of the image obtained in step 6), and the elements in the horizontal, vertical and diagonal directions are compressed to obtain a quasi-contour curve. The area contained in the specified contour is further used as a comparison value, and the contour with the largest area in the contour detection is selected as the ridge boundary, and then an all-0 array is created, that is, a completely black image. The area calculation result is then drawn on the all-0 array, and the contour line index is set to -1, that is, the entire contour is drawn. Finally, the contour line color is set to white, that is, B, G, R = 255, and the contour line thickness is set to 1.

[0057] Step 8): Extract the ridge midpoint based on the detected ridge boundary

[0058] Through steps 1) to 7), the image information of ridged cabbage has been extracted. In order to successfully fit the ridge center navigation line, the pixel midpoint point cloud on both sides of the ridge is selected as the reference point for straight line fitting. The image height, i.e., the number of rows, and the width, i.e., the number of columns, are calculated. An empty list is initialized to store coordinate information. A for loop is established to find n pixel points (y1, x1), (y1, x2), ..., (y1, x3) with a pixel value of 255 in each row. n ), (y2, x1)…, (y height , x n ), 2≤n≤6, calculate the average coordinates of each row of pixels, that is And round it off and store it in an empty list as the fitting reference point. Skip this line when looping, and exit the loop after looping height times.

[0059] Step 9): Generate a navigation line based on the extracted ridge midpoint fitting

[0060] Initialize the sliding list to store the slope and intercept of the line, and set the size to 18 storage cells. Determine whether the number of stored point clouds is greater than 1, that is, at least two points are required for straight line fitting;

[0061] Use list analysis to convert all x and y coordinates into NumPy arrays to meet the input requirements of the RANSAC regression model. Create a RANSAC regression model with 100 iterations. Use the RANSAC algorithm to fit the straight line as (slope, intercept). Store (slope, intercept) in a storage cell of the sliding list. When the storage cell of the sliding list is full, sort all the lines in the list by slope from small to large to obtain (slope1, intercept1), (slope2, intercept2), ..., (slope 18 , intercept 18 ), and then use the truncated mean method to intercept (slope9, intercept9), (slope 10 , intercept 10 ), (slope 11 , intercept 11 ), calculate the new straight line as It is recorded as (avg_slope, avg_intercept), and the straight line is output as the navigation line fitted by the improved RANSAC algorithm. Then, the straight line with the smallest slope (slope1, intercept1) in the original list is removed, and the straight line fitted by the next frame of image is added to the sliding list and re-sorted to obtain a new list. The above process is repeated to output a new straight line until the new navigation line of the last frame of image is output.

[0062] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for ridge ditch identification and navigation line extraction for a self-propelled transport platform for ridge-planted vegetables, characterized in that: The specific steps are as follows: Step 1): collect images of ridge-grown vegetables; Step 2): performing boundary smoothing processing on the collected ridge-grown vegetable images; Step 3): Perform color conversion on the ridge vegetable image after boundary smoothing; Step 4): performing color separation on the color-converted ridge-grown vegetable image to obtain a grayscale image; Step 5): Perform threshold segmentation on the color-separated ridge vegetable image to obtain a binary image; Step 6): De-noising the binary image; Step 7): edge detection is performed on the ridge-grown vegetable image after denoising; Step 8): Extract the ridge midpoint according to the detected ridge boundary; Step 9): Generate a navigation line based on the extracted ridge midpoint fitting.

2. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The image acquisition is based on the self-propelled auxiliary transportation platform to collect data from the video segment of the operation scene, decompose the collected video image frame by frame and establish a coordinate system, and use the top left corner of the image as the coordinate origin (0,0), the top right corner coordinate of the image as (width,0), and the bottom left corner coordinate of the image as (0,height); The camera is set at a height of 20 to 100 cm above the ground, with a downward inclination angle of 10 to 90°. The focal length is adjusted so that the camera field of view covers two ridges. Video images of vegetables grown in ridges in the field are captured with a pixel resolution of 1920×1080 and a frame rate of 30 fps.

3. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The boundary smoothing process needs to perform Gaussian blur filtering and smoothing on the image collected in step 1) according to the growth characteristics of the vegetables, thereby obtaining the boundary between the ridge and the vegetable; Set up the second-order Gaussian function equation (1): The Gaussian kernel size is 21×21 and the standard deviation is 21.

4. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The color conversion needs to be based on the color difference between the ridges and the vegetables, and the difference in the lighting conditions at different times in the field. The image obtained after the processing in step 2) is converted into hue (H), saturation (S), and brightness (V) according to the blue (B), green (G), and red (R) color model. The conversion equation is as follows: V=Cmax (R,G,B)′=(R,G,B) / 255 Cmax=max(R′,G′,B′) Cmin=min(R′,G′,B′) Δ=Cmax-Cmin.

5. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The color separation to obtain the grayscale image is to separate the three color channels after the HSV three-color conversion, and then compare and observe the H and S components respectively, and obtain the H component that is not affected by the lighting conditions and has clear ridge boundaries through observation, and finally select the grayscale image of the H component for the next step of processing.

6. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The threshold segmentation to obtain the binary image is to perform threshold segmentation on the H component grayscale image obtained after comparison in step 4), so as to distinguish the vegetables and furrows in the image; First, create a mask and define the grayscale range of interest of the mask as 50 to 150. Then use the mask to enhance the grayscale information of the H component of the grayscale image obtained in step 4). Finally, use the maximum between-class variance method (OTSU) to perform threshold segmentation to obtain a binary image.

7. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The denoising process for the binary image is to fine-tune the image using the dilation function in morphology, and then use a closing operation of dilation followed by corrosion, and an opening operation of corrosion followed by dilation to remove small dots, burrs and small bridges in the image, so as to remove noise and compensate for lost image information at the same time; Set the processing kernel size of the expansion function to 5×5, the number of iterations to 1, and set the processing kernel size of the closing and opening operations to 40×40.

8. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The edge detection is to detect the outermost contours of all ridge vegetable images after denoising in step 6) using the contour function (findContours), and compress the elements in the horizontal, vertical and diagonal directions to obtain a quasi-contour curve, and further use the area contained in the specified contour as a comparison value, and select the contour with the largest area in the contour detection as the ridge boundary, and then create an all-0 array, that is, a completely black image, and then draw the area calculation result on the all-0 array, and set the contour line index to -1, that is, draw the entire contour, and finally set the contour line color to white, that is, B, G, R = 255, and set the contour line thickness to 1.

9. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The ridge midpoint extraction is to find each row of pixels with 255, i.e., white points, by line-by-line scanning, and calculate the midpoint point cloud of each row of white point pixel coordinates as a reference point and then as a straight line fitting object; Calculate the image height, i.e. the number of rows, and width, i.e. the number of columns, as height. Initialize an empty list to store coordinate information. Create a for loop to find n pixel points (y1, x1), (y1, x2), ..., (y1, x3) with a pixel value of 255 in each row. n ), (y2, x1)…, (y height , x n ), 2≤n≤6, calculate the average coordinates of each row of pixels, that is And round it off and store it in an empty list as the fitting reference point. Skip this line when looping, and exit the loop after looping height times.

10. The method for ridge ditch identification and navigation line extraction of a self-propelled transport platform for ridge-planted vegetables according to claim 1, characterized in that: The fitting to generate the navigation line is to use the point cloud extracted in step 8) and improve the RANSAC algorithm to fit the navigation line according to the distribution characteristics of the point cloud; Initialize the sliding list to store the slope and intercept of the line, and set the size to 18 storage cells. Determine whether the number of stored point clouds is greater than 1, that is, at least two points are required for straight line fitting; Use list analysis to convert all x and y coordinates into NumPy arrays to meet the input requirements of the RANSAC regression model. Create a RANSAC regression model with 100 iterations. Use the RANSAC algorithm to fit the straight line as (slope, intercept). Store (slope, intercept) in a storage cell of the sliding list. When the storage cell of the sliding list is full, sort all the lines in the list by slope from small to large to obtain (slope1, intercept1), (slope2, intercept2), ..., (slope 18 , intercept 18 ), and then use the truncated mean method to intercept (slope9, intercept9), (slope 10 , intercept 10 ), (slope 11 , intercept 11 ), calculate the new straight line as It is recorded as (avg_slope, avg_intercept), and the straight line is output as the navigation line fitted by the improved RANSAC algorithm. Then, the straight line with the smallest slope (slope1, intercept1) in the original list is removed, and the straight line fitted by the next frame of image is added to the sliding list and re-sorted to obtain a new list. The above process is repeated to output a new straight line until the new navigation line of the last frame of image is output.

Images