Field crop row region instance segmentation method based on line mask scanning and boundary finding
Through the field crop row area example segmentation method based on linear mask scanning, the problem of inaccurate crop row identification in hybrid rice field navigation is solved, and fast and accurate crop row area segmentation is achieved, which improves the accuracy of field navigation and Real-time, suitable for precision agricultural operations.
Patent Information
- Application Number
- CN202111567408.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-21
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-21
AI Technical Summary
The prior art is inaccurate in the identification of crop rows in hybrid rice field navigation, resulting in high labor intensity and low efficiency of artificial pollination, and semantic segmentation technology based on convolutional neural networks has challenges in model training and real-time performance.
Using the field crop row area example segmentation method based on linear mask scanning, images are collected through color cameras, grayscale and image perspective imaging rules and implicit information are used to quickly and accurately realize crop row area segmentation, combined with Linear-like mask scanning and rotation scanning techniques generate scanning curve clusters to determine crop row area lines and boundary lines.
It realizes efficient and accurate crop row area segmentation on grayscale maps, without the need for complex image preprocessing, takes into account the accuracy and real-time nature of the recognition process, and is suitable for precise agricultural operations such as field navigation.
Smart Images

Figure CN114266784B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of agricultural automation, and specifically relates to image processing technology, and in particular to a field crop row region instance segmentation method based on linear mask scanning and boundary finding. Background Art
[0002] Hybrid rice seeds are the material basis of hybrid rice production. At present, in the field hybrid rice seed production process, the transplanting operation has been basically mechanized and partially automated. Mechanized transplanting makes the seedlings have better linearity and more regular row spacing, which provides convenience for subsequent field automation operations.
[0003] Rice is a strictly self-pollinating crop, and the natural hybridization rate is extremely low in the natural environment. Therefore, a large amount of manpower is required to assist in pollination during the hybrid rice seed production process. The process of artificial assisted pollination has problems such as high labor intensity and low operation efficiency, which hinders the further development of seed production productivity.
[0004] With the continuous development of precision agriculture technology, the use of automated agricultural machinery to replace manual pollination operations has become the best choice to solve the above problems. The premise of realizing the automation of agricultural machinery operations is to realize the automatic navigation of agricultural machinery in the field. Visual sensors have gradually become a hot sensor in the research of agricultural machinery automatic navigation systems due to their advantages such as low price, flexible application and rich information collection. The core of visual navigation technology lies in the identification of crop rows and the planning of navigation paths.
[0005] At present, most of the field crop row recognition based on traditional image processing methods use Hough transform, vertical projection or linear fitting methods to extract the center line of the crop row, but such methods are not applicable to the field navigation task of hybrid rice during the pollination period. The reason is that the row spacing between the male and female rows in the hybrid rice seed production field is extremely small, and the crop row recognition algorithm needs to correctly judge the position information of the crop row boundary to avoid the occurrence of pollination machinery crushing the crop.
[0006] In recent years, with the rapid development of convolutional neural networks, semantic segmentation technology based on convolutional neural networks has gradually been applied to the task of field crop row recognition. Semantic segmentation technology has the technical advantages of strong robustness and applicability, and can complete the segmentation of crop row areas. However, in the process of model training, a large number of crop row image samples need to be collected, and the sample labeling process is time-consuming and laborious. In addition, in the process of model recognition, it is difficult to balance recognition accuracy and real-time performance. Summary of the invention
[0007] In view of the shortcomings of the prior art, the present invention proposes a method for instance segmentation of field crop row areas based on linear mask scanning and boundary finding. It utilizes the image perspective imaging rules and implicit information of crop row arrangement to quickly and accurately achieve instance segmentation of crop row areas during the pollination period, providing a basis for realizing mechanical automation of field pollination operations.
[0008] The object of the present invention is to provide a method for segmenting field crop row regions based on linear mask scanning and boundary finding in view of the deficiencies of the prior art.
[0009] The object of the present invention is achieved through the following technical solution: a method for segmenting field crop row regions based on line mask scanning and boundary finding. The method comprises the following steps:
[0010] S01. Use a color camera to collect the original image of the crop row. During the collection process, the camera is at an angle of 15° to 45° with the horizontal direction and is about 1 to 3 meters above the crop row, so that the crop rows in the original image are roughly symmetrically distributed;
[0011] S02. graying the original image into a grayscale image;
[0012] S03. judging whether each pixel row is a foreground pixel row or a background pixel row according to the relative size relationship between the sum of pixel values of each pixel row in the edge pixel column on the left or right side of the grayscale image; determining the main transition point position in a series of transition points from the foreground pixel row to the background pixel row according to the relative size relationship between the length values of the continuously distributed segments of the background pixel row in the edge pixel column; and dividing the grayscale image into a plurality of horizontal strip-shaped sub-images having the same crop row arrangement pattern from the main transition position;
[0013] S04. Use the sub-image in the horizontal direction Scanning a linear mask and generating a cluster of scanning curves , according to the scanning curve cluster Upper feature point group The relative position relationship of several main feature points in the original image is calculated to obtain the vanishing point of the crop row in the original image;
[0014] S05. Using the vanishing point The linear mask is used to rotate and scan the grayscale image to generate a cluster of scanning curves. , according to the scanning curve cluster Upper feature point group The relative size relationship of the contour height values at several main feature points in the image is used to calculate the crop row area line and the area boundary line in the original image;
[0015] S06. Perform instance segmentation on the crop row area in the original image according to the relative position relationship between the crop row area line and the area boundary line.
[0016] Furthermore, in step S04:
[0017] (1) The line-like mask scanning path starts at the left or right edge of the sub-image and ends at the opposite edge;
[0018] (2) The length of the line-like mask is the number of pixel rows corresponding to the sub-image, and the width is 1 pixel;
[0019] (3) The scanning curve cluster It includes three smoothed scanning curves corresponding to each sub-image, which are respectively a sub-image scanning area pixel value sum distribution curve, a sub-image scanned area pixel value proportion curve and a sub-image unscanned area pixel value proportion curve.
[0020] Furthermore, in step S05:
[0021] (1) The line-like mask scanning path starts at the upper edge point on the left or right side of the grayscale image and ends at the upper edge point on the opposite side;
[0022] (2) The length of the linear mask is the distance from the vanishing point to the edge pixel of the grayscale image on the current scanning path, and the width is 1 pixel;
[0023] (3) The scanning curve cluster It includes a distribution curve of the sum of pixel values in the smoothed grayscale image scanning area and a distribution curve of the change rate of the sum of pixel values in the smoothed grayscale image scanning area.
[0024] Furthermore, the crop row is a male hybrid rice row during the pollination period.
[0025] The beneficial effect of the present invention is that the method of the present invention realizes instance segmentation of crop row areas on grayscale images without the need for complex image preprocessing steps, takes into account the accuracy and real-time performance of the recognition process, is suitable for precision agricultural operations such as field navigation, and has high promotion and application value. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] FIG. 1 is a schematic diagram showing the flow of a method for segmenting a field crop row region based on line mask scanning and boundary finding in an example of the present invention.
[0027] Figure 2 Schematic diagram of a crop row image in an example of the present invention.
[0028] Figure 3 It is a schematic diagram of an image of a crop row after grayscale processing in an example of the present invention.
[0029] Figure 4 A schematic diagram of obtaining cutting line positions of horizontal strip sub-images having the same crop row arrangement pattern in an example of the present invention.
[0030] Figure 5 It is a schematic diagram of the process of solving the endpoints of the main crop row area at the bottom center of the sub-image by horizontally scanning the sub-image using a linear mask in an example of the present invention.
[0031] Figure 6 Schematic diagram of the process of solving the vanishing point of a crop row in an example of the present invention.
[0032] Figure 7 This is a schematic diagram of the process of solving the crop row area lines and area boundary lines by using a linear mask starting from a vanishing point to perform rotational scanning on a grayscale image in an example of the present invention.
[0033] Figure 8 It is a schematic diagram of a distribution curve of pixels and values in a scanning area and a distribution curve of the change rate of pixels and values in a scanning area generated when a grayscale image is rotationally scanned using a linear mask starting from a vanishing point in an example of the present invention.
[0034] Fig. 9 It is a schematic diagram of the solution process of obtaining the crop row region true value line and the region boundary true value line according to the curve peak profile height classification in an example of the present invention.
[0035] Fig.10 It is a schematic diagram of the crop row region true value line and the region boundary true value line in an example of the present invention.
[0036] Fig.11 Schematic diagram of the segmentation result of the crop row area in the example of the present invention. DETAILED DESCRIPTION
[0037] The specific implementation methods of the invention are further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and cannot be used to limit the protection scope of the present invention. The following embodiments select the image of a hybrid rice crop row under natural light strips during pollination as an example for detailed description.
[0038] Figure 1 FIG. 1 is a schematic diagram showing a process of a method for segmenting a field crop row region instance based on line mask scanning and boundary finding provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:
[0039] S01. Use a color camera to collect the original image of the crop row. During the collection process, the camera is at an angle of 15°~45° to the horizontal direction and is about 1~3m above the crop row. The crop rows in the original image are roughly symmetrically distributed. The collected original image is as follows: Figure 2 As shown, the image resolution is 640*360.
[0040] S02. Use grayscale factor The original image is grayed out into a grayscale image, where R is the pixel value of the red channel of the original image, and G is the pixel value of the green channel of the original image. The obtained grayscale image is as follows: Figure 3 shown.
[0041] S03, dividing the grayscale image into a number of horizontal strip-shaped sub-images with the same crop row arrangement pattern, the dividing process is as follows:
[0042] (1) Take the 10 pixel columns on the left edge of the grayscale image, corresponding to Figure 4 The image selected in the solid rectangular frame in (part b) is calculated by summing the pixel values of each row of the selected image to generate the following Figure 4 The dashed line in (part a) shows the variation curve of the sum of row pixel values.
[0043] (2) According to the relative size of the sum of row pixel values, the rows whose sum of row pixel values is greater than or equal to 10% of the sum of the maximum row pixel values are set as foreground pixel rows, otherwise they are set as background pixel rows. The following is generated based on the judgment results: Figure 4 The row category change curve represented by the solid line in (part a).
[0044] (3) Calculate the length of the continuous distribution segment of the background pixel row on the row category change curve. If the length of the continuous distribution segment is greater than the set threshold, the image segment is considered to be the background image, and the starting pixel row position of the image segment is taken as the sub-image dividing line position. The judgment threshold is set according to the original image height, which is set to 50 pixels here. The selected background pixel row continuous distribution segment is Figure 4 The part selected by the dotted rectangle in (part a) has the sub-image dividing line determined as Figure 4 (Part b) straight dash-dot line.
[0045] S04, use the sub-image in the horizontal direction Scanning a linear mask and generating a cluster of scanning curves , according to the scanning curve cluster Upper feature point group The relative position relationship of several main feature points in the original image is calculated to obtain the vanishing point of the crop row in the original image. The specific steps are as follows:
[0046] (1) First, generate the required Linear mask, here The length of the linear mask is the number of pixel rows of the corresponding sub-image, the width is 1 pixel, and the scanning path starts at the left edge of the sub-image and ends at the right edge of the sub-image. Schematic diagram of linear mask and scanning direction are Figure 5 Indicated by the vertical solid line and arrow in (part a).
[0047] (2) The scanning process is performed once, and three corresponding scanning curves are generated for each sub-image during the scanning process, namely, the sum distribution curve of the pixel values in the scanned area of the sub-image, the pixel value ratio curve of the scanned area of the sub-image, and the pixel value ratio curve of the unscanned area of the sub-image, which together constitute a scanning curve cluster. , where the pixel value ratio is equal to the ratio between the sum of the pixel values in the image area considered in the current calculation process and the area value of the image area. The original graphs of the three curves are Figure 5 As shown by the dashed lines in (b, c, and d), the three curves after smoothing are Figure 5 Shown as solid lines in (parts b, c, and d).
[0048] (3) Search for characteristic point groups on three smooth curves . Search for the peak points on the sub-image scanning area pixel value sum distribution curve and include them in the feature point group In the calculation, the peak profile height is calculated and used as Figure 5 The straight dot-dash line shown in (part b) indicates that the curve valley points are searched on the pixel value ratio curve of the scanned area of the sub-image and the pixel value ratio curve of the unscanned area of the sub-image and included in the feature point group In the calculation, the trough profile height is calculated and used as Figure 5 The straight lines shown in (parts c and d) are indicated by dot-dash lines.
[0049] (4) Search for the peak point corresponding to the maximum value of the peak contour height on the sub-image scanning area pixel value sum distribution curve, and take it as the main feature point of the curve, denoted as P1, and Figure 5 (Part b) is marked with a downward triangle symbol.
[0050] (5) On the pixel value ratio curve of the scanned area of the sub-image, search for the first trough point to the right of point P1 and take it as the main feature point of the curve, denoted as P2, and Figure 5 (Part c) is marked using the upper triangle symbol.
[0051] (6) On the pixel value ratio curve of the unscanned area of the sub-image, search for the first trough point on the left side of point P1 and take it as the characteristic point of the curve, denoted as P3, and Figure 5 (Part d) is marked with an upper triangle symbol.
[0052] (7) The left and right endpoints of the main crop row area at the bottom center of the sub-image are Figure 5 As shown in part (a), the left and right endpoints are indicated by crosses. The horizontal coordinate values correspond to the horizontal coordinate values of the main feature points P3 and P2. The vertical coordinate values of the left and right endpoints are equal to the row coordinate values of the position of the lower edge pixel row in the grayscale image when the sub-image is not segmented. The left endpoint position coordinate is recorded as , and put it into an array In the example, the coordinates of the right endpoint are , and put it into an array Among them , is the number of sub-graphs.
[0053] (8) Use the Random sample consensus (RANSAN) method to , Linear fitting is performed on the data points in to obtain two fitting lines, such as Figure 6 As shown by the solid line in the figure, the intersection of the two straight lines is Figure 6 As shown by the middle dot, the intersection is the vanishing point of the crop row in the image.
[0054] S05. Use the The linear mask is used to rotate and scan the grayscale image to generate a cluster of scanning curves. , according to the scanning curve cluster Upper feature point group The relative size relationship of the contour height values at several main feature points in the image is calculated to obtain the crop row area line and area boundary line in the original image. The specific calculation steps are as follows:
[0055] (1) First generate the grayscale image required for scanning Linear mask, here The scanning path of the linear mask starts at the upper edge point on the left side of the grayscale image and ends at the upper edge point on the right side. The length is the distance from the vanishing point to the edge pixel point of the grayscale image on the current scanning path, and the width is 1 pixel. Schematic diagram of linear mask and scanning direction are Figure 7 Indicated by solid line and arrow.
[0056] (2) The scanning process is performed once, and two corresponding scanning curves are generated for the grayscale image during the scanning process, namely, the distribution curve of the sum of pixel values in the grayscale image scanning area and the distribution curve of the change rate of the sum of pixel values in the grayscale image scanning area. The two curves together constitute a scanning curve cluster. The two smoothed curves are Figure 8 Shown as solid lines in (parts a and b).
[0057] (3) Searching for characteristic point groups on two smooth curves . Search for the peak points on the grayscale image scanning area pixel value sum distribution curve and include them in the feature point group In the calculation, the peak profile height is calculated and used as Figure 8 The straight dot-dash line shown in (part a) indicates that the peak points and valley points of the curve are searched on the distribution curve of the sum of pixel values in the grayscale image scanning area and included in the feature point group In the calculation, the peak profile height and the trough profile height are calculated and used as follows Figure 8 The straight line shown in (part b) is represented by a dot-dash line.
[0058] (4) The peak points found on the distribution curve of the sum of pixel values in the grayscale image scanning area are classified into two categories according to the relative size of their corresponding contour height values. The classification results are as follows: Fig. 9 As shown, Fig. 9 The horizontal thick dashed line in the middle is the classification line of the two types of feature points. The feature point with the larger contour height value is taken as the main feature point. The coordinate value of the corresponding crop row area line in the grayscale image can be obtained according to the horizontal coordinate value of the main feature point. Fig.10 Indicated by thick solid line.
[0059] (5) The peak points and trough points found on the grayscale image scanning area pixel value sum change rate distribution curve are classified into two categories according to the relative size of their corresponding contour height values. The classification method is the same as above. The feature points with larger contour height values are selected as the main feature points. The coordinate values of the corresponding crop row area boundary lines in the grayscale image can be obtained according to the horizontal coordinate values of the main feature points. Fig.10 Indicated by thin solid line.
[0060] S06, performing instance segmentation on the crop row region in the original image according to the relative position relationship between the crop row region line and the region boundary line. The specific steps are as follows:
[0061] The positions of the crop row area lines and area boundary lines are judged. Starting from the first area line, if there are adjacent area boundary lines on both sides, it is determined that the crop row is detected in the area. The area is segmented by the two adjacent area boundary lines. The segmentation results are shown in the figure. Fig.11 shown.
[0062] The embodiments described above can be further combined or replaced, and the embodiments are only descriptions of preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various changes and improvements made by ordinary technicians in this field to the technical solution of the present invention belong to the protection scope of the present invention. The protection scope of the present invention is given by the attached claims and any equivalents thereof.
Claims
1. A method for segmenting field crop row regions based on linear mask scanning and bounding, characterized in that: The following steps are involved: S01. Use a color camera to collect the original image of the crop row. During the collection process, the camera is at an angle of 15° to 45° with the horizontal direction and is 1 to 3 meters above the crop row, so that the crop rows in the original image are symmetrically distributed; S02. graying the original image into a grayscale image; S03. According to the relative size relationship between the sum of pixel values of each pixel row of the edge pixel column on the left or right side of the grayscale image, determine whether each pixel row is a foreground pixel row or a background pixel row; according to the relative size relationship between the length values of the continuously distributed segments of the background pixel row in the edge pixel column, determine the main jump point position among all the jump points from the foreground pixel row to the background pixel row; and divide the grayscale image into a plurality of horizontal strip-shaped sub-images having the same crop row arrangement pattern from the main jump point position; S04. Use the sub-image in the horizontal direction Scanning a linear mask and generating a cluster of scanning curves , according to the scanning curve cluster Upper feature point group The relative position relationship of several main feature points in the original image is calculated to obtain the vanishing point of the crop row in the original image; S05. Using the vanishing point The linear mask is used to rotate and scan the grayscale image to generate a cluster of scanning curves. , according to the scanning curve cluster Upper feature point group The relative size relationship of the contour height values at several main feature points in the original image is used to calculate the crop row area line and the area boundary line in the original image; S06. Perform instance segmentation on the crop row area in the original image according to the relative position relationship between the crop row area line and the area boundary line.
2. The method according to claim 1, characterized in that In step S04: (4.1) The line-like mask scanning path starts at the left or right edge of the sub-image and ends at the opposite edge; (4.2) The length of the line-like mask is the number of pixel rows corresponding to the sub-image, and the width is 1 pixel; (4.3) The scanning curve cluster Including three smoothed scanning curves corresponding to each sub-image, which are respectively a sub-image scanning area pixel value sum distribution curve, a sub-image scanned area pixel value proportion curve and a sub-image unscanned area pixel value proportion curve; (4.4) The characteristic point group It includes the peak points of the curve on the distribution curve of the sum of pixel values in the sub-image scanning area, and the trough points of the curve on the pixel value proportion curve of the sub-image scanned area and the pixel value proportion curve of the sub-image unscanned area.
3. The method according to claim 1, characterized in that In step S05: (5.1) The line-like mask scanning path starts at the upper edge point on the left or right side of the grayscale image and ends at the upper edge point on the opposite side; (5.2) The length of the linear mask is the distance from the vanishing point to the edge pixel of the grayscale image on the current scanning path, and the width is 1 pixel; (5.3) The scanning curve cluster including a distribution curve of the sum of pixel values in the smoothed grayscale image scanning area and a distribution curve of the change rate of the sum of pixel values in the smoothed grayscale image scanning area; (5.3) The characteristic point group It includes the curve peak points on the distribution curve of the sum of pixel values in the grayscale image scanning area, and the curve peak points and trough points on the distribution curve of the sum of pixel values in the grayscale image scanning area.
4. The method according to claim 1, characterized in that: The crop line is a hybrid rice male line during the pollination period.
