Corn growth early stage row center line extraction method and system based on unmanned aerial vehicle image

Abstract

The application discloses a corn growth initial stage row center line extraction method and system based on unmanned aerial vehicle images. First, unmanned aerial vehicle image data of the corn growth initial stage is acquired, and the image is cut; then, background segmentation processing is performed on the image, and green plants and soil are segmented; then, the number of crop rows is acquired through a vertical projection method; and a micro-ROI (micro region of interest) is used to acquire a centroid point, i.e., a feature point; the feature point is used to remove false feature points through a position clustering method, so that the final accurate feature point is acquired; finally, all the feature points are fitted through a least square linear regression method, so that the row center line is acquired; and the corn missing seedling position and quantity are extracted based on the row center line information, thereby providing technical support for subsequent seedling supplement operations.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a corn growth early stage row center line extraction method and system based on unmanned aerial vehicle images. BACKGROUND

[0002] In recent years, the automation and intelligence of agriculture have attracted widespread attention. As a large agricultural country, China is particularly important in the application of agricultural automation and intelligence. In the process of agricultural production, automatic detection of crop rows is very crucial for precision farming, automatic navigation and agricultural robots, and is widely used in planting, fertilization, plant protection, weeding and harvesting. Based on unmanned aerial vehicle images, accurate detection of crop rows can guide the automatic driving of vehicles to realize row guidance automation, which not only reduces the fatigue of operators, but also improves the positioning accuracy of vehicles to improve agricultural productivity, reduce costs, reduce labor and standardize planting. Therefore, with the development of smart agriculture, the requirements for crop row recognition accuracy and speed are continuously increasing, and the development of reliable and real-time crop row detection methods is of great significance in the field of precision agriculture.

[0003] Accurate crop row center line detection is one of the key problems to realize automatic agriculture. Hough transform, linear regression, spot analysis, stereo vision and horizontal strip are the main methods for crop row center line extraction.

[0004] Hough transform is the most commonly used target detection algorithm at present, and this method is one of the most commonly used machine vision methods for crop row recognition. It can not only identify straight lines, but also find the parameters of graphics in a complex parameter space through the voting step in the algorithm, so as to identify any shape, such as circle and ellipse. It is widely used in image analysis, computer vision and image processing. Linear regression method is used to detect lines fitted to outliers as a method of identifying crop row guidance information. The spot analysis method divides the continuous white pixel area in the image, then calculates the center of gravity in the area, and finally connects the center of gravity to obtain the crop row center line. The stereo vision method is usually suitable for crops and weeds with a significant distance from the ground height, and the height of crops and weeds is different. The horizontal strip method can determine the crop row only by image analysis without segmentation.

[0005] As can be seen from the above, most of the existing research methods take continuous crops as the research object. Due to the influence of external environment and other factors, there will be a phenomenon of crop missing in the crop row in the actual field, which makes it difficult to guarantee the detection accuracy of the crop row center line. In view of the above problems, the present research proposes a missing seedling area detection algorithm and a seedling position supplement algorithm to detect the missing seedling position and provide technical support for automatic seedling supplement. SUMMARY

[0006] Therefore, the present application aims to provide a method and system for extracting row center line in early growth stage of corn based on UAV image.

[0007] To achieve the above object, the present application provides the following technical scheme.

[0008] The method for extracting row center line in early growth stage of corn based on UAV image provided by the present application comprises the following steps.

[0009] Obtaining UAV image in early growth stage of corn, and cutting the image;

[0010] Carrying out background segmentation processing on the image, and segmenting green plants and soil;

[0011] Obtaining the number of crop rows by vertical projection method;

[0012] Obtaining the centroid point, i.e. feature point, by using micro-ROI;

[0013] Removing the pseudo feature points from the obtained feature points by using position clustering method, so as to obtain the final accurate feature points;

[0014] Fitting all the feature points by using linear regression method based on least square, so as to obtain the row center line;

[0015] After obtaining the positions of all the feature points, calculating the seedling deficiency rate by using seedling deficiency area detection algorithm, and accurately obtaining the position of seedling to be supplemented.

[0016] Further, the background segmentation is determined according to the following steps.

[0017] Firstly, the improved green vegetation index ExGG is adopted, and the image is processed by ExGG gray scale processing; then the particle swarm algorithm and OTSU algorithm are combined, the OTSU method is used as the optimization objective function of the particle swarm algorithm, the efficiency of image segmentation is improved, the best threshold value of image segmentation is quickly found, and the original image is converted from a color image to a binary image. Finally, a large number of weed points are removed by linear median filtering.

[0018] Further, the number of crop rows is determined according to the following steps.

[0019] The white pixels are projected in the vertical direction according to the binary image, and the cumulative sum of the white pixels in the vertical direction is calculated by the following method.

[0020]

[0021] where s(j) represents the accumulated sum of white pixels in the vertical direction of the jth column, and I(i,j) is the gray value at point (i,j)

[0022]

[0023] If s(k-1) < m < s(k) or s(k) < m < s(k+1), both of which can be counted, the final crop row number is N = count / 2.

[0024] Further, the acquisition of the micro-ROI is determined according to the following steps:

[0025] The binary image is cut into N parts by column average from left to right, and the feature point image is scanned from bottom to top and left to right in each image matrix. The first white pixel point scanned in each image matrix is the starting point of each column crop row.

[0026] Further, on the basis of determining the starting point, we can determine other feature points according to the basic direction of the crop row

[0027] Further, on the basis of the above binary image being divided into N parts from left to right by column, it is further cut into suitable horizontal strips from bottom to top by row. In order to ensure the accuracy of feature point extraction, each horizontal strip must be composed of more than one pixel row;

[0028] Further, then take an appropriate region above the starting point as the micro-ROI

[18] Then take its centroid point, which is the second feature point;

[0029] Further, the above steps are continuously operated according to the basic direction of the crop row to obtain other feature points;

[0030] Further, the position clustering method is used to remove false feature points according to the following steps:

[0031] The position clustering method divides the binary image into N columns from left to right by column, then finds the initial point according to the order from bottom to top and left to right, and finally removes false feature points according to two conditions of distance constraint and angle constraint to screen out the feature point set meeting the conditions;

[0032] Further, the fitting of all feature points is determined according to the following steps:

[0033] The linear regression least squares method is used to fit the feature point set, so that the sum of squared deviations of each feature point to the fitting straight line is minimized.

[0034] Further, the calculation of the missing rate is determined according to the following steps:

[0035] The position of each pixel point in the image is acquired, the distance between adjacent pixel points is calculated, if greater than the distance threshold, the distance is put into the designated set D, all elements in the set D are added, and the sum is obtained, h is the row height of the crop row, and the calculation formula of the missing rate is shown:

[0036]

[0037] Further, the position of the replanted seedling is determined according to the following steps:

[0038] If the value range of D[i] / t1 is (2, 3), it indicates that one seedling is missing between the two seedlings, and the missing seedling can be replanted on the crop line between the two seedlings; if D[i] / t1 is greater than 3, it indicates that (D[i] / t1-1) seedlings are missing between the two seedlings, and the missing seedlings can be evenly replanted on the crop line between the two seedlings.

[0039] Wherein, t1 represents the threshold of the distance between two crops.

[0040] The method and system for extracting a row center line of corn in an early growth stage based on unmanned aerial vehicle images provided by the application include a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to realize the steps of the above method.

[0041] The beneficial effects of the application are as follows:

[0042] The method and system for extracting a row center line of corn in an early growth stage based on unmanned aerial vehicle images provided by the application, compared with Hough transformation, through the accumulation of votes and the detection of peak values in the parameter space, high weed pressure can lead to false detection of the row crop line, and even unable to detect the line, but the method of the present application can accurately obtain the row center line. The method uses a vertical projection method to obtain the number of crop rows, and then uses a position clustering algorithm of distance constraint and angle constraint to remove false feature points, and finally screens out accurate feature points to be fitted

[0043] Other advantages, objects, and features of the application will be set forth in part in the following specification, and in part will become apparent to those skilled in the art upon examination of the following specification, or can be learned from practice of the application. The objects and other advantages of the application can be realized and attained by the methods and systems particularly pointed out in the specification. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to make the objects, technical solutions and beneficial effects of the application clearer, the application provides the following drawings for illustration:

[0045] Figure 1The overall method flowchart of the present application.

[0046] Figure 2 The orthographic image of the research area by the unmanned aerial vehicle and the picture of the corn in flight in the specific embodiment of the present application.

[0047] Figure 3 The field landscape picture and close-up picture in the specific embodiment of the present application.

[0048] Figure 4 The effect picture of background segmentation in the specific embodiment of the present application.

[0049] Figure 5 The vertical projection picture in the specific embodiment of the present application.

[0050] Figure 6 The schematic diagram of the centroid point obtained by using the micro region of interest in the specific embodiment of the present application.

[0051] Figure 7 The schematic diagram of the pseudo feature point removed by using the position clustering method in the specific embodiment of the present application.

[0052] Figure 8 The row center line fitting picture in the specific embodiment of the present application.

[0053] Figure 9 The flowchart of the calculation of the seedling deficiency rate and the reseeding process in the specific embodiment of the present application.

[0054] Figure 10 The effect picture of the determination of the reseeding position in the specific embodiment of the present application.

[0055] Figure 11 The comparison picture of the crop lines drawn by the research algorithm and the comparison algorithm in the specific embodiment of the present application. DETAILED DESCRIPTION

[0056] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not limiting to the present application.

[0057] Embodiment 1

[0058] The background segmentation method provided in the embodiment is specifically as follows:

[0059] Firstly, the image is subjected to ExGG grayscale processing to separate the green plants (including the corn seedlings and the weeds) and the soil, and the case is specifically performed according to the following steps:

[0060] ExGG = ExG * g (1)

[0061] ExG = 2g - b - r (2)

[0062] The value range of r, g, b is 0-255, and the calculation process is as follows:

[0063]

[0064] The value of R', G', B' in the formula is the normalized RGB coordinate in the range of [0, 1], and the specific calculation method is as follows:

[0065]

[0066] Where Rm = Gm = Bm = 255

[0067] Then, combined with PSO and OTSU double threshold processing, the processing is specifically performed according to the following steps:

[0068] The OTSU method is used as the optimization objective function of the PSO particle swarm algorithm, which can quickly find the best threshold of image segmentation and improve the efficiency of image segmentation. The particle updates its speed and position by the following formula.

[0069] Vi = Vi + rand() × (pbesti - Xi) + c2 × rand() × (gbesti - Xi) (5)

[0070] Xi = Xi + Vi (6)

[0071] Where i = 1, 2,..., N, N is the total number of particles in the group; Vi is the speed of the particle, and the maximum value of Vi Vmax > 0, if Vi > Vmax, then Vi = Vmax; rand() is a random number between (0, 1); Xi is the current position of the particle; c1, c2 are learning factors, usually c1 = c2 = 2; based on the above formula (5) and (6), the standard form of PSO is formed as follows:

[0072] vi = w × vi × rand() × (pbesti - xi) + c2 × rand() × (gbesti - xi) (7)

[0073] Where w is called inertia factor, and its value is non-negative. If its value is large, the global optimization ability is strong and the local optimization ability is weak; if its value is small, the global optimization ability is weak and the local optimization ability is strong. Dynamic w can obtain better optimization results than fixed value.

[0074] Finally, the median filter is used to remove the pseudo feature points with strong interference force;

[0075] Example 2

[0076] Figure 1 The overall method flowchart of the present application. Figure 2 The orthographic image of the research area by the unmanned aerial vehicle and the picture of the corn in flight in the specific embodiment of the present application. Figure 3 The field landscape picture and the close-up picture in the specific embodiment of the present application. Figure 4 The effect picture of background segmentation in the specific embodiment of the present application. Figure 5 The vertical projection picture.

[0077] The vertical projection method provided in the embodiment has the following specific steps:

[0078] The white pixels in the binary image are projected in the vertical direction, and the cumulative sum of the white pixels in the vertical direction is calculated by the following method:

[0079]

[0080] Where s(j) represents the cumulative sum of the white pixels in the vertical direction of the jth column, and I(i, j) is the gray value at the point (i, j)

[0081]

[0082] The relationship between s(j) and the average value m is determined, and if s(k-1) < m < s(k) or s(k) < m < s(k+1), both of the two cases can be counted count, and the final crop row number is N = count / 2.

[0083] Embodiment 3

[0084] The method for obtaining the feature points provided in the embodiment has the following specific steps:

[0085] First, the starting point is determined, which is performed according to the following steps:

[0086] The binary image is divided into N parts from left to right by column, and for each part, the white pixel points are scanned in the order of from bottom to top and from left to right until the column with white pixel points is scanned, and the starting point is located in this column. The specific starting point position is located at the center position of all white pixel points in this row arranged from left to right.

[0087] Then, other feature points are determined, and the determination steps are performed according to the following steps:

[0088] Step 1: The binary image is cut into N parts from left to right by column, and then cut into N1 horizontal bands from bottom to top by row, and each horizontal band is composed of h / N1 rows of pixels. The finer the horizontal band is divided, the more accurate the feature points are extracted.

[0089] Step 2: Take a micro-ROI in each horizontal strip, and the height of the micro-ROI and the horizontal strip are the same, h / N1, and the width Lw is selected to make the micro-ROI contain all the white pixels in the horizontal strip, to ensure the accurate calculation of the subsequent centroid.

[0090] Step 3: Take Lw / 2 pixels to the left and right of the obtained starting point as the width, and take h pixels as the height, to construct a micro-ROI (the micro-ROI contains most crop feature points). The specific placement position of the first micro-ROI is as shown in Figure 3 The formula for calculating the centroid point is as follows:

[0091]

[0092] Where xi, yi represents the coordinates of the segmented white pixels in the micro-ROI, A represents the number of all white pixels in the micro-ROI, and the expression reduces the influence of isolated pixels.

[0093] Step 4: If there is no white pixel point in the micro-ROI, it is regarded as a void, and the centroid of the void is the centroid of the micro-ROI.

[0094] Step 5: To obtain the next centroid point, we need to obtain the starting point of the next micro-ROI (i.e. the red cross point above in (a)), and the specific steps are as follows: Figure 6 We can preliminarily obtain the coordinate information of all white pixel points on the binary image, and then preliminarily fit each column of crop points, and the intersection point of the fitted straight line and the upper edge of the previous micro-ROI is the starting point of the next micro-ROI. Place other micro-ROIs, and their starting points are the intersection points with the fitted straight line.

[0095]

[0096] Step 6: Repeat steps 2, 3, 4, and 5 above until all the centroid points of the column are obtained, as shown in Figure 6 (b). Then repeat the operation for other crop rows until all the centroid points on the entire image are obtained.

[0097] Example 4:

[0098] The position clustering method and least square method for fitting a straight line provided in this embodiment for removing false feature points, and the specific steps are as follows:

[0099] As shown in Figure 7 ​As shown, first, the position clustering method uses two conditions of distance constraint and angle constraint to divide the image into N columns from left to right, and then finds the initial point in the order from bottom to top and from left to right, and then screens the feature point set meeting the conditions according to the two conditions of distance constraint and angle constraint.

[0100] After determining the feature point set, the feature points are assigned to each crop row, and the feature points are fitted to a straight line. The fitting method used in this study is a linear regression method based on least squares, which fits the feature point set so that the sum of the squared deviations of each feature point to the fitted straight line is minimized. The final fitting straight line is shown in Figure 8 . The comparison diagram of the crop line fitted by the algorithm of this study and the comparative algorithm is shown in Figure 11 .

[0101] Example 5:

[0102] The missing seedling area detection algorithm and the method for determining the reseeding position provided in this embodiment are as follows:

[0103] The flowchart of the missing seedling area detection algorithm is shown in Figure 9 (a), and the specific steps are as follows:

[0104] Step 1: initialize the crop row number counter i = 0, d represents the distance between the feature point and the current point, Td represents the distance threshold, P 0i represents the starting point of the i-th crop row;

[0105] Step 2: point P 0i belongs to the n-th feature point from top to bottom of the crop row in this column, let n = n(P 0i );

[0106] Step 3: find the (n-1)-th feature point from top to bottom, let n = n-1;

[0107] Step 4: calculate the distance d, if d > Td, insert d into the set D, and insert the feature point and the current point into the sets P1 and P2 respectively, and then execute step 3 again; if the distance d < Td, repeat step 3;

[0108] Step 5: judge whether n is equal to 1, if not, jump to step 3; if yes, the i-th crop row missing detection algorithm is completed, and jump to step 6;

[0109] Step 6: let i = i+1, if the current crop row counter i is equal to N, execute the last step step 7; otherwise, jump to step 2 to continue execution, and continue to study the missing situation of other crop rows;

[0110] Step 7: add all elements in the set D, and the sum is sum; h is the row height of the crop row, and the calculation formula of the missing rate is as follows:

[0111]

[0112] The flow chart of the seed filling position algorithm is shown in Fig. 2 (b), and the seed filling map is shown in Fig. 2 (c). The specific method of seed filling is as follows: Figure 9 Figure 10

[0113] In the calculation of the missing seed rate, we can obtain the set P1, P2 of the two adjacent points above and below the missing seed position, the distance set D (the set composed of the difference between the corresponding elements in the set P1, P2), and the lengths of the three sets P1, P2, D are the same, all being L. Next, the seed filling position algorithm of the present research is specifically described as follows:

[0114] Step 1: initialize i = 1, and t1 represents the threshold value of the distance between two crops.

[0115] Step 2: if the value range of D[i] / t1 is (2, 3), it indicates that one seed is missing between P1[i] and P2[i], and the missing seed is planted on the crop line between the two seeds; if D[i] / t1 is greater than 3, it indicates that (D[i] / t1-1) seeds are missing between P1[i] and P2[i], and the missing seeds are evenly planted on the crop line between P1[i] and P2[i].

[0116] Step 3: make i = i+1, if the current i is not equal to L+1, jump to step 2; if the current i is equal to L+1, it indicates that the seed filling of the current crop row is completed.

[0117] The above-described embodiments are only the preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. The equivalent substitutions or transformations made by the person skilled in the art on the basis of the present application are within the protection scope of the present application. The protection scope of the present application is subject to the claims.​​

Claims

1. A method for extracting a row centerline in an early growth stage of corn based on images of a UAV, characterized in that: The method comprises the following steps: Unmanned aerial vehicle image of corn in early growth stage is acquired, and the image is cut; Green plants and soil are segmented by background segmentation processing of the image; The number of crop rows is obtained by vertical projection method; Micro-ROI is used to obtain the centroid point, i.e., the feature point; The feature points obtained are removed by position clustering method to obtain the final accurate feature points; Linear regression method based on least square is used to fit all the feature points to obtain the row center line. The micro-ROI is used to obtain the centroid point according to the following steps: The binary image obtained by the background segmentation processing is divided into N parts from left to right according to column, for each part, the white pixel points are scanned from bottom to top and from left to right in turn until the column with white pixel points is scanned, and the starting point is located in the column, and the specific starting point is located at the center position of all white pixel points arranged from left to right in the row; According to the starting point, the pixel coordinates of the crop points are preliminarily fitted to obtain a fitting straight line, the extension direction of the straight line is taken as the basic direction of the crop row, and other feature points are determined according to the intersection of the micro-ROI and the fitting straight line based on the starting point; On the basis of the binary image divided into N parts from left to right according to column, the binary image is further divided into suitable horizontal strips from bottom to top according to row, and in order to ensure the accuracy of feature point extraction, each horizontal strip is composed of more than one pixel row; According to the starting point, a rectangular region with the same height as the horizontal strip and containing all the white pixels in the horizontal strip is selected as the micro-ROI, and the centroid point of the rectangular region is taken as the second feature point; Finally, the above steps are continuously operated according to the extension direction of the straight line obtained by fitting the crop points in the column to obtain other feature points. The position clustering method is used to remove the pseudo feature points according to the following steps: According to the position clustering method, the image is divided into N columns from left to right according to column, and the initial point is found from bottom to top and from left to right, and finally the feature point set meeting the distance constraint and angle constraint conditions is screened out. 2.The unmanned aerial vehicle image-based corn growth initial stage row center line extraction method of claim 1, wherein: The background segmentation is determined according to the following steps: Firstly, the improved green vegetation index ExGG is used to perform ExGG gray processing on the image; then the particle swarm algorithm and the OTSU algorithm are combined, the OTSU method is taken as the optimization objective function of the particle swarm algorithm, the efficiency of image segmentation is improved, the best threshold value of image segmentation is quickly found, and the original image is converted from a color image to a binary image; finally, a large number of impurity points are removed by linear median filtering.

3. The method for extracting the center line of maize rows in the early stage of growth based on UAV imagery as described in claim 1, characterized in that: The number of crop rows is obtained by vertical projection method according to the following steps: The white pixels are projected down in the vertical direction, and the cumulative sum of the white pixels in the vertical direction is calculated as follows: Wherein s(j) represents the cumulative sum of the white pixels in the vertical direction of the jth column, and I(i,j) is the gray value of point (i,j). If s(k-1)<m<s(k) or s(k)<m<s(k+1), both of which can be counted as count, and the final crop row number is N=count / 2. 4.The method of claim 1, wherein the method further comprises: determining a center line of a row of corn plants in the image based on the identified first and second points. The fitting of all feature points by the linear regression method based on least square is determined according to the following steps: The method based on linear regression is used to fit the feature point set by least square method, so that the deviation square sum of each feature point to the fitting straight line is minimized.

5. The method of calculating the stand mortality according to any one of claims 1-4, characterized in that: The method comprises the following steps: According to the position of each pixel point in the acquired image, the distance between adjacent pixel points is calculated, if greater than the distance threshold, the distance is put into the designated set D, all elements in the set D are added up, and the sum is sum; h is the row height of the crop row, and the formula for calculating the missing rate is shown: The position of the seedling is determined according to the following steps: If the value range of D[i] / t1 is (2, 3), it indicates that one seedling is missing between the two seedlings, and the missing seedling needs to be planted on the crop line between the two seedlings; if D[i] / t1 is greater than 3, it indicates that (D[i] / t1-1) seedlings are missing between the two seedlings, and the missing seedlings need to be evenly planted on the crop line between the two seedlings; wherein, t1 represents the threshold value of the distance between two crops.

6. A system for extracting a row centerline in the early growth stage of corn based on UAV images, characterized in that: A computer program product comprising a memory, a processor and a computer program stored on the memory and loadable on the processor, the processor implementing any of the methods of claims 1-4 when executing the program.

7. A corn growth early stage reseeding system based on unmanned aerial vehicle images, characterized in that: A computer program product comprising a memory, a processor and a computer program stored on the memory and loadable on the processor, the processor implementing the method of claim 5 when executing the program.

Images