A cotton terminal bud positioning method based on vector calibration

Abstract

The present invention discloses a cotton terminal bud positioning method based on vector calibration, including: performing leaf segmentation on the collected cotton image, extracting the image area of ​​each leaf, and further obtaining the diagonal length, leaf base point and center point of each leaf image area; connecting the center point and leaf base point in the same leaf image area respectively, and calculating the slope and constant term of the obtained line segment; extending the connecting line segment of the center point and leaf base point in the same leaf image area in a directed manner, and the extension line terminates at the re-determined updated leaf base point coordinates to obtain the positioning line segment; obtaining all intersection points of the obtained positioning line segment and obtaining the distance matrix between them; deduplicating the distance matrix, and all the intersection points obtained after deduplication are the coordinate positioning of all the terminal buds in the cotton image. The method of the present invention has good accuracy and rapid detection capability in complex target crop scenes.

Description

Technical Field

[0001] The invention relates to the technical field of crop automatic identification and positioning, and in particular to a cotton terminal bud positioning method based on vector calibration. Background Art

[0002] Cotton terminal bud positioning technology is an underlying core algorithm technology for realizing large-scale centralized automated cotton topping. The algorithm can be used in application scenarios such as targeted pesticide spraying, automated terminal bud removal, and pest detection at fixed plant sites.

[0003] The existing methods for locating cotton terminal buds are mainly the following (Hao Yanjie et al., Research status and prospects of cotton terminal bud identification and positioning technology, Chinese Journal of Agricultural Mechanization, 2018(11): 72-78):

[0004] 1. Detection and positioning method based on laser beam emission. This type of method mainly places the laser transmitter and receiver in parallel, and uses the principle of laser penetration (when there is an object blocking the transmitter and receiver, the receiver cannot obtain the laser and outputs a low-level signal, otherwise it outputs a high-level signal) to achieve the positioning of the apical bud height. This method has good practicality due to the advantages of laser anti-interference, good monochromaticity and low power consumption, but this method has problems such as high cost and difficult manufacturing process.

[0005] 2. Positioning methods based on ultrasonic detection. This type of method mainly relies on the advantages of ultrasonic waves such as high frequency, short wavelength, and good directionality. The ultrasonic transmitter is used to transmit ultrasonic waves in a certain direction and time them. Once the ultrasonic wave encounters an obstacle, it will be reflected back immediately. At this time, the receiver receives the reflected wave and stops timing. The distance between the transmitting position and the obstacle can be calculated by the propagation speed of the ultrasonic wave in a certain medium and the corresponding time difference. The specific height of the apical bud can then be obtained by calculation, thereby locating the apical bud. However, this method is easily affected by surrounding environmental factors (temperature, light, etc.) and has poor stability.

[0006] 3. Based on the detection and positioning method of the profiling plate, this method uses the contact between the profiling plate and the top of the cotton to generate action and reaction force. The top bud of the cotton supports the profiling plate, and the angle sensor connected to the profiling plate produces an angle change. The actual height of the top bud can be calculated through the angle change. This method is easily affected by the weight of the profiling plate and has a large error.

[0007] 4. Based on machine vision detection and positioning method, this method has the disadvantages of high computational complexity, poor real-time performance and excessive light sensitivity. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a cotton apical bud positioning method based on vector calibration with good accuracy and rapid detection capability in complex target crop scenes (including occlusion by target crop leaves themselves, occlusion by artificial obstacles, uneven lighting, etc.).

[0009] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0010] A cotton terminal bud positioning method based on vector calibration comprises: performing leaf segmentation on a collected cotton image, extracting the image area of ​​each leaf, and further obtaining the diagonal length, leaf base point coordinates and center point coordinates of each leaf image area; connecting the center point and leaf base point in the same leaf image area respectively, and calculating the slope and constant term of the obtained line segment; extending the connecting line segment between the center point and the leaf base point in the same leaf image area in a directed manner, the extension line terminates at the updated leaf base point coordinates that are re-determined according to the diagonal length of each leaf image area, the leaf base point coordinates, the center point coordinates, and the slope and constant term of the connecting line segment between the center point and the leaf base point, and obtaining the positioning line segment; obtaining all intersection points of the obtained positioning line segment and obtaining its distance matrix; deduplicating the distance matrix, and all the intersection points obtained after deduplication are the coordinate positioning of all the terminal buds in the cotton image.

[0011] The cotton terminal bud positioning method based on vector calibration described in the present invention specifically comprises the following steps:

[0012] S1: Collect cotton images;

[0013] S2: extracting the image area of ​​each leaf in the cotton image;

[0014] S3: Obtain the diagonal length L of each leaf image area 对角线 , the unit is pixel;

[0015] S4: Obtain the leaf base point coordinates B (x b ,y b );

[0016] S5: Obtain the center point coordinates C (x c ,y c );

[0017] S6: Connect the center point and the leaf base point in the same leaf image area respectively to obtain the connecting line segments of each leaf image area Calculate each line segment The slope k and constant term b;

[0018] S7: Each connecting line segment The extended line ends at the updated leaf base point coordinates to obtain n extended line segments. Each connecting line segment Extend the line segment Together they form a positioning line segment That is, we get n positioning line segments n is 0 or a positive integer; the updated leaf base point coordinates are determined according to the following conditions, as follows:

[0019] When x b >x c When k>0, the updated leaf base point coordinates are (x b +L 对角线 , k×(x b +L 对角线 )+b);

[0020] When x b >x c When k<0, the updated leaf base point coordinates are (y b -L 对角线 -b / k), y b -L 对角线 );

[0021] When x b <x c When k>0, the updated leaf base point coordinates are (x b -L 对角线 , k×(x b -L 对角线 )+b);

[0022] When x b <x c When k<0, the updated leaf base point coordinates are (y b +L 对角线 -b / k), y b +L 对角线 );

[0023] When x b =x c And y b >y c When k=0, it means the connecting line segment Perpendicular to the x-axis, the updated leaf base point coordinates are (x b ,y b +L 对角线 );

[0024] When x b =x c And y b <y c When k=0, it means the connecting line segment Perpendicular to the x-axis, the updated leaf base point coordinates are (x b ,yb -L 对角线 );

[0025] S8: Determine whether n is 0. If n is 0, it means that no extended line segment is identified in the collected cotton image. The program determines that there is no terminal bud in the cotton image, and returns the terminal bud positioning result as None, and the identification ends; otherwise, execute step S9;

[0026] S9: When n>0 (n is a positive integer), the n positioning line segments obtained are Recorded as And obtain all the intersection points of the n positioning line segments

[0027] S10: Obtain the distance matrix between all intersection points obtained in step S9;

[0028] S11: De-duplication is performed on the distance matrix obtained in step S10, and all the intersection points obtained after de-duplication are the coordinate locations of all the terminal buds in the cotton image.

[0029] In step S1 of the above method, the cotton image is collected and uploaded using the existing technology. In step S2, the leaf area of ​​the cotton leaf is segmented using the existing technology to extract the image area of ​​each cotton leaf in the image (such as using the Mask R-CNN target detection model to implement cotton leaf segmentation on the cotton image, target leaf segmentation based on a convolutional neural network, or image segmentation based on transfer learning, etc.), and the extracted image area of ​​the cotton leaf (i.e., the leaf contour) is usually a rectangle or a square. Steps S3 to S5 are all implemented using existing conventional technologies, and steps S3, S4, and S5 can be performed in no particular order. In the present application, it is preferred to use a Mask R-CNN convolutional neural network to obtain the image area of ​​each leaf and further obtain the diagonal length based on the area, use the SVM algorithm to extract the contour mark of each leaf image area and its leaf base point coordinates, and use the minimum circumscribed rectangle algorithm to obtain the center point coordinates of each leaf image area (the intersection of the diagonal lines of the circumscribed rectangle is the center point coordinates).

[0030] In step S6 of the above method, each line segment The slope k and the constant term (also called the intercept) b are calculated according to the following formulas: The resulting line segment The equation is

[0031]

[0032] In step S10 of the above method, the distance matrix can be obtained by using conventional methods such as calculating Euclidean distance, cosine similarity, Hamming distance or Mahalanobis distance. In the present application, it is preferred to first traverse all intersections obtained in step S9 and obtain the distance matrix by calculating the Euclidean distance between each two.

[0033] In step S11 of the above method, conventional existing methods such as k-nearest neighbor algorithm or k-means algorithm can be used to achieve the purpose of deduplication. In the present application, it is preferred to first set a distance error threshold, then traverse the distance matrix obtained in step S10, and perform deduplication according to the distance error threshold. Considering that the actual cotton top bud position allows a deviation of 0 to 3 cm, the machine vision distance error threshold is usually set to 5.

[0034] Compared with the prior art, the method of the present invention uses the biological characteristic information of the leaves of the target crop and the spatial position information between the leaves of the target crop, and finally fuses the above characteristics into vector information for abstract expression, and finally realizes the identification of the terminal bud of the target crop by using the characteristics of direction and size of the vector. The method of the present invention does not directly locate the terminal bud area of ​​the target crop, but is an indirect terminal bud positioning method by fusing the biological characteristics and spatial information of the leaves around the terminal bud. With the help of the vector positioning method, many problems in complex scenes such as the occlusion of the target crop leaves themselves, the occlusion of artificial obstacles, and the uneven illumination of the terminal bud area can be overcome, and it has good operability, accuracy and rapid detection capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the cotton terminal bud positioning method based on vector calibration according to the present invention;

[0036] Figure 2 The obtained cotton image (i.e. the cotton image to be uploaded);

[0037] Figure 3 For Figure 2 The image shown is the result after leaf segmentation;

[0038] Figure 4 For Figure 3 A cotton leaf is intercepted separately, and a diagonal identification diagram is segmented for the image area;

[0039] Figure 5 For Figure 3 The image shown is a result of contour marking of cotton leaves after leaf segmentation;

[0040] Figure 6 For Figure 3 The image shown is a result of marking the center point and leaf base point of the cotton leaf after leaf segmentation;

[0041] Figure 7 For Figure 3 The cotton apical bud vector positioning result diagram of the cotton leaf after leaf segmentation in the image shown;

[0042] Figure 8 Connecting line between cotton leaf center point and leaf base point Under the condition x b >x c When k>0, locate the line segment Generated schematics;

[0043] Fig. 9 Connect the line segments for the center point and base point of the cotton leaf Under the condition x b >x c When k<0, locate the line segment Generated schematics;

[0044] Fig.10 Connect the line segments for the center point and base point of the cotton leaf Under the condition x b <x c When k>0, locate the line segment Generated schematics;

[0045] Fig.11 Connect the line segments for the center point and base point of the cotton leaf Under the condition x b <x c When k<0, locate the line segment Generated schematics;

[0046] Fig.12 Connect the line segments for the center point and base point of the cotton leaf Under the condition x b =x c And y b >y c When , the positioning line segment Generated schematics;

[0047] Fig.13 Connect the line segments for the center point and base point of the cotton leaf Under the condition x b =x c And y b <y c When , the positioning line segment Generated schematic. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings to provide a better understanding of the content of the present invention, but the present invention is not limited to the following embodiments.

[0049] The cotton terminal bud positioning method based on vector calibration of the present invention comprises: performing leaf segmentation on the collected cotton image, extracting the image area of ​​each leaf, and further obtaining the diagonal length, leaf base point coordinates and center point coordinates of each leaf image area; connecting the center point and leaf base point in the same leaf image area respectively, and calculating the slope and constant term of the obtained line segment; extending the connecting line segment of the center point and the leaf base point in the same leaf image area in a directed manner, and the extension line terminates at the updated leaf base point coordinates that are re-determined according to the diagonal length of each leaf image area, the leaf base point coordinates, the center point coordinates, and the slope and constant term of the connecting line segment between the center point and the leaf base point, to obtain the positioning line segment; obtaining all intersection points of the obtained positioning line segment and obtaining its distance matrix; deduplicating the distance matrix, and all the intersection points obtained after deduplication are the coordinate positioning of all the terminal buds in the cotton image.

[0050] like Figure 1 As shown, in a specific embodiment, the cotton terminal bud positioning method based on vector calibration of the present invention specifically includes the following steps:

[0051] S1: Collect cotton images;

[0052] by Figure 2 The image shown is taken as the cotton image and uploaded.

[0053] S2: extracting the image area of ​​each leaf in the cotton image;

[0054] Using the existing Mask R-CNN target detection model Figure 2 The cotton image shown in the figure is segmented into cotton leaves, retaining a large number of leaf areas and extracting the image area of ​​each leaf. The result is as follows Figure 3 shown.

[0055] S3: Obtain the diagonal length L of each leaf image area 对角线 , the unit is pixel (px);

[0056] To capture individually Figure 3 Take a cotton leaf image region as an example (e.g. Figure 4 ), obtain the diagonal length L of the leaf image area 对角线 , the unit is pixel (px).

[0057] S4: Obtain the leaf base point coordinates B (x b ,y b );

[0058] In the image area of ​​each leaf, the SVM binary classification model is used to obtain the leaf contour mark (such as Figure 5 As shown in Figure 2, the leaf base point position B(xb ,y b ), the leaf base points of each leaf are marked on the cotton image, such as Figure 6 shown.

[0059] S5: Obtain the center point coordinates C (x c ,y c );

[0060] The center point position C(x c ,y c ), mark the center points of each leaf on the cotton image, such as Figure 6 shown.

[0061] S6: Connect the center point and the leaf base point in the same leaf image area respectively to obtain the connecting line segments of each leaf image area Calculate each line segment The slope k and constant term b;

[0062] Each line segment The slope k and constant term b are calculated according to the following formulas: Each line segment The equation is expressed as

[0063] S7: Each connecting line segment The extended line ends at the updated leaf base point coordinates to obtain n extended line segments. Each connecting line segment Extend the line segment Together they form a positioning line segment That is, we get n positioning line segments n is 0 or a positive integer; the updated leaf base point coordinates are determined according to the following conditions, as follows:

[0064] When x b >x c When k>0, the updated leaf base point coordinates are (x b +L 对角线 , k×(x b +L 对角线 )+b); under this condition, a positioning line segment is generated The schematic diagram is as follows Figure 8 As shown in the figure, the updated leaf base points are as follows Figure 8 A(x a ,y a );

[0065] When x b >x cWhen k<0, the updated leaf base point coordinates are (y b -L 对角线 -b / k), y b -L 对角线 ); Under this condition, a positioning line segment is generated The schematic diagram is as follows Fig. 9 As shown in the figure, the updated leaf base points are as follows Fig. 9 A(x a ,y a );

[0066] When x b <x c When k>0, the updated leaf base point coordinates are (x b -L 对角线 , k×(x b -L 对角线 )+b); under this condition, a positioning line segment is generated The schematic diagram is as follows Fig.10 As shown in Fig.10 A(x a ,y a );

[0067] When x b <x c When k<0, the updated leaf base point coordinates are (y b +L 对角线 -b / k), y b +L 对角线 ); Under this condition, a positioning line segment is generated The schematic diagram is as follows Fig.11 As shown in the figure, the updated leaf base points are as follows Fig.11 A(x a ,y a );

[0068] When x b =x c And y b >y c When k=0, it means the connecting line segment Perpendicular to the x-axis, the updated leaf base point coordinates are (x b ,y b +L 对角线 ); Under this condition, a positioning line segment is generated The schematic diagram is as follows Fig.12 As shown in Fig.12 A(x a ,y a );

[0069] When x b =x c And yb <y c When k=0, it means the connecting line segment Perpendicular to the x-axis, the updated leaf base point coordinates are (x b ,y b -L 对角线 ); Under this condition, a positioning line segment is generated The schematic diagram is as follows Fig.13 As shown in Fig.13 A(x a ,y a ).

[0070] S8: Determine whether n is 0. If n is 0, it means that no extended line segment is identified in the collected cotton image. The program determines that there is no terminal bud in the cotton image, and returns the terminal bud positioning result as None, and the identification ends; otherwise, execute step S9;

[0071] S9: When n is a positive integer, the n positioning line segments obtained are Recorded as And obtain all the intersection points of the n positioning line segments

[0072]

[0073] In this embodiment, the Bentley-Ottmann algorithm is used to quickly obtain all intersection points of the n positioning line segments;

[0074] S10: traverse all the intersection points obtained in step S9, and obtain the following distance matrix by calculating the Euclidean distance between each two;

[0075] <![CDATA[Cr 1 ]]> <![CDATA[Cr 2 ]]> …… <![CDATA[Cr n′ ]]> <![CDATA[Cr 1 ]]> <![CDATA[d(Cr 1 ,Cr 1 )]]> <![CDATA[d(Cr 1 ,Cr 2 )]]> …… <![CDATA[d(Cr 1 ,Cr n′ )]]> <![CDATA[Cr 2 ]]> <![CDATA[d(Cr 2 ,Cr 1 )]]> <![CDATA[d(Cr 2 ,Cr 2 )]]> …… <![CDATA[d(Cr 2 ,Cr n′ )]]> …… …… …… …… …… <![CDATA[Cr n′ ]]> <![CDATA[d(Cr n′ ,Cr 1 )]]> <![CDATA[d(Cr n′ ,Cr 2 )]]> …… <![CDATA[d(Cr n′ ,Cr n′ )]]>

[0076] The Euclidean distance calculation formula is d(Cr 1 ,Cr 2 ) as an example:

[0077]

[0078] S11: First set the distance error threshold Threshold, then traverse the distance matrix obtained in step S10, remove duplicates according to the distance error threshold, and obtain all the intersections after deduplication That is the coordinate location of all the terminal buds in the cotton image.

[0079] In this embodiment, considering that the actual cotton bud position allows a deviation of 0 to 3 cm, the machine vision distance error threshold Threshold is set to 5 px, that is, d(Cr n′-1 ,Cr n′-2)≤5(n'>2), merge the two points into a new coordinate point, the coordinate value of the new coordinate point is By analogy, all new intersection points after deduplication are finally obtained as These new intersection points are the coordinate locations of all the apical buds in the cotton image (e.g. Figure 7 shown).

[0080] The positioning accuracy of the terminal bud in the above specific embodiment was tested:

[0081] Method: The circular error probability (CEP) is used to measure the accuracy of the predicted terminal bud position. Considering that the actual cotton terminal bud position allows a deviation of 0 to 3 cm, we finally selected 0.5 cm CEP (i.e., a circle with a radius of 0.5 cm is drawn, and the predicted point can appear within the circle) as the measurement index value in the test experiment, where 0.5 cm contains about 13 pixels (px), so the final predicted terminal bud coordinate value and the manually marked terminal bud coordinate value allow the Euclidean distance error range [0,13px], that is, the Euclidean distance between the manually marked terminal bud coordinate and the predicted terminal bud coordinate is less than or equal to 13, which can be judged that the algorithm prediction is accurate. This experiment first uses manually marked terminal bud images, in which a total of 100 images are marked (267 manually marked terminal buds), and then the terminal bud positioning test is performed using the method described in the present invention. The final test results are shown in the following table.

[0082]

Claims

1. A cotton terminal bud positioning method based on vector calibration, characterized in that: The collected cotton image is segmented into leaves, the image area of ​​each leaf is extracted, and the diagonal length, leaf base point coordinates and center point coordinates of each leaf image area are further obtained; the center point and leaf base point in the same leaf image area are respectively connected, and the slope and constant term of the obtained line segment are calculated; the connecting line segment of the center point and the leaf base point in the same leaf image area is extended in a directed manner, and the extension line ends at the updated leaf base point coordinates that are re-determined according to the diagonal length of each leaf image area, the leaf base point coordinates, the center point coordinates, and the slope and constant term of the connecting line segment between the center point and the leaf base point, to obtain the positioning line segment; all intersection points of the obtained positioning line segments are obtained and the distance matrix between them is obtained; the distance matrix is ​​deduplicated, and all the intersection points after deduplication are the coordinate positioning of all the terminal buds in the cotton image; the specific steps include: S1: Collect cotton images; S2: extracting the image area of ​​each leaf in the cotton image; S3: Obtain the diagonal length of each leaf image area L 对角线 , the unit is pixel; S4: Obtain the leaf base point coordinates B (x b ,y b ); S5: Obtain the center point coordinates C (x c ,y c ); S6: Connect the center point and the leaf base point in the same leaf image area respectively to obtain the connecting line segments of each leaf image area , calculate each line segment The slope k and constant term b ; S7: Each connecting line segment Based on the leaf base point, the extension line ends at the updated leaf base point coordinates, and we get n Extended line segments , each connecting line segment Extend the line segment Together they form a positioning line segment , that is, n Positioning line segments , n is 0 or a positive integer; the updated leaf base point coordinates are determined according to the following conditions, as follows: When x b >x c and k >0, the updated leaf base point coordinates are (x b + L 对角线 , k ×(x b + L 对角线 )+ b ); When x b >x c and k <0, the updated leaf base point coordinates are (y b - L 对角线 - b / k ), y b - L 对角线 ); When x b <x c and k >0, the updated leaf base point coordinates are (x b - L 对角线 , k ×(x b - )+b); When x b <x c and k <0, the updated leaf base point coordinates are (y b + L 对角线 - b / k ), y b + ); When x b =x c And y b >y c At this time k =0, indicating a connecting line segment Perpendicular to the x-axis, the updated leaf base point coordinates are (x b ,y b + ); When x b =x c And y b <y c At this time k =0, indicating a connecting line segment Perpendicular to the x-axis, the updated leaf base point coordinates are (x b ,y b - ); S8: Judgment n Is it 0? n If it is 0, it means that no extended line segment is identified in the collected cotton image, and the program determines that there is no terminal bud in the cotton image, and returns the terminal bud positioning result as None, and the identification ends; otherwise, execute step S9; S9: When n >0, respectively obtain n Positioning line segments Recorded as , and obtain the n All intersection points of the positioning line segments [Cr 1 ( ), Cr 2 ( ), Cr 3 ( )……Cr n’ ( )]; S10: Obtain the distance matrix between all intersection points obtained in step S9; S11: De-duplication is performed on the distance matrix obtained in step S10, and all the intersection points obtained after de-duplication are the coordinate locations of all the terminal buds in the cotton image.

2. The cotton terminal bud positioning method based on vector calibration according to claim 1, characterized in that: In step S10, all intersection points obtained in step S9 are first traversed, and a distance matrix is ​​obtained by calculating the Euclidean distance between each two intersection points.

3. The cotton terminal bud positioning method based on vector calibration according to claim 1, characterized in that: In step S11, a distance error threshold is first set, and then the distance matrix obtained in step S10 is traversed to remove duplicates according to the distance error threshold.

Images