Computer Vision-Based Plant Phenotype Measurement Method, System and Device

Abstract

The present invention discloses a plant phenotype measurement method, system and device based on computer vision. The method includes obtaining an original plant image for calibration processing, and then performing segmentation processing to obtain a binary image of the plant; obtaining a skeleton topological structure image based on the binary image of the plant, and further obtaining relevant information of the plant; separating each leaf from the connection between the petiole and the stem to obtain the contour information of each leaf, and combining the skeleton topology to obtain the plant leaf phenotype information; obtaining a binary image of the stem to obtain the stem phenotype information. The present invention processes the plant image information to obtain the plant skeleton topological structure information, and obtains the plant phenotype information according to the above topological structure in combination with the tangent perpendicular measurement method and the plant structure characteristics. By directly processing and analyzing the image information of the plant, the automation of plant phenotype measurement is realized, and the efficiency and accuracy of plant phenotype measurement are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to a method, system, and device for measuring plant phenotypes based on computer vision. Background Art

[0002] Measuring plant phenotypes is of great significance in scientific research and plant cultivation applications. Currently, in the application of automated plant phenotype measurement technologies, it is mainly for measuring leaf morphology. By collecting independent leaf images and using computer vision technologies, automatic detection of leaf morphological parameters is achieved. There are no publicly disclosed technical methods or application cases for measuring the morphology of various parts of the whole plant.

[0003] Existing plant phenotype measurement methods are mostly separated contact measurement methods, that is, by using physical tools to manually measure the morphological parameters of various parts of tobacco, which are used as a guide for formulating plant cultivation plans and as a reference for the research and development of new varieties.

[0004] In the application of computer vision in crop morphology measurement, it mainly uses image processing methods to segment, detect, analyze, and measure crops, so as to achieve the purpose of growth trend detection and morphology measurement. In recent years, computer vision technologies have gradually been applied to the morphology measurement of crops such as corn and potatoes, mainly for applications related to growth laws and single-object detection, such as only measuring crop height, only measuring crop leaf area, and standardizing the growth direction of crop leaves.

[0005] For example, some experts and scholars use an infrared lighting device and a near-infrared filter CCD camera to detect plant growth and count the crop growth rate. This method has good adaptability to the surrounding environment. The disadvantage is that it is only applicable to fixed scenarios, and the measurement parameter is only the height of the plant from the horizontal plane to the top of the plant.

[0006] Or, potato leaf images are collected from different angles, and through filtering, threshold segmentation, and morphological detection methods, the leaf area can be measured more accurately. On the one hand, this method does not consider the influence of image distortion on the measurement results; on the other hand, the designed algorithm mainly targets single and relatively regular-shaped leaves, and each image collection only contains an image of one leaf.

[0007] There are also experts who measure the heights of field crops such as corn and wheat through image processing, extract the plant type information of corn and wheat with the help of artificial markers, and measure the leaf length and leaf inclination angle by marking the leaves. It can achieve automatic measurement of crop plant types and reduce the pressure of manual measurement. However, due to the need for marking, there are also problems of poor repeatability and low efficiency; at the same time, the growth trends of wheat and corn plant types are relatively regular, and the topological structure is relatively simple. Technical solutions can be designed through hard coding, which is not applicable to application scenarios where the growth of leaves and branches is not standardized.

[0008] There is also a method for measuring the leaf area and leaf angle distribution by using image processing methods. The measurement results of the method proposed in the article are more comprehensive. The main problem is still that the topological structure of the maize plant type is relatively regular and the universality of the method is weak. And so on. At present, most operations are not intelligent and streamlined enough.

[0009] Therefore, how to propose a solution that can simply and quickly realize the automatic measurement of plant phenotype parameters has become an urgent problem to be solved. Summary of the Invention

[0010] The present invention aims at the disadvantages in the prior art and provides a method, a system and a device for measuring plant phenotypes based on computer vision.

[0011] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0012] A method for measuring plant phenotypes based on computer vision includes the following steps:

[0013] Obtain the original plant image and perform calibration processing in combination with four identifiers to obtain the calibrated plant image, wherein the four identifiers are at the four corners of the original plant image;

[0014] Perform segmentation processing on the calibrated plant image to obtain a binary image of the plant;

[0015] Based on the binary image of the plant, obtain a skeleton topological structure image, and then obtain relevant information of the plant. The relevant information at least includes skeleton information and intersection set;

[0016] Based on the skeleton information and the intersection set, perform single-source path analysis and the perpendicular cutting line measurement method to obtain the cutting line for separating the leaf from the stem, separate each leaf from the connection between the petiole and the stem to obtain the contour information of each leaf, and combine the skeleton topology to obtain the plant leaf phenotype information, where the plant leaf phenotype information includes width, length, perimeter, area, and the number of whole plant leaves;

[0017] Strip the contour information of the leaf from the binary image of the plant to obtain a binary image of the stem, and obtain the stem phenotype information by extracting the stem skeleton topology and using the perpendicular cutting line measurement method, where the stem phenotype information at least includes plant height and stem diameter.

[0018] As an implementable manner, the calibration processing in combination with four identifiers includes the following steps:

[0019] Set the central coordinates of the four identifiers as C1, C2, C3, and C4 in clockwise or counterclockwise order. Among them, C1 is the central coordinate of the first corner identifier, C2 is the central coordinate of the second corner identifier, C3 is the central coordinate of the third corner identifier, and C4 is the central coordinate of the fourth corner identifier. And set the aspect ratio of the rectangle formed by the central coordinates as a:b;

[0020] Perform rectangular fitting on the central coordinates C1 and C4 of two adjacent corners to obtain the transformed central coordinates C2' and C3';

[0021] Obtain the transformation matrix based on the central coordinate set [C1, C2, C3, C4] and the transformed central coordinate set [C1, C2', C3', C4];

[0022] Perform global transformation processing on the original plant image based on the transformation matrix to obtain the second image;

[0023] Set the aspect ratio of the rectangle formed by the transformed central coordinate set as a:c, and combine the aspect ratio a:b to perform scale transformation on the width of the second image with a transformation ratio of b / c to obtain the corrected plant image.

[0024] As an implementable manner, the segmentation processing of the corrected plant image includes the following steps:

[0025] Perform channel separation processing on the corrected plant image to obtain grayscale images of the R, G, and B channels respectively;

[0026] Perform weighted average calculation on the grayscale images of the R, G, and B channels to obtain the luminance image;

[0027] Obtain the plant color representation map according to the color characteristics of the plant;

[0028] Obtain the transformation coefficient based on the luminance image and the plant color representation map, obtain the second grayscale image based on the transformation coefficient, and then obtain the plant binary image through automatic threshold segmentation processing.

[0029] As an implementable manner, obtaining the relevant information of the plant based on the plant binary image includes the following steps:

[0030] Perform skeleton extraction processing on the plant binary image to obtain the skeleton topological structure;

[0031] Perform analysis on the plant skeleton topological structure of the skeleton topological structure to obtain the set of endpoint points and the set of intersection points. Among them, the set of endpoint points is the set of all leaf tip endpoints, plant stem growth points, and stem base endpoints, and the set of intersection points is the set of intersection points at the connections between leaves and leaf stalks and stems;

[0032] Perform a distance transformation on the path points in the skeleton topology structure in the binary image of the plant to obtain a result image, which is a skeleton topology structure diagram excluding the petioles and stems, and then obtain the leaf skeleton diagram;

[0033] By selecting and analyzing the characteristics of the leaf skeleton diagram, obtain the skeleton information characterizing each leaf.

[0034] As an implementable method, the following steps are included to obtain the cutting line for separating the leaf from the stem by performing single-source path analysis and the perpendicular cutting line measurement method, separate each leaf from the connection between the petiole and the stem to obtain the contour information of each independent leaf, and combine the skeleton topology structure to obtain the plant leaf phenotype information:

[0035] Denote the intersection point set as (c1, c2, …, c n ) and the skeleton information as (p1, p2, …, p k );

[0036] Obtain the single-source path set formed by any point in the skeleton information and the intersection point set, and select the path with the shortest single-source path length from the single-source path set. This single-source path is the connection point between the leaf and the stem;

[0037] Based on the single-source path set, traverse from the end point of the single-source path to the start point direction to obtain the tangents and corresponding slopes of each point on the single-source path;

[0038] Obtain the perpendicular slope passing through the corresponding point according to the slope and draw a perpendicular line. The total number of point sets in the connected domain passed by the perpendicular line represents the leaf width or the width at the connection between the leaf and the stem;

[0039] According to the mutation of the angle at the connection between the leaf and the stem and the mutation of the width at the connection, take the second derivative of the tangent set and the width set of each point respectively. Stop the calculation when the inflection points occur simultaneously at the corresponding points for the first time. Then the corresponding positions of each point are the best points for cutting the leaf from the stem, obtain the cutting line, and further obtain each leaf image;

[0040] According to each leaf image, obtain the contour information and the main vein skeleton structure information of each leaf, and further obtain the plant leaf phenotype information of each plant.

[0041] As an implementable method, the following steps are included to strip the leaf from the binary image of the plant to obtain the binary image of the stem, and obtain the stem phenotype information by extracting the stem skeleton topology structure and using the perpendicular cutting line measurement method:

[0042] Obtain the intersection path with the shortest distance from any point on the leaf skeleton diagram to the stem, draw a tangent line at each position of the intersection path, and the area passed through by the perpendicular line perpendicular to the tangent line is the leaf width. Determine the cutting line at the separation of the leaf and the stem according to all the tangent angles and leaf width information, and separate the leaf according to this cutting line to obtain the leaf contour and skeleton structure information until the leaf information is obtained;

[0043] Separate the leaf information in the leaf skeleton diagram to obtain the binary image of the stem, and then obtain the stem information, where the stem information includes the stem skeleton and the set of stem skeleton endpoints;

[0044] Obtain the stem diameter at each position of the stem according to the intersection information of the perpendicular lines cut at each position of the stem skeleton and the binary image of the stem;

[0045] Obtain the single-source path between the two points according to the growth point of the stem skeleton and the base point of the stem, and select the path with the longest length in the single-source path, which is the path from the starting point to the ending point of the stem, and the length corresponding to this path is the plant height.

[0046] As an implementable manner, the four identifiers are set in the image acquisition device, the image acquisition device includes a background board, the background board contains 4 identifiers, the identifiers are distributed at the four corners of the background board, the color of the identifiers has an obvious contrast with the color of the background board, and the aspect ratio of the rectangle formed by connecting the centers of the identifiers is 4:3.

[0047] A plant phenotype measurement system based on computer vision includes an image correction module, an image segmentation module, a skeleton topology structure extraction module, a leaf detection module, and a stem detection module;

[0048] The image correction module is used to obtain the original plant image and perform correction processing in combination with four identifiers to obtain the corrected plant image, where the four identifiers are at the four corners of the original plant image;

[0049] The image segmentation module is used to perform segmentation processing on the corrected plant image to obtain the binary image of the plant;

[0050] The skeleton topology structure extraction module obtains the skeleton topology structure image based on the binary image of the plant, and then obtains the relevant information of the plant. The relevant information includes at least skeleton information and intersection set;

[0051] The leaf detection module, based on the skeleton information and intersection set, performs single-source path analysis and perpendicular line measurement method to obtain the cutting line for separating the leaf and the stem, separates each leaf from the connection between the petiole and the stem to obtain the contour information of each independent leaf, and combines the skeleton topology structure to obtain the plant leaf phenotype information, where the plant leaf phenotype information includes width, length, perimeter, area, and the number of whole-plant leaves;

[0052] The stem detection module strips the leaves from the binary image of the plant to obtain a binary image of the stem, and obtains the stem phenotypic information by extracting the topological structure of the stem skeleton and using the tangent perpendicular measurement method. Among them, the stem phenotypic information at least includes plant height and stem diameter.

[0053] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described above are implemented.

[0054] A plant phenotypic measurement device based on computer vision includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method steps described above are implemented.

[0055] Due to the adoption of the above technical solutions, the present invention has significant technical effects:

[0056] The present invention processes the plant image information to obtain the topological structure information of the plant skeleton, and combines the above topological structure with the tangent perpendicular measurement method and the plant structure characteristics to obtain the plant phenotypic information. By directly processing and analyzing the image information of the plant, the automation of plant phenotypic measurement is realized, and the efficiency and accuracy of plant phenotypic measurement are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1 is a schematic flow chart of the method of the present invention;

[0059] Figure 2 is a schematic system structure diagram of the present invention;

[0060] Figure 3 is a schematic diagram of the image acquisition device of the present invention;

[0061] Figure 4 is a schematic diagram of obtaining a binary image of a plant based on three color channels of the present invention;

[0062] Figures 5 - 6 is a pixel 8-neighborhood structure diagram in the analysis process of the endpoints and intersection points of the skeleton topological structure;

[0063] Figure 7 is a schematic diagram of the calculation principle of the image difference method in the embodiment of the present invention;

[0064] Figure 8 is the schematic diagram of the perpendicular cutting line measurement in the embodiment of the present invention;

[0065] Figure 9 The binary image of the plant can be obtained through adaptive threshold segmentation;

[0066] Figure 10 is the result display diagram of the method of the present invention. Detailed implementation manners

[0067] The present invention will be further described in detail below in conjunction with embodiments. The following embodiments are explanations of the present invention, and the present invention is not limited to the following embodiments.

[0068] Embodiment 1:

[0069] A plant phenotype measurement method based on computer vision, as Figure 1 shown, includes the following steps:

[0070] S100. Obtain the original plant image and perform calibration processing in combination with four identifiers to obtain the calibrated plant image, wherein the four identifiers are at the four corners of the original plant image;

[0071] S200. Perform segmentation processing on the calibrated plant image to obtain the binary image of the plant;

[0072] S300. Obtain the skeleton topological structure image based on the binary image of the plant, and further obtain the relevant information of the plant. The relevant information at least includes skeleton information and intersection set;

[0073] S400. Based on the skeleton information and intersection set, perform single-source path analysis and perpendicular cutting line measurement method to obtain the cutting line for separating the leaf and the stem, separate each leaf from the connection between the petiole and the stem to obtain the contour information of each leaf, and combine the skeleton topological structure to obtain the plant leaf phenotype information, wherein the plant leaf phenotype information includes width, length, perimeter, area, and the number of whole plant leaves;

[0074] S500. Strip the contour information of the leaf from the binary image of the plant to obtain the binary image of the stem, and obtain the stem phenotype information by extracting the stem skeleton topological structure and using the perpendicular cutting line measurement method, wherein the stem phenotype information at least includes plant height and stem diameter.

[0075] In step S100, the calibration processing in combination with four identifiers includes the following steps:

[0076] Set the central coordinates of the four identifiers in clockwise or counterclockwise order as C1, C2, C3, and C4. Among them, C1 is the central coordinate of the first corner identifier, C2 is the central coordinate of the second corner identifier, C3 is the central coordinate of the third corner identifier, and C4 is the central coordinate of the fourth corner identifier. And set the aspect ratio of the rectangle formed by the central coordinates as a:b;

[0077] Perform rectangular fitting on the central coordinates C1 and C4 of two adjacent corners to obtain the transformed central coordinates C2' and C3';

[0078] Obtain the transformation matrix based on the central coordinate set [C1, C2, C3, C4] and the transformed central coordinate set [C1, C2', C3', C4];

[0079] Perform global transformation processing on the original plant image based on the transformation matrix to obtain the second image;

[0080] Set the aspect ratio of the rectangle formed by the transformed central coordinate set as a:c. Combine the aspect ratio a:b to perform scale transformation on the width of the second image with a transformation ratio of b / c to obtain the corrected plant image. That is, transform the width of the second image to b / c times to obtain the corrected plant image.

[0081] In this embodiment, the four identifiers are set in the image acquisition device. The image acquisition device includes a background board. The background board contains 4 identifiers. The identifiers are distributed at the four corners of the background board. The color of the identifier has a distinct contrast with the color of the background board. The aspect ratio of the rectangle formed by connecting the centers of the identifiers is 4:3. In addition, the image acquisition device will surely also include an image acquisition device. See the appendix Figure 3 As shown, the background board 10 is composed of a solid color, an opaque and non-reflective flat panel 101, and an identifier 102 with a color different from that of the flat panel 101; and the shape of the identifier 102 can be circular or square; the image acquisition device 20 is an instrument with a camera or photographing function, which is not specifically limited in the embodiment of the present invention.

[0082] In one embodiment, in step S200, the segmentation processing of the corrected plant image includes the following steps:

[0083] Perform channel separation processing on the corrected plant image to obtain grayscale images of the R, G, and B channels respectively;

[0084] Perform weighted average calculation on the grayscale images of the R, G, and B channels to obtain a luminance image;

[0085] Obtain a plant color characterization map according to the color characteristics of the plant;

[0086] The transformation coefficients are obtained based on the luminance image and the plant color representation map. The second grayscale image is obtained based on the transformation coefficients, and then the plant binary map is obtained through automatic threshold segmentation processing.

[0087] For the three color channels imageRed, imageGreen, and imageBlue, the luminance image is obtained by weighted averaging the chromaticity values of the three channels, where imageHue = imageRed / 3.0 + imageGreen / 3.0 + imageBlue / 3.0; according to the color characteristics of the background board and the plant color characteristics, the image corresponding to the color characteristics is obtained. Here, taking green as an example, the calculation method is imageColor = 2 * imageGreen - imageBlue - imageRed; according to the pixel value distribution characteristics of the luminance image imageHue and the plant color representation map imageColor, the coefficient k is calculated. According to the formula image4 = k * imageHue + (1 - k) * imageColor, the second grayscale image image4 is obtained, where the value of k is in the interval (0, 1). The plant binary map image5 can be obtained through adaptive threshold segmentation. The entire process can be seen in the appendix Figure 4 。

[0088] In one embodiment, obtaining the relevant information of the plant based on the plant binary map in step S300 includes the following steps:

[0089] Performing skeleton extraction processing on the plant binary map to obtain the skeleton topological structure;

[0090] Performing plant skeleton topological structure analysis on the skeleton topological structure to obtain the endpoint set and the intersection point set. Among them, the endpoint set is the set of all leaf tip endpoints, plant stem growth points, and stem base endpoints, and the intersection point set is the set of intersection points at the connections between leaves and leaf stalks and stems;

[0091] Performing distance transformation on the path points in the skeleton topological structure in the plant binary map to obtain the result image. The result image is the skeleton topological structure diagram excluding the leaf stalks and stems, and then the leaf skeleton diagram is obtained;

[0092] By selecting and analyzing the characteristics of the leaf skeleton diagram, the skeleton information representing each leaf is obtained.

[0093] The specific implementation steps are as follows:

[0094] According to the plant binary map image5, the Zhang-Suen skeleton extraction algorithm is used to obtain the skeleton topological structure image6; analyzing the skeleton topological structure image6 to obtain the endpoint set (e1, e2,..., e m ) and the intersection point set (c1, c2,..., cn );According to the image6 path points of the skeleton topology structure, perform distance transformation on the plant binary image image5 to obtain the result image image7, where the result image image7 is a skeleton topology structure image only containing petioles and stalks; In the embodiment of the present invention, the differential image method is used to realize the rough extraction of plant leaves, and the calculation formula is leaf skeleton image image8 = skeleton topology structure image6 - result image image7; By feature selection and analysis of the leaf skeleton image image8, the skeleton information (p1, p2,..., p k ) can be obtained, where each point p i represents a point on the skeleton of the i-th leaf;

[0095] See Figures 5 - 6 is the 8-neighborhood structure diagram of any pixel in the image during the skeleton extraction process, where P1 is the target pixel. By performing structural analysis on each pixel in the image, the plant skeleton topology structure diagram is extracted, specifically:

[0096] Step1: Loop through all foreground pixel points and mark the pixel points that meet the following conditions as deleted.

[0097] (a) 2 <= B(P1) <= 6

[0098] (b) A(P1) = 1

[0099] (c) P2 * P4 * P6 = 0

[0100] (d) P4 * P6 * P8 = 0

[0101] Condition (a), the sum of the number of target pixels (1 in the binary value) around the central pixel P1 is between 2 and 6; Condition (b), among the 8-neighborhood pixels, in the clockwise direction, the number of times the adjacent two pixels appear 0->1.

[0102] Step2: It is very similar to Step1, and conditions (a) and (b) are exactly the same, except that conditions (c) and (d) are slightly different. The pixel P1 that meets the following conditions is marked as deleted, and the conditions are as follows:

[0103] (a) 2 <= B(P1) <= 6

[0104] (b) A(P1) = 1

[0105] (c) P2 * P4 * P8 = 0

[0106] (d) P2 * P6 * P8 = 0

[0107] Loop the above two steps until no pixel is marked as deleted in both steps, and the output result is the skeleton after the binary image is thinned.

[0108] Figure 7 It is a 8-neighborhood structure diagram of pixels during the analysis of the endpoints and intersection points of the skeleton topology structure, where the "5" corresponds to the black position in the figure, that is, the position of the target pixel to be analyzed. Specifically:

[0109] For the solution of the skeleton intersection points, analyze according to the pixel values of "1, 2, 3, 4, 6, 7, 8, 9" in the following table. According to the establishment conditions and termination conditions in the table, when the establishment conditions are met, that is, the pixel at the corresponding position under the establishment conditions is the foreground, this point is the intersection point;

[0110]

[0111] For the solution of the skeleton endpoints, if there is only one position in the 8-neighborhood of this pixel that is the foreground, it is the intersection point, otherwise it is not;

[0112] And attached Figure 8 This is the schematic diagram of the calculation principle of the image difference method in the embodiment of the present invention, obtaining the difference information between two images. Specifically: Subtract two images with the same size and the same data type. When the difference value is greater than 0, the corresponding value of the result image is 255, and when it is less than or equal to 0, the corresponding value of the result image is 0, thus obtaining the difference information between the two images. For the original plant skeleton topology structure image image6, after distance transformation according to the width information at the leaf petiole and the stem, the transformed image is the transformed image image7. According to image6 - image7, the leaf skeleton diagram can be obtained. Each mesophyll skeleton can represent the local information of a specific leaf, and is used to search for the connection point corresponding to the leaf and the plant stem according to this j local information.

[0113] In one embodiment, in step 400, the single-source path analysis and the perpendicular bisector measurement method are used to obtain the cutting line for separating the leaf from the stem, and the contour information of each independent leaf is obtained by separating each leaf from the connection between the petiole and the stem, and the plant leaf phenotype information is obtained by combining the skeleton topology structure, including the following steps:

[0114] Denote the intersection point set as (c1, c2, …, c n ), and denote the skeleton information as (p1, p2, …, p k );

[0115] Obtain the set of single-source paths formed by any point in the skeleton information and the intersection point set, and select the path with the shortest single-source path length from the set of single-source paths. This single-source path is the connection point between the leaf and the stem;

[0116] Based on the set of single-source paths, traverse from the end point of the single-source path to the start point direction to obtain the tangents and the corresponding slopes of each point on the single-source path;

[0117] The perpendicular slope passing through the corresponding point is obtained according to the slope, and a perpendicular line is drawn. The sum of the number of points in the connected region passed by the perpendicular line represents the leaf width or the width at the connection between the leaf and the stem;

[0118] According to the mutation of the angle at the connection between the leaf and the stem and the mutation of the width at the connection, the second derivatives are respectively calculated for the tangent set and the width set of each point. When the inflection points occur simultaneously at the corresponding points for the first time, the calculation stops. Then the positions corresponding to each point are the optimal points for cutting the leaf and the stem, the cutting line is obtained, and thus the images of each leaf are obtained;

[0119] According to the contour information and the main vein skeleton structure information of each leaf obtained from the images of each leaf, the leaf phenotypic information of each plant is further obtained.

[0120] According to the original plant skeleton topological structure image image6, the intersection point set (c1, c2, …, c n ), and the skeleton information (p1, p2, …, p k ), the path from each leaf point to the connection between the leaf and the stem is obtained, that is, among the single-source path sets formed by the point p i and the intersection point set (c1, c2, …, c n ), the one with the shortest path length is selected as the connection point between the leaf and the stem. The single-source path calculation method uses the Dijkstra algorithm, specifically:

[0121] DIJKSTRA(G, w, s)

[0122] initialize-single-source(G, s)

[0123] S = null

[0124] Q = G.V

[0125] While Q! = null

[0126] u = extract-min(Q)

[0127] S = S ∪ {u}

[0128] For each vertex v ∈ G.Adj[u]

[0129] RELAX{u, v, w}

[0130] Among them, G is the graph constructed from the skeleton image, w is the weight of each edge, and s is the starting point, that is, p i; First, initialize an empty set S to store the vertices for which the shortest path has been determined; initialize the set Q, which contains all vertices G.V in the graph. Q is a custom data structure that uses a minimum priority queue, with the key being the shortest distance from s to each vertex. From lines 4 - 8, extract the vertex u with the minimum shortest distance from s to the vertex from Q, add u to S, and recalculate the path lengths for each vertex adjacent to u to replace the current shortest path that is greater than the new path.

[0131] Since the starting point is known and the ending points are enumerable, each path can be obtained by traversing from the starting point to all ending points, thereby obtaining the single - source paths at the connections between each leaf and the stem.

[0132] According to the single - source path, traverse from the end point of the single - source path towards the starting point (i.e., from the connection towards the leaf vertex), calculate the tangent line and its corresponding slope angle1 at each point on the single - source path, calculate the slope angle2 of the perpendicular line passing through the corresponding point based on this slope and draw the perpendicular line line. The number of connected domain points passed by the perpendicular line is the width of the leaf or the width at the connection between the leaf and the stem width. Based on the mutation of the angle at the connection between the leaf and the stem and the mutation of the width at the connection, take the second - order derivatives of the tangent line set (angle11, angle12, …, angle1 n ) and the width set (width1, width2, …, width n ) respectively. Stop the calculation when the inflection points occur simultaneously at the corresponding points for the first time. This position is the optimal cutting point between the leaf and the stem. The cutting line can be obtained from the area passed by the perpendicular tangent line, thereby obtaining the leaf image image9;

[0133] Based on the obtained leaf image, calculate the contour information of each leaf, calculate the main vein skeleton of each leaf, and obtain the leaf phenotypic information of each plant. The leaf phenotypic information includes leaf length, leaf width, leaf perimeter, leaf area, and the number of leaves of the whole plant. Of course, it can also include other information such as leaf defect information, which is not specifically limited in the embodiments of the present invention.

[0134] In one embodiment, the steps of separating the leaf from the binary image of the plant to obtain the binary image of the stem and obtaining the stem phenotypic information by extracting the topological structure of the stem skeleton and using the perpendicular tangent measurement method are as follows:

[0135] Obtain the intersection path with the shortest distance from any point on the leaf skeleton graph to the stem. Make a tangent line at each point on the intersection path. The area passed by the perpendicular line perpendicular to this tangent line is the leaf width. Determine the cutting line at the separation between the leaf and the stem based on all tangent angles and leaf width information. Separate the leaf according to this cutting line to obtain the leaf contour and skeleton structure information until the leaf information is obtained;

[0136] Separate the leaf information in the leaf skeleton diagram to obtain the stalk binary diagram, and then obtain the stalk information, where the stalk information includes the stalk skeleton and the set of endpoints of the stalk skeleton;

[0137] Obtain the stalk thickness at each position of the stalk according to the intersection information of the perpendicular lines cut at each position of the stalk skeleton and the stalk binary diagram;

[0138] Obtain the single-source path between two points according to the growth point of the stalk skeleton and the base point of the stalk. Select the path with the longest length among the single-source paths, which is the path from the starting point to the ending point of the stalk, and the length corresponding to this path is the plant height.

[0139] This step is actually the same as the single-source path search method and the perpendicular line cutting measurement method in step S400 in terms of principle. The difference between the detection scheme and R4 lies in the plant height path search logic. Specifically, according to the obtained leaf information, remove it from the original binary diagram to obtain the stalk binary diagram image10, that is, the stalk binary diagram image10 = the plant binary diagram image5 - the leaf image image9; obtain the topological structure and endpoint information of the stalk image skeleton, calculate the path with the longest length among the paths formed in the set of endpoints, which is the path from the starting point (growth point) to the ending point (stalk base) of the stalk, and the corresponding length is the plant height; similarly, according to the stalk path information, calculate the width of the perpendicular line cut at each position, which is the stalk thickness at each position of the stalk. Thus, the plant height and stalk thickness information of the stalk phenotypic information are obtained. Of course, the growth and defect information of the stalk can be detected by performing morphological analysis, texture analysis, chromaticity analysis, etc. on the stalk image. The embodiments of the present invention do not make specific limitations.

[0140] In addition, attached Figure 9 is the schematic diagram of the perpendicular line cutting measurement principle in the embodiments of the present invention, which is used to obtain the leaf cutting line information and the stalk thickness measurement information. Specifically: Take any point P on the skeleton (black dotted line) at a certain part (green) of the plant in the figure as an example. Fit the tangent line at this point according to the foreground pixel distribution in the area where point P is located. Inversely solve the perpendicular line slope of the corresponding tangent line according to the tangent line slope, and obtain the perpendicular line cut by combining the coordinates of point P. The intersection path of the perpendicular line cut and the plant is the width of the plant at this position. According to the obtained plant width and tangent line slope information, combined with the structural characteristics of different parts of the plant, obtain the leaf cutting line information and the stalk thickness measurement information.

[0141] Embodiment 2:

[0142] A plant phenotypic measurement system based on computer vision, as Figure 2 shown, includes an image correction module 100, an image segmentation module 200, a skeleton topological structure extraction module 300, a leaf detection module 400, and a stalk detection module 500;

[0143] The image correction module 100 is used to obtain the original plant image and perform correction processing in combination with four identifiers to obtain the corrected plant image, wherein the four identifiers are at the four corners of the original plant image;

[0144] The image segmentation module 200 is used to perform segmentation processing on the corrected plant image to obtain a binary plant image;

[0145] The skeleton topological structure extraction module 300 obtains a skeleton topological structure image based on the binary plant image, and further obtains relevant information of the plant, where the relevant information at least includes skeleton information and an intersection set;

[0146] The leaf detection module 400 performs single-source path analysis and the perpendicular cutting line measurement method based on the skeleton information and the intersection set to obtain the cutting line for separating the leaf from the stem, separates each leaf from the connection between the petiole and the stem to obtain the contour information of each independent leaf, and combines the skeleton topological structure to obtain the plant leaf phenotypic information, where the plant leaf phenotypic information includes width, length, perimeter, area, and the total number of leaves of the whole plant;

[0147] The stem detection module 500 strips the leaves from the binary plant image to obtain a binary stem image, and obtains the stem phenotypic information by extracting the stem skeleton topological structure and using the perpendicular cutting line measurement method, where the stem phenotypic information at least includes plant height and stem diameter.

[0148] Example 3:

[0149] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0150] S100. Obtain the original plant image and perform correction processing in combination with four identifiers to obtain the corrected plant image, wherein the four identifiers are at the four corners of the original plant image;

[0151] S200. Perform segmentation processing on the corrected plant image to obtain a binary plant image;

[0152] S300. Obtain a skeleton topological structure image based on the binary plant image, and further obtain relevant information of the plant, where the relevant information at least includes skeleton information and an intersection set;

[0153] S400. Perform single-source path analysis and the perpendicular cutting line measurement method based on the skeleton information and the intersection set to obtain the cutting line for separating the leaf from the stem, separate each leaf from the connection between the petiole and the stem to obtain the contour information of each leaf, and combine the skeleton topological structure to obtain the plant leaf phenotypic information, where the plant leaf phenotypic information includes width, length, perimeter, area, and the total number of leaves of the whole plant;

[0154] S500. Strip the contour information of the leaves from the binary image of the plant to obtain the binary image of the stem. Obtain the stem phenotype information by extracting the topological structure of the stem skeleton and using the tangent perpendicular measurement method. Among them, the stem phenotype information at least includes plant height and stem diameter.

[0155] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0156] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The present invention is described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate for implementing in the process Figure 1 a process or multiple processes and / or blocks Figure 1 a block or multiple blocks the device with the specified functions.

[0158] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements in the process Figure 1 a process or multiple processes and / or blocks Figure 1 a block or multiple blocks the specified functions.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide for implementing in the process Figure 1 a process or multiple processes and / or blocks Figure 1Steps of the functions specified in one or more boxes.

[0160] It should be noted that:

[0161] The "one embodiment" or "embodiments" mentioned in the specification means that the specific features, structures or characteristics described in connection with the embodiments are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiments" that appear throughout the specification do not necessarily refer to the same embodiment.

[0162] In addition, it should be noted that for the specific embodiments described in this specification, the shapes, names of the components, etc. can be different. Any equivalent or simple changes made according to the structure, features and principles described in the inventive concept of the present invention are included in the protection scope of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the described specific embodiments or use similar ways to substitute, as long as they do not deviate from the structure of the present invention or exceed the scope defined by this claim, they should fall within the protection scope of the present invention.

Claims

1. A plant phenotype measurement method based on computer vision, characterized in that, The method includes the following steps: Obtain the original plant image and perform calibration processing in combination with four identifiers to obtain the calibrated plant image, wherein the four identifiers are located at the four corners of the original plant image; Perform segmentation processing on the calibrated plant image to obtain a binary plant image; Based on the binary plant image, obtain a skeleton topological structure image, and further obtain relevant information of the plant, where the relevant information at least includes skeleton information and an intersection set; Based on the skeleton information and the intersection set, perform single-source path analysis and the perpendicular bisector measurement method to obtain the cutting line for separating the leaf from the stem, separate each leaf from the connection between the petiole and the stem to obtain the contour information of each leaf, and combine the skeleton topology to obtain the plant leaf phenotypic information, where the plant leaf phenotypic information includes width, length, perimeter, area, and the total number of leaves of the whole plant; Strip the contour information of the leaf from the binary plant image to obtain a binary stem image, and obtain the stem phenotypic information by extracting the stem skeleton topology and using the perpendicular bisector measurement method, where the stem phenotypic information at least includes plant height and stem diameter; Perform skeleton extraction processing on the binary plant image to obtain a skeleton topology; Perform plant skeleton topology analysis on the skeleton topology to obtain an endpoint set and an intersection set, where the endpoint set is the set of all leaf tip endpoints, plant stem growth points, and stem base endpoints, and the intersection set is the set of intersections of the leaf with the petiole and the stem connection; Perform distance transformation on the path points in the skeleton topology in the binary plant image to obtain a result image, where the result image is a skeleton topology diagram excluding the petiole and the stem, and further obtain a leaf skeleton diagram; By selecting and analyzing the features of the leaf skeleton diagram, obtain the skeleton information characterizing each leaf; Denote the set of intersection points as (c1, c2,..., c n ), and denote the skeleton information as (p1, p2,..., p k ); Obtain a single-source path set formed by any point in the skeleton information and the intersection set, and select the path with the shortest single-source path length from the single-source path set. This single-source path is the connection point between the leaf and the stem; Based on the single-source path set, traverse from the end point of the single-source path to the starting point direction to obtain the tangents and corresponding slopes of each point on the single-source path; Obtain the perpendicular slope passing through the corresponding point according to the slope and draw a perpendicular line. The total number of point sets of the connected domain passed by the perpendicular line represents the leaf width or the width at the connection between the leaf and the stem; According to the mutation of the angle and the mutation of the width at the connection between the leaf and the stem, take the second derivative of each point tangent set and width set respectively. When the inflection point occurs simultaneously at the corresponding points for the first time, stop the calculation. Then the corresponding positions of each point are the optimal points for cutting the leaf and the stem, obtain the cutting line, and further obtain each leaf image; According to each leaf image, obtain the contour information and main vein skeleton structure information of each leaf, and further obtain the plant leaf phenotypic information of each plant; 2. The method for measuring plant phenotypes based on computer vision according to claim 1, characterized in that The calibration process in combination with the four identifiers includes the following steps: Set the central coordinates of the four identifiers as C1, C2, C3, and C4 in clockwise or counterclockwise order. Among them, C1 is the central coordinate of the first corner identifier, C2 is the central coordinate of the second corner identifier, C3 is the central coordinate of the third corner identifier, and C4 is the central coordinate of the fourth corner identifier. And set the aspect ratio of the rectangle formed by the central coordinates as a:b; Perform rectangular fitting on the central coordinates C1 and C4 of two adjacent corners of the identifier to obtain the transformed central coordinates C2′ and C3′; Obtain the transformation matrix based on the central coordinate set [C1, C2, C3, C4] and the transformed central coordinate set [C1, C2′, C3′, C4]; Perform global transformation processing on the original plant image based on the transformation matrix to obtain the second image; Set the aspect ratio of the rectangle formed by the transformed central coordinate set as a:c, and combine the aspect ratio a:b to perform scale transformation on the width of the second image with a transformation ratio of b / c to obtain the corrected plant image.

3. The method for measuring plant phenotypes based on computer vision according to claim 1, characterized in that, The segmentation process of the corrected plant image includes the following steps: Perform channel separation processing on the corrected plant image to obtain grayscale images of the R, G, and B channels respectively; Perform weighted average calculation on the grayscale images of the R, G, and B channels to obtain the luminance image; Obtain the plant color representation map according to the color characteristics of the plant; Obtain the transformation coefficient based on the luminance image and the plant color representation map, obtain the second grayscale image based on the transformation coefficient, and then obtain the plant binary image through automatic threshold segmentation processing.

4. The method for measuring plant phenotypes based on computer vision according to claim 1, characterized in that The process of peeling the leaves from the plant binary image to obtain the stem binary image and obtaining the stem phenotype information by extracting the stem skeleton topological structure and using the tangent perpendicular measurement method includes the following steps: Obtain the intersection path with the shortest distance from any point in the leaf skeleton map to the stem. Make a tangent line to the points at each position of the intersection path, and the area passed through by the perpendicular line perpendicular to the tangent line is the leaf width. Determine the cutting line at the separation of the leaf and the stem according to the tangent angle and leaf width information. Separate the leaf according to this cutting line to obtain the leaf contour and skeleton structure information until the leaf information is obtained; Separate the leaf information in the leaf skeleton map to obtain the stem binary image, and then obtain the stem information. The stem information includes the stem skeleton and the stem skeleton endpoint set; Obtain the stem diameter at each position of the stem according to the intersection information of the perpendicular tangent line at each position of the stem skeleton and the stem binary image; Obtain the single-source path between two points according to the growth point of the stem skeleton and the base point of the stem. Select the longest path among the single-source paths, which is the path from the starting point to the end point of the stem, and the length corresponding to this path is the plant height.

5. The plant phenotype measurement method based on computer vision according to claim 1, characterized in that, The four identifiers are set in the image acquisition device. The image acquisition device includes a background board. The background board contains 4 identifiers. The identifiers are distributed at the four corners of the background board. The color of the identifier has a distinct contrast with the color of the background board. The aspect ratio of the rectangle formed by connecting the centers of the identifiers is 4:

3.

6. A plant phenotype measurement system based on computer vision, characterized in that, Including an image correction module, an image segmentation module, a skeleton topological structure extraction module, a leaf detection module, and a stem detection module; An image correction module, configured to obtain an original plant image and perform correction processing in combination with four identifiers to obtain a corrected plant image, wherein the four identifiers are at the four corners of the original plant image; An image segmentation module, configured to perform segmentation processing on the corrected plant image to obtain a binary plant image; A skeleton topological structure extraction module, which obtains a skeleton topological structure image based on the binary plant image, and further obtains relevant information of the plant, where the relevant information at least includes skeleton information and an intersection set; A leaf detection module, based on the skeleton information and the intersection set, performs single-source path analysis and perpendicular cutting line measurement method to obtain a cutting line for separating the leaf from the stem, separates each leaf from the connection between the petiole and the stem to obtain the contour information of each independent leaf, and combines the skeleton topology to obtain the plant leaf phenotypic information, where the plant leaf phenotypic information includes width, length, perimeter, area, and the number of leaves of the whole plant; a stem detection module, strips the leaves from the binary plant image to obtain a binary stem image, and obtains the stem phenotypic information by extracting the stem skeleton topology and using the perpendicular cutting line measurement method, where the stem phenotypic information at least includes plant height and stem diameter; Perform skeleton extraction processing on the binary plant image to obtain a skeleton topological structure; Perform plant skeleton topological structure analysis on the skeleton topological structure to obtain an end point set and an intersection set, where the end point set is a set of all leaf tip end points, plant stem growth points, and stem base end points, and the intersection set is a set of intersections of the leaf with the petiole and the stem connection; Perform distance transformation on the path points in the skeleton topological structure in the binary plant image to obtain a result image, where the result image is a skeleton topological structure diagram excluding the petiole and the stem, and further obtain a leaf skeleton diagram; By selecting and analyzing the features of the leaf skeleton diagram, obtain the skeleton information characterizing each leaf; Let the set of intersection points be denoted as (c1, c2,..., c n ), and the skeleton information be denoted as (p1, p2,..., p k ); Obtain a single-source path set formed by any point in the skeleton information and the intersection set, and select the path with the shortest single-source path length from the single-source path set, and this single-source path is the connection point between the leaf and the stem; Based on the single-source path set, traverse from the end point of the single-source path to the starting point direction to obtain the tangent lines and corresponding slopes of each point on the single-source path; Obtain the perpendicular slope passing through the corresponding point according to the slope and draw a perpendicular line, and the total number of point sets of the connected domain passed by the perpendicular line represents the leaf width or the width at the connection between the leaf and the stem; According to the mutation of the angle at the connection between the leaf and the stem and the mutation of the width at the connection, take the second derivative of each point tangent set and width set respectively. When the inflection point occurs simultaneously at the corresponding points for the first time, stop the calculation. Then the corresponding positions of each point are the best points for cutting the leaf and the stem, obtain the cutting line, and further obtain each leaf image; According to each leaf image, obtain the contour information and main vein skeleton structure information of each leaf, and further obtain the plant leaf phenotypic information of each plant.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method steps described in any one of claims 1 to 5.

8. A plant phenotype measurement device based on computer vision, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method steps described in any one of claims 1 to 5.

Images