Hemerocallis citrina baroni phenotype measuring method based on machine vision

Abstract

The invention discloses a daylily phenotype measurement method based on machine vision, and relates to the technical field of machine vision size detection, and the method comprises the following steps: adjusting an image collection device, and obtaining a mapping relation between a pixel size and a physical size; then collecting image data of daylily buds; preprocessing the image data into a binary image to obtain a contour image of the flower bud; according to the intersection points of the plurality of line segments and the flower bud edge line, calculating ridge line coordinate points of the flower bud contour, and calculating the pixel length of the flower bud; according to the intersection points of the plurality of line segments and the edge lines of the flower buds, calculating the maximum pixel width of the flower buds by solving the feature points on the two edge lines which enable the pixel distance between the coordinates of the intersection points to be maximum; converting the mapping relation into a physical distance to obtain the length and the maximum width of the flower bud; and finally, on the basis of the three characteristics of the bud pixel area, the bud length and the maximum width and the true-value bud fresh weight, fitting a linear function according to a least square method to predict the bud fresh weight.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision size detection, in particular to a daylily phenotype measurement method based on machine vision. Background Art

[0002] The phenotypic trait parameters of daylily buds are a direct reflection of the agronomic traits of daylily during its growth process, and play a key role in the breeding of high-quality daylily varieties. Traditional daylily bud phenotypic measurement faces many difficulties. The manual measurement method has problems such as low detection efficiency and high labor intensity, and can no longer meet the needs of modern crop phenotypic characteristic measurement.

[0003] When measuring the phenotypic parameters of daylily buds manually, it is difficult to obtain high-precision data due to the visual errors of the human eye and the accuracy limitations of the measuring tools. Moreover, when measuring irregular shape parameters such as the width of the buds, manual measurement can often only obtain rough estimates and cannot accurately reflect the true morphological characteristics of the buds. This has brought great obstacles to the work that requires accurate data for variety selection and growth research.

[0004] In the traditional process of manually measuring the phenotype of daylily buds, data recording mainly relies on manual paper and pen records; a large number of paper records need to be manually entered into spreadsheets one by one for analysis, which is time-consuming and labor-intensive, and increases the risk of errors in the data transcription process; at the same time, paper records are not conducive to the long-term preservation and sharing of data, and it is difficult to achieve efficient management and utilization of data, which cannot meet the needs of modern crop research for efficient data processing and comprehensive analysis.

[0005] Based on the above problems, the present invention proposes a method for measuring daylily phenotype based on machine vision. Summary of the invention

[0006] The object of the present invention is to provide a method for measuring the phenotype of daylily based on machine vision to solve the problems raised in the prior art.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A method for measuring the phenotype of daylily based on machine vision comprises the following steps:

[0009] S1. Adjust the acquired image device to obtain the mapping relationship between pixel size and physical size;

[0010] S2, collecting image data of daylily buds;

[0011] S3, preprocessing the image data into a binary image; then screening the contours that meet the conditions according to the pixel area size of the connected area to obtain the contour image of the flower bud;

[0012] S4, according to the intersection points of the multiple line segments and the edge line of the flower bud, find the ridge coordinate points of the flower bud outline, obtain the highest point coordinate and the lowest point coordinate of the flower bud outline, and calculate the pixel length of the flower bud;

[0013] S5. According to the intersection points of the multiple line segments and the edge line of the flower bud, find the feature points on the two edge lines that make the pixel distance between the intersection point coordinates the largest, and calculate the maximum pixel width of the flower bud based on the two feature points;

[0014] S6, converting the physical distance into the length and maximum width of the flower bud according to the mapping relationship;

[0015] S7. Based on the three features of flower bud pixel area, flower bud length and maximum width and the true value of flower bud fresh weight, the flower bud fresh weight is predicted by fitting the linear function according to the least squares method.

[0016] S1.1: Connect the industrial camera to the computer and adjust the parameters through the camera control software in the computer;

[0017] S1.2: Adjust the object distance and focal length, as well as the light source, so that the light distribution in the camera's field of view is uniform;

[0018] S1.3: Calculate the width and height of the camera's field of view based on the object distance, focal length, horizontal field of view angle, and vertical field of view angle. The calculation formula for the width W of the camera's field of view is:

[0019]

[0020] Where d is the distance from the camera to the object, θ h is the horizontal field of view;

[0021] The calculation formula of the camera field of view height H is:

[0022]

[0023] Where d is the distance from the camera to the object, θ v is the vertical field of view;

[0024] S1.4: Calculate the mapping relationship between the pixel size and the physical size according to the resolution X of the camera width. The calculation formula of the mapping relationship is as follows:

[0025]

[0026] Among them, R is the mapping relationship, which represents the mapping relationship between the actual object size in the image and the corresponding pixel size in the image, and W is the width of the camera field of view.

[0027] The method of collecting the image data of the daylily buds comprises the following steps:

[0028] S2.1: Place the flower bud on a black curtain, with the stem of the flower bud parallel to the y-axis of the pixel coordinate system and the head of the flower bud pointing to the x-axis of the pixel coordinate system;

[0029] S2.2: Use an industrial camera to capture the original color image of the flower bud.

[0030] The preprocessing of the image data into a binary image comprises the following steps:

[0031] S3.1: First, grayscale the collected image data, then use the maximum inter-class variance method to obtain the default threshold for binarization of the grayscale processed image, and use median filtering to remove speckle noise and salt and pepper noise;

[0032] S3.2: The processed data is subjected to morphological processing of first closing operation and then opening operation to remove small black dots in the outline, and the outline of the flower buds is screened out according to the pixel area of ​​the connected area. The screening condition is: the pixel area of ​​the connected area is between 5000 and 25000, which is the flower bud outline area, otherwise it is discarded; the setting of the screening condition is based on the occupancy ratio of the flower bud pixels in the original image of the present invention.

[0033] The method of obtaining the ridge coordinate points of the flower bud outline according to the intersection points of the multiple line segments and the flower bud edge line, obtaining the highest point coordinates and the lowest point coordinates of the flower bud outline, and calculating the pixel length of the flower bud comprises the following steps:

[0034] S4.1: According to the screening result of step S3.2, the four corner coordinates, deflection angle, width and height of the minimum circumscribed rectangle of the flower bud outline are obtained; then the four corner coordinates and the rectangular side of the minimum circumscribed rectangle are redefined according to the deflection angle of the minimum circumscribed rectangle, so that no matter what the deflection direction of the minimum circumscribed rectangle is, the position definition of the four corner coordinates and the rectangular side of the rectangle will not change due to the deflection direction; and the short side of the minimum circumscribed rectangle is extended by 50 pixels to both sides, and the long side is moved by 50 pixels to both sides according to the direction of the short side;

[0035] S4.2: Let the intercepts of the two short sides of the minimum circumscribed rectangle be C0 and C1. Make an arithmetic progression from small to large within the intercept range. Considering the calculation time and the size measurement error, divide the intercept range into 80 equal parts. Store the data in the list L1, L1 = [C0, C1] i (i=1,2...78),C 79 ], list L1 stores the short side intercepts of the minimum circumscribed rectangle C0, C 79 and 78 intercept values ​​C generated by arithmetic progression i , do C iis the intercept, and the slope is equal to the 78 line segments of the short side of the minimum circumscribed rectangle, and the two ends of the 78 line segments are respectively located on the long sides of the minimum circumscribed rectangle;

[0036] S4.3: According to the processing result of step S4.2, each line segment has two intersection coordinates with the edge lines on both sides of the flower bud, and the intersections of 78 line segments with the edge lines of the flower bud contour have 78 coordinate pairs. The middle coordinate point of the two coordinates in each coordinate pair is calculated; the coordinates of the 78 middle points are obtained and stored in the list L2;

[0037] S4.4: Obtain the highest and lowest coordinate points of the flower bud outline according to the minimum circumscribed rectangle. The method is as follows: traverse all pixel coordinate points of the flower bud outline, find the two coordinate points with the maximum y value and the minimum y value, the coordinate point with the maximum y value is the lowest coordinate point, and the coordinate point with the minimum y value is the highest coordinate point; then insert the coordinate of the highest point into the head of list L2, and insert the coordinate of the lowest point into the tail of list L2, and obtain a total of 80 intermediate coordinate points that constitute the ridge features of the flower bud outline, and then calculate the distances between adjacent coordinate points in L2 in turn and accumulate them to obtain the pixel length of the flower bud.

[0038] The method of finding the feature points on the two edge lines that make the pixel distance between the intersection coordinates of the multiple line segments and the edge line of the flower bud the largest according to the intersection points of the multiple line segments and the edge line of the flower bud, and calculating the maximum pixel width of the flower bud according to the two feature points comprises the following steps:

[0039] S5.1: Calculate the pixel distances between the coordinates of the intersections of the line segment and the edge of the flower bud contour in sequence using the Euclidean distance calculation formula, and find the two coordinate points (x1, y1) and (x2, y2) that maximize the pixel distance;

[0040] S5.2: Calculate the tangents of the two coordinate points on the edge line according to the bilateral difference method, draw a perpendicular line through the two coordinate points and the tangent line, and generate two other intersection coordinates (x1`, y1`) and (x2`, y2`) between the perpendicular line and the contour edge line. Use the Euclidean distance calculation formula to calculate the distance D1 between (x1, y1) and (x1`, y1`), and the distance D2 between (x2, y2) and (x2`, y2`);

[0041] S5.3: Finally, the two pixel distances D1 and D2 are averaged to obtain an approximate estimate of the maximum width of the flower bud.

[0042] The step of converting the mapping relationship into physical distance to obtain the length and maximum width of the flower bud comprises the following steps:

[0043] S6.1: According to the mapping relationship between the pixel size and the physical size described in S1, the pixel length and the maximum width of the daylily bud are converted into the actual physical length in mm to complete the phenotypic measurement of the daylily bud. The calculation formula of the mapping relationship described in S1 is:

[0044] Where X is the width resolution of the camera in pixels, W is the width scene range seen through the camera lens in mm, and R represents the mapping relationship between the actual object size in the image and the corresponding pixel size in the image.

[0045] The method of predicting the fresh weight of flower buds by fitting a linear function using the least square method based on the three features of the flower bud pixel area, the flower bud length and the maximum width and the true value of the fresh weight of the flower buds comprises the following steps:

[0046] S7.1: Given n sets of data, each set of data contains three independent variables and one dependent variable. The independent variables are the pixel area of ​​the flower bud a1, the length of the flower bud a2 and the maximum width of the flower bud a3. The dependent variable is the fresh weight of the flower bud b.

[0047] S7.2: Normalize the data of independent and dependent variables;

[0048] S7.3: Based on the data normalization results, the least squares method is used to minimize the sum of squares of the residuals between the n response values ​​of b and the values ​​calculated by its fitting model, and the independent variable coefficients and constant terms are solved.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. Improve measurement efficiency: The present invention constructs an automated phenotypic measurement method based on machine vision, which can automatically obtain the phenotypic data of daylily flower buds in a high-throughput manner. Compared with the traditional manual measurement method, it greatly reduces the measurement time, can process a large number of flower bud samples in a short time, and significantly improves the measurement efficiency.

[0051] 2. Save manpower and material resources: Using machine vision technology to replace manual measurement avoids the labor intensity problem caused by manual operation. It does not require a large amount of manpower to be invested in tedious measurement work, and it also reduces the material resources such as various measurement tools that may be required for manual measurement, saving manpower and material costs.

[0052] 3. Provide accurate and reliable data: The length and maximum width of the flower buds are calculated by the pixel coordinate point distance measurement method and mapping relationship, and the fresh weight of the flower buds is predicted using the fitting function, so that more accurate data can be obtained. Compared with the traditional manual measurement, which is greatly affected by human factors and has low precision, the present invention provides more accurate and reliable relevant information for the breeding of daylily varieties, which is conducive to the development of variety breeding work. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the definition of the short side and the long side of the minimum circumscribed rectangle under different deflection directions of a method for measuring the daylily phenotype based on machine vision of the present invention;

[0054] Figure 2 It is a schematic diagram of the definition of the four-corner coordinates of the minimum circumscribed rectangle under different deflection directions of a daylily phenotype measurement method based on machine vision of the present invention;

[0055] Figure 3 It is a schematic diagram of the four corner coordinates of the minimum circumscribed rectangle of the daylily phenotype measurement method based on machine vision of the present invention after redefinition;

[0056] Figure 4 A schematic diagram of the four corner coordinates and four sides of the rectangle after the minimum circumscribed rectangle is redefined in a method for measuring daylily phenotype based on machine vision of the present invention;

[0057] Figure 5 It is a schematic diagram of the minimum circumscribed rectangle of the daylily phenotype measurement method based on machine vision of the present invention after the short side is expanded and the long side is moved;

[0058] Figure 6 A line segment (M, N) drawing diagram and a pixel coordinate drawing diagram of a daylily phenotype measurement method based on machine vision of the present invention;

[0059] Figure 7 A schematic diagram of a pair of intersection coordinates generated by a line segment and a contour edge in a method for measuring daylily phenotype based on machine vision of the present invention;

[0060] Figure 8 A schematic diagram of 80 ridge coordinate points of a method for measuring daylily phenotype based on machine vision according to the present invention;

[0061] Fig. 9 This is a statistical result diagram of the bud length measurement of a daylily phenotype measurement method based on machine vision of the present invention;

[0062] Fig.10 This is a statistical result diagram of bud width measurement of a daylily phenotype measurement method based on machine vision of the present invention;

[0063] Fig.11 A residual graph of the pixel area, length and maximum width of the flower buds of a method for measuring the daylily phenotype based on machine vision according to the present invention;

[0064] Fig.12 This is a statistical result diagram of flower bud fresh weight prediction of a daylily phenotype measurement method based on machine vision in the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Example: Figure 1-Figure 12 As shown, the present invention provides a technical solution, a method for measuring the phenotype of daylily based on machine vision;

[0067] A method for measuring the phenotype of daylily based on machine vision comprises the following steps:

[0068] S1. Adjust the acquired image device to obtain the mapping relationship between pixel size and physical size;

[0069] S2, collecting image data of daylily buds;

[0070] S3, preprocessing the image data into a binary image; then screening the contours that meet the conditions according to the pixel area size of the connected area to obtain the contour image of the flower bud;

[0071] S4, according to the intersection points of the multiple line segments and the edge line of the flower bud, find the ridge coordinate points of the flower bud outline, obtain the highest point coordinate and the lowest point coordinate of the flower bud outline, and calculate the pixel length of the flower bud;

[0072] S5. According to the intersection points of the multiple line segments and the edge line of the flower bud, find the feature points on the two edge lines that make the pixel distance between the intersection point coordinates the largest, and calculate the maximum pixel width of the flower bud based on the two feature points;

[0073] S6, converting the physical distance into the length and maximum width of the flower bud according to the mapping relationship;

[0074] S7. Based on the three features of flower bud pixel area, flower bud length and maximum width and the true value of flower bud fresh weight, the flower bud fresh weight is predicted by fitting the linear function according to the least squares method.

[0075] S1.1: Connect the industrial camera to the computer and adjust the parameters through the camera control software in the computer;

[0076] S1.2: Adjust the object distance and focal length, as well as the light source, so that the light distribution in the camera's field of view is uniform;

[0077] S1.3: Calculate the width and height of the camera's field of view based on the object distance, focal length, horizontal field of view angle, and vertical field of view angle. The calculation formula for the width W of the camera's field of view is:

[0078]

[0079] Where d is the distance from the camera to the object, θ h is the horizontal field of view;

[0080] The calculation formula of the camera field of view height H is:

[0081]

[0082] Where d is the distance from the camera to the object, θ v is the vertical field of view;

[0083] S1.4: Calculate the mapping relationship between the pixel size and the physical size according to the resolution X of the camera width. The calculation formula of the mapping relationship is as follows:

[0084]

[0085] Among them, R is the mapping relationship, which represents the mapping relationship between the actual object size in the image and the corresponding pixel size in the image, and W is the width of the camera field of view.

[0086] The method of collecting the image data of the daylily buds comprises the following steps:

[0087] S2.1: Place the flower bud on a black curtain, with the stem of the flower bud parallel to the y-axis of the pixel coordinate system and the head of the flower bud pointing to the x-axis of the pixel coordinate system;

[0088] S2.2: Use an industrial camera to capture the original color image of the flower bud.

[0089] The preprocessing of the image data into a binary image comprises the following steps:

[0090] S3.1: First, grayscale the collected image data, then use the maximum inter-class variance method to obtain the default threshold for binarization of the grayscale processed image, and use median filtering to remove speckle noise and salt and pepper noise;

[0091] S3.2: The processed data is subjected to morphological processing of first closing operation and then opening operation to remove small black dots in the outline, and the outline of the flower buds is screened out according to the pixel area of ​​the connected area. The screening condition is: the pixel area of ​​the connected area is between 5000 and 25000, which is the flower bud outline area, otherwise it is discarded; the setting of the screening condition is based on the occupancy ratio of the flower bud pixels in the original image of the present invention.

[0092] The method of obtaining the ridge coordinate points of the flower bud outline according to the intersection points of the multiple line segments and the flower bud edge line, obtaining the highest point coordinates and the lowest point coordinates of the flower bud outline, and calculating the pixel length of the flower bud comprises the following steps:

[0093] S4.1: According to the screening result of step S3.2, the four corner coordinates, deflection angle, width and height of the minimum circumscribed rectangle of the flower bud outline are obtained; then the four corner coordinates and the rectangular side of the minimum circumscribed rectangle are redefined according to the deflection angle of the minimum circumscribed rectangle, so that no matter what the deflection direction of the minimum circumscribed rectangle is, the position definition of the four corner coordinates and the rectangular side of the rectangle will not change due to the deflection direction; and the short side of the minimum circumscribed rectangle is extended by 50 pixels to both sides, and the long side is moved by 50 pixels to both sides according to the direction of the short side;

[0094] S4.2: Let the intercepts of the two short sides of the minimum circumscribed rectangle be C0 and C1. Make an arithmetic progression from small to large within the intercept range. Considering the calculation time and the size measurement error, divide the intercept range into 80 equal parts. Store the data in the list L1, L1 = [C0, C1] i (i=1,2...78),C 79 ], list L1 stores the short side intercepts of the minimum circumscribed rectangle C0, C 79 and 78 intercept values ​​C generated by arithmetic progression i , do C i is the intercept, and the slope is equal to the 78 line segments of the short side of the minimum circumscribed rectangle, and the two ends of the 78 line segments are respectively located on the long sides of the minimum circumscribed rectangle;

[0095] S4.3: According to the processing result of step S4.2, each line segment has two intersection coordinates with the edge lines on both sides of the flower bud, and the intersections of 78 line segments with the edge lines of the flower bud contour have 78 coordinate pairs. The middle coordinate point of the two coordinates in each coordinate pair is calculated; the coordinates of the 78 middle points are obtained and stored in the list L2;

[0096] S4.4: Obtain the highest and lowest coordinate points of the flower bud outline according to the minimum circumscribed rectangle. The method is as follows: traverse all pixel coordinate points of the flower bud outline, find the two coordinate points with the maximum y value and the minimum y value, the coordinate point with the maximum y value is the lowest coordinate point, and the coordinate point with the minimum y value is the highest coordinate point; then insert the coordinate of the highest point into the head of list L2, and insert the coordinate of the lowest point into the tail of list L2, and obtain a total of 80 intermediate coordinate points that constitute the ridge features of the flower bud outline, and then calculate the distances between adjacent coordinate points in L2 in turn and accumulate them to obtain the pixel length of the flower bud.

[0097] The method of finding the feature points on the two edge lines that make the pixel distance between the intersection coordinates of the multiple line segments and the edge line of the flower bud the largest according to the intersection points of the multiple line segments and the edge line of the flower bud, and calculating the maximum pixel width of the flower bud according to the two feature points comprises the following steps:

[0098] S5.1: Calculate the pixel distances between the coordinates of the intersections of the line segment and the edge of the flower bud contour in sequence using the Euclidean distance calculation formula, and find the two coordinate points (x1, y1) and (x2, y2) that maximize the pixel distance;

[0099] S5.2: Calculate the tangents of the two coordinate points on the edge line according to the bilateral difference method, draw a perpendicular line through the two coordinate points and the tangent line, and generate two other intersection coordinates (x1`, y1`) and (x2`, y2`) between the perpendicular line and the contour edge line. Use the Euclidean distance calculation formula to calculate the distance D1 between (x1, y1) and (x1`, y1`), and the distance D2 between (x2, y2) and (x2`, y2`);

[0100] S5.3: Finally, the two pixel distances D1 and D2 are averaged to obtain an approximate estimate of the maximum width of the flower bud.

[0101] The step of converting the mapping relationship into physical distance to obtain the length and maximum width of the flower bud comprises the following steps:

[0102] S6.1: According to the mapping relationship between the pixel size and the physical size described in S1, the pixel length and the maximum width of the daylily bud are converted into the actual physical length in mm to complete the phenotypic measurement of the daylily bud. The calculation formula of the mapping relationship described in S1 is:

[0103] Where X is the width resolution of the camera in pixels, W is the width scene range seen through the camera lens in mm, and R represents the mapping relationship between the actual object size in the image and the corresponding pixel size in the image.

[0104] The method of predicting the fresh weight of flower buds by fitting a linear function using the least square method based on the three features of the flower bud pixel area, the flower bud length and the maximum width and the true value of the fresh weight of the flower buds comprises the following steps:

[0105] S7.1: Given n sets of data, each set of data contains three independent variables and one dependent variable. The independent variables are the pixel area of ​​the flower bud a1, the length of the flower bud a2 and the maximum width of the flower bud a3. The dependent variable is the fresh weight of the flower bud b.

[0106] S7.2: Normalize the data of independent and dependent variables;

[0107] S7.3: Based on the data normalization results, the least squares method is used to minimize the sum of squares of the residuals between the n response values ​​of b and the values ​​calculated by its fitting model, and the independent variable coefficients and constant terms are solved.

[0108] Example

[0109] S1. Connect the shooting device, obtain the internal and external parameters of the camera according to various factors of the shooting environment, and calculate the mapping relationship between the pixel size and the physical size.

[0110] In this example, the industrial camera model used is MV-CS200-10GC, with MVL-KF3528M-12MP lens. The pixel size is 2.4μm×2.4μm, the focal length is 35mm, the distance between the camera imaging unit and the measured flower bud is 680mm, and according to the calculation formula, the camera field of view range is: width is 255mm, height is 170mm. The image data in this embodiment is obtained in a dark box in the laboratory. The flower buds are placed on a black panel, and the light source, exposure, etc. are adjusted. A suitable light intensity should be selected to ensure a clear image.

[0111] S2. Collect image data of daylily buds.

[0112] S3, grayscale processing is performed on the image information of the flower bud obtained in S2, and a suitable threshold is calculated according to the grayscale information of all pixels in the grayscale image by the maximum inter-class variance method, and the grayscale image is converted into a binary image according to the threshold to obtain a flower bud outline image; wherein the flower bud area is white and the background is black. The salt and pepper noise is removed by median filtering, and then the image is processed by the method of closing operation first and then opening operation, and the small black dots in the white area are removed to obtain the flower bud outline image.

[0113] In this example, in order to obtain an image with the maximum difference between the bud outline area and the black background, it is necessary to smooth the image, which is conducive to extracting object features for analysis. The smoothing technology used in this example is median filtering. Median filtering replaces the grayscale value of the pixel with the median of the grayscale value of the pixel neighborhood. It is necessary to traverse each pixel in the image and calculate the grayscale value of the pixel according to the size of the mask. The size of the mask must be greater than 1 and must be an odd number. The larger the value, the stronger the filtering. According to the experimental process in this example, a mask of 3*3 is used to ensure that most of the salt and pepper noise is eliminated.

[0114] After the median filter operation, there may be some small black dots in the white area of ​​the flower buds, and there may also be some small white dots in the black area of ​​the background. Therefore, the method of closing operation first and then opening operation is used to fill the small black dots in the white area of ​​the flower buds with white. The specific process is:

[0115] S301, select a structural element. In this example, the environment is relatively fixed, and a 3*3 structural element is used, which has the effect of improving image quality without causing too much impact on the contour edge. The larger the size, the greater the impact on each pixel of the image, and the easier it is for the image to be distorted.

[0116] S302. Erosion is performed first and then dilation to obtain the opening operation, which can remove some small objects in the image, and can also be used to remove noise, isolated points, and burrs, while the overall position and shape remain unchanged.

[0117] S303. The closing operation is the opposite of the opening operation, that is, dilation is performed first and then erosion. It is usually used to fill small holes in the white area or erase small black dots in the white area. The closing operation first enlarges the white part and squeezes out the small black parts, and then restores some large black dot parts on this basis. The overall effect is to erase the small black dots on the white area.

[0118] S304. Processing the image with the closing operation first and then the opening operation can obtain the main object in the image while ensuring that the outline edge of the image is not distorted as much as possible, and can reduce the measurement error caused by filtering effects.

[0119] S4. According to the intersection points of multiple line segments and the flower bud edge line, find the ridge line coordinate points of the flower bud contour, obtain the highest point coordinate and the lowest point coordinate of the flower bud contour, and calculate the pixel length of the flower bud; the specific steps are as follows:

[0120] S401. After the image is processed in step S3, a binary contour with the largest difference between the daylily flower bud and the background will be obtained. In this example, the contour of the flower bud is accurately identified according to the size of the contour pixel area. Considering the image resolution and the size of the daylily flower bud contour pixel area, the screening condition for confirming the flower bud contour is: 5000 pixels < contour pixel area < 25000 pixels.

[0121] S402. Obtain the four corner coordinates, deflection angle, width, and height of the minimum bounding rectangle of the flower bud contour, and redefine the four corner pixel coordinate points and the rectangle sides of the minimum bounding rectangle according to the deflection direction of the minimum bounding rectangle.

[0122] S403. Determine the width and height of the minimum bounding rectangle according to the deflection direction of the minimum bounding rectangle. The confirmation method is to rotate the x-axis counterclockwise. The side that coincides with or is parallel to a certain side of the minimum bounding rectangle first is the width, as Figure 1 shown.

[0123] S404. Perform the pose determination of the minimum bounding rectangle. If width < height, the minimum bounding rectangle deflects to the left; if width > height, the minimum bounding rectangle deflects to the right. Considering the shape feature of the daylily flower bud is slender, the situation where width = height will not occur.

[0124] S405. Obtain the four vertex pixel coordinates and the four rectangle sides of the minimum bounding rectangle, and redefine them.

[0125] S406, according to the deflection direction of the minimum bounding rectangle, the four pixel coordinate points are matched with the positions of the rectangle vertices. In the coordinate system of OpenCV, the minimum bounding rectangle has the smallest vertical coordinate at top_point, the largest vertical coordinate at bottom_point, the smallest horizontal coordinate at left_point, and the largest horizontal coordinate at right_point; that is, Figure 2 shown.

[0126] S407, redefine the pixel coordinates of the four vertices and the names of the four rectangular edges to facilitate subsequent operations; Figure 3 As shown, Figure 3 The left picture is before definition, and the right picture is after definition.

[0127] S408. After redefining the pixel coordinates of the four vertices and the names of the four rectangle sides, the minimum circumscribed rectangle has a total of four line segments, namely Line1 = ((X1, Y1), (X2, Y2)), Line2 = ((X1, Y1), (X3, Y3)), Line3 = ((X2, Y2), (X4, Y4)), Line4 = ((X3, Y3), (X4, Y4)). It is known that Line1 is parallel to Line4, and Line1 and Line4 pass through the coordinates of the highest and lowest points of the daylily bud contour respectively. Figure 4 shown.

[0128] S409, calculate the function expression of the extension lines of Line1 and Line4, and find the intercepts C0 and C1 of the function expression of the extension lines of the two line segments. 79 The calculation formula is as follows:

[0129]

[0130] S410, according to C0 to C 79 The size of the 79 Make an arithmetic progression with 80 elements, and make 78 arithmetic progressions with values ​​greater than C0 and less than C. 79 And it is parallel to Line1 and Line4.

[0131] S411. According to the deflection direction of the minimum circumscribed rectangle, the two smallest sides of the minimum circumscribed rectangle are extended 50 pixels outward from both ends of the line segment. This is to prevent the intersection coordinates of the 78 straight lines with the sides Line2 and Line3 of the minimum circumscribed rectangle from being within the contour of the daylily bud, and to prevent the subsequent defined line segments from not intersecting with the contour edge line because one side of the minimum circumscribed rectangle is tangent to the contour edge, resulting in local data loss, and finally causing problems in size measurement. The specific implementation is as follows:

[0132] S412, the minimum bounding rectangle is deflected to the left, so that the width (width), that is, the line segments Line1 and Line4, are extended 50 pixels from the end points of the line segments. The minimum bounding rectangle is deflected to the right, so that the height (height), that is, Line1 and Line4, are extended 50 pixels from the end points of the line segments. The above operation only changes the length of Line1 and Line4, while the length of Line2 and Line3 does not change. At this point, a new bounding rectangle is generated, and the effect is as follows: Figure 5 shown.

[0133] S413, record the pixel coordinate points generated by the intersection of 78 straight lines parallel to line segments Line1 and Line4 with line segments Line2 and Line3. Each straight line has two intersection coordinates with Line2 and Line3, with a total of 156 pixel coordinate points. Every two coordinate points constitute the two ends of a line segment, with a total of 78 line segments. Recorded as line segment (M, N):

[0134] (M,N)=[[(m1,n1),(m2,n2)],[(m3,n3),(m4,n4)],[(m5,n5),(m6,n6)],...,[(m155,n155),(m156,n156)]]; where [(m1,n1),(m2,n2)] is a line segment, (m1,n1) and (m2,n2) are two pixel coordinate points. To illustrate the operation process of this example, we will draw a diagram of the operation process, as shown below: Figure 6 The effect diagram of S414 in the example is as follows Figure 6 As shown in the left figure: the white line segment is the line segment in (M,N) [(m i ,n i ), (m i+1 ,n i+1 )], i = 2*n+1, n = (0,1,2...77). In the actual operation, the edges of the new circumscribed rectangle, Line1 and Line4, are also drawn, but the intersection of these two edges with the contour edge is not included in the calculation. The part of the 78 straight line intercepted by the long side of the new circumscribed rectangle is defined as a line segment. The coordinates of the two endpoints of the line segment are located on the long side of the new circumscribed rectangle. The pixel coordinates of the two endpoints of each line segment are represented by gray dots, as shown in Figure 6 as shown in the figure on the right.

[0135] S414. In this example, the intersection points of all line segments in (M, N) and the edge of the contour of the daylily bud are recorded as (E, F):

[0136] (E,F)=[[(e1,f1),(e2,f2)],[(e3,f3),(e4,f4)],[(e5,f5),(e6,f6)],...,[(e155,f155),(e156,f156)]]; where [(e1,f1),(e2,f2)] are the coordinates of the two intersection points of the first line segment [(m1,n1),(m2,n2)] of line segment (M,N) and the edge of the contour of the daylily bud. The effect of S414 in the example Figure 7 As shown: The pixel coordinates of the gray circle on the edge line of the flower bud outline are the intersection coordinates of the line segment (M, N) and the edge of the daylily flower bud outline. The intersections of Line1 and Line4 with the edge of the outline are not included in the calculation.

[0137] S415, calculate the coordinates of the two intersection points of each line segment in (M, N) and the edge of the contour of the daylily bud [(e i ,f i ),(e i+1 ,f i+1 )], i = 2*n+1, n = (0, 1, 2, ..., 77), the middle coordinates are marked as [P1, P2, P3, ..., P78], P is a list; the calculation method is as follows:

[0138]

[0139] in

[0140] S416, obtaining the coordinates of the highest point and the lowest point of the daylily bud contour, defining the smaller ordinate value of the two coordinate points as the highest point, which is located at the top of the front end of the bud, and vice versa, defining it as the lowest point, which is located at the tail end of the perianth tube.

[0141] S417, insert the coordinates of the highest point into the top of the list P, that is, the position with the list subscript 0, and insert the coordinates of the lowest point into the end of the list, that is, the position with the list subscript 79. So far, the list has 80 list elements, that is, list P = [P0, P1, P2, P3, ..., P78, P79]. To illustrate the operation process of this example, the 80 pixel coordinate points in the list P are drawn in the image, represented by gray dots, and the effect is shown in the figure below. Figure 8 shown.

[0142] S418, calculate the pixel length of the daylily bud, and replace the list elements in list P with p i =(x i ,y i ), i = (0, 1, 2, ..., 79) means that the pixel distance d between adjacent coordinate points is calculated starting from p0. j , and finally d jThe pixel length D of the daylily bud is obtained by performing cumulative sum operation, and the calculation formula is as follows:

[0143]

[0144] S419, calculating the pixel length D of the daylily bud according to the mapping relationship between the pixel size and the physical size in S1 to obtain the physical length of the daylily bud.

[0145] The accuracy of the daylily bud length calculated in this embodiment was verified, and the machine vision measurement value was compared with the average value of multiple manual measurements to display the results as shown in Table 1, with a total of 29 groups of data;

[0146] Table 1

[0147]

[0148] As shown in Table 1, the root mean square error (RMSE) is 0.4326 mm, and the machine vision measurement is highly correlated with the manual measurement (R 2 =0.9987). Fig. 9 As shown, machine vision measurements are highly correlated with manual measurements (R 2 =0.9987). The parameter accuracy obtained by this algorithm can meet the accuracy required for daylily breeding in terms of daylily bud phenotype measurement.

[0149] S501. According to step S415, the coordinates of the intersection of the line segment (M, N) and the edge of the flower bud are (E, F), (E, F) = [[(e1, f1), (e2, f2)], [(e3, f3), (e4, f4)], [(e5, f5), (e6, f6)], ..., [(e155, f155), (e156, f156)]]. Each line segment in the line segment (M, N) has two intersection coordinates with the edge of the flower bud. Calculate the pixel distance between the two coordinate points where each line segment in the line segment (M, N) intersects with the edge of the flower bud, and find the two intersection coordinates that make the pixel distance the largest [(e i ,f i ),(e i+1 ,f i+1 )] and line segment [(m i ,n i ),(m i+1 ,n i+1 )], and then find the previous line segment of the line segment [(m i-2 ,n i-2 ),(m i-1 ,n i-1 )] and the next line segment [(m i+2 ,n i+2 ),(m i+3,n i+3 )] and the coordinates of the intersection of these two line segments and the edge of the bud [(e i-2 ,f i-2 ),(e i-1 ,f i-1 )] and [(e i+2 ,f i+2 ),(e i+3 ,f i+3 )]. Using the double-sided difference method, [(e i ,f i ),(e i+1 ,f i+1 )] The approximate tangent line between the two pixel coordinate points on the edge line of the flower bud, the tangent slopes are S1 and S2. The calculation formula is as follows:

[0150] S502, according to the two tangent lines obtained in step S421, pass through the pixel coordinate point [(e i ,f i ),(e i+1 ,f i+1 )] The perpendicular line on the corresponding tangent line intersects with the other side of the bud edge line to generate two new coordinate points (e i ′,f i ′) and (e i+1 ′,f i+1 ′); the coordinates of the intersection of the two vertical lines and the edge of the bud are [(e i ,f i ),(e i ′,f i ′)] and [(e i+1 ,f i+1 ),(e i+1 ′,f i+1 ′)]; the distances D1 and D2 between the coordinates of the two intersection points of the vertical line and the edge of the flower bud are calculated using the Euclidean distance calculation formula, and then the two distance values ​​are averaged to obtain the maximum pixel width of the approximate flower bud.

[0151] S503 , calculating the maximum pixel width of the daylily bud according to the mapping relationship between the pixel size and the physical size in S1 to obtain the maximum physical width of the daylily bud.

[0152] S504, verifying the accuracy of the maximum width of the daylily bud calculated in this embodiment, selecting the machine vision measurement value and the average value of multiple manual measurements for comparison and displaying the results as shown in Table 2, with a total of 29 groups of data;

[0153] Table 2

[0154]

[0155] As shown in Table 2, the root mean square error (RMSE) is 0.195 mm. Machine vision measurement is highly correlated with manual measurement (R 2 =0.9381), such as Fig.10 As shown in the figure, the parameter accuracy obtained by the algorithm can meet the accuracy required for daylily breeding in terms of daylily bud phenotype measurement.

[0156] Using least squares method to fit multivariate linear function to predict fresh weight

[0157] S601. Normalize n groups of response variables y (fresh weight of flower buds) and independent variables x1 (pixel area), x2 (length of flower buds), and x3 (maximum width of flower buds) to ensure that each variable has a fair impact on the result and avoid deviations caused by different dimensions. The operation is as follows:

[0158]

[0159] Where Y, X1, X2, and X3 are the average values ​​of the corresponding data. The normalization operation is to divide each data by the average value of the corresponding vector. i ′, x1′ i , x2′ i , x3′ i They are the corresponding data after normalization.

[0160] The formula for the relationship between the response variable y and the independent variables x1, x2, x3 is as follows:

[0161] y=β0x1+β1x2+β2x3+μ

[0162] In the formula, β0x1+β1x2+β2x3 is the fitting model, μ is the error term, and β0, β1, and β2 are model parameters;

[0163] By minimizing the sum of squared errors to find the model parameters and the error term.

[0164] The β0, β1, and β2 determined by the multivariate linear function fitted by the least squares method are 0.576, 0.55468, and 0.096, respectively, and u is -0.2272. The correlation between the independent variables x1, x2, and x3 and the response variable y is 92.23%.

[0165] The residual plots of the respective variables are as follows Fig.11 shown.

[0166] The accuracy of the fresh weight of the daylily buds predicted in this embodiment was verified, and the predicted values ​​were compared with the manually measured values ​​and the results were displayed as shown in Table 3, with a total of 29 groups of data;

[0167] Table 3

[0168]

[0169] As shown in Table 3, the root mean square error (RMSE) is 0.0765 g. The predicted fresh weight is highly correlated with manual measurement (R 2 =0.9291), such as Fig.12 As shown in the figure, the parameter accuracy obtained by the linear function prediction model for fresh weight can meet the accuracy required for daylily breeding in terms of daylily bud phenotype measurement.

[0170] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for measuring daylily phenotype based on machine vision, characterized in that: The following steps are involved: S1. Adjust the acquired image device to obtain the mapping relationship between pixel size and physical size; S2, collecting image data of daylily buds; S3, preprocessing the image data into a binary image; then screening the contours that meet the conditions according to the pixel area size of the connected area to obtain the contour image of the flower bud; S4, according to the intersection points of the multiple line segments and the edge line of the flower bud, find the ridge coordinate points of the flower bud outline, obtain the highest point coordinate and the lowest point coordinate of the flower bud outline, and calculate the pixel length of the flower bud; S5. According to the intersection points of the multiple line segments and the edge line of the flower bud, find the feature points on the two edge lines that make the pixel distance between the intersection point coordinates the largest, and calculate the maximum pixel width of the flower bud based on the two feature points; S6, converting the physical distance into the length and maximum width of the flower bud according to the mapping relationship; S7. Based on the three features of flower bud pixel area, flower bud length and maximum width and the true value of flower bud fresh weight, the flower bud fresh weight is predicted by fitting the linear function according to the least squares method.

2. The method for measuring daylily phenotype based on machine vision according to claim 1, characterized in that: The step of adjusting the captured image device to obtain the mapping relationship between the pixel size and the physical size includes the following steps: S1.1: Connect the industrial camera to the computer and adjust the parameters through the camera control software in the computer; S1.2: Adjust the object distance and focal length, as well as the light source, so that the light distribution in the camera's field of view is uniform; S1.3: Calculate the width and height of the camera's field of view based on the object distance, focal length, horizontal field of view angle, and vertical field of view angle. The calculation formula for the width W of the camera's field of view is: Where d is the distance from the camera to the object, θ h is the horizontal field of view; The calculation formula of the camera field of view height H is: Where d is the distance from the camera to the object, θ v is the vertical field of view; S1.4: Calculate the mapping relationship between the pixel size and the physical size according to the resolution X of the camera width. The calculation formula of the mapping relationship is as follows: Among them, R is the mapping relationship, which represents the mapping relationship between the actual object size in the image and the corresponding pixel size in the image, and W is the width of the camera field of view.

3. The method for measuring daylily phenotype based on machine vision according to claim 1, characterized in that: The method of collecting the image data of the daylily buds comprises the following steps: S2.1: Place the flower bud on a black curtain, with the stem of the flower bud parallel to the y-axis of the pixel coordinate system and the head of the flower bud pointing to the x-axis of the pixel coordinate system; S2.2: Use an industrial camera to capture the original color image of the flower bud.

4. The method for measuring daylily phenotype based on machine vision according to claim 1, characterized in that: The preprocessing of the image data into a binary image comprises the following steps: S3.1: First, grayscale the collected image data, then use the maximum inter-class variance method to obtain the default threshold for binarization of the grayscale processed image, and use median filtering to remove speckle noise and salt and pepper noise; S3.2: The processed data is subjected to morphological processing of first closing operation and then opening operation to remove small black dots in the outline, and the outline of the flower buds is screened out according to the pixel area of ​​the connected area. The screening condition is: the pixel area of ​​the connected area is between 5000 and 25000, which is the flower bud outline area, otherwise it is discarded; the setting of the screening condition is based on the occupancy ratio of the flower bud pixels in the original image of the present invention.

5. The method for measuring daylily phenotype based on machine vision according to claim 4, characterized in that: The method of obtaining the ridge coordinate points of the flower bud outline according to the intersection points of the multiple line segments and the flower bud edge line, obtaining the highest point coordinates and the lowest point coordinates of the flower bud outline, and calculating the pixel length of the flower bud comprises the following steps: S4.1: According to the screening result of step S3.2, the four corner coordinates, deflection angle, width and height of the minimum circumscribed rectangle of the flower bud outline are obtained; then the four corner coordinates and the rectangular side of the minimum circumscribed rectangle are redefined according to the deflection angle of the minimum circumscribed rectangle, so that no matter what the deflection direction of the minimum circumscribed rectangle is, the position definition of the four corner coordinates and the rectangular side of the rectangle will not change due to the deflection direction; and the short side of the minimum circumscribed rectangle is extended by 50 pixels to both sides, and the long side is moved by 50 pixels to both sides according to the direction of the short side; S4.2: Let the intercepts of the two short sides of the minimum circumscribed rectangle be C0 and C1. Make an arithmetic progression from small to large within the intercept range. Considering the calculation time and the size measurement error, divide the intercept range into 80 equal parts. Store the data in the list L1, L1 = [C0, C1] i (i=1,2...78), C 79 ], list L1 stores the short side intercepts of the minimum circumscribed rectangle C0, C 79 and 78 intercept values ​​C generated by arithmetic progression i , do C i is the intercept, and the slope is equal to the 78 line segments of the short side of the minimum circumscribed rectangle, and the two ends of the 78 line segments are respectively located on the long sides of the minimum circumscribed rectangle; S4.3: According to the processing result of step S4.2, each line segment has two intersection coordinates with the edge lines on both sides of the flower bud, and the intersections of 78 line segments with the edge lines of the flower bud contour have 78 coordinate pairs. The middle coordinate point of the two coordinates in each coordinate pair is calculated; the coordinates of the 78 middle points are obtained and stored in the list L2; S4.4: Get the highest and lowest coordinate points of the flower bud outline. The method is: traverse all the pixel coordinate points of the flower bud outline, find the two coordinate points with the maximum y and the minimum y, the coordinate point with the maximum y is the lowest coordinate point, and the coordinate point with the minimum y is the highest coordinate point; then insert the coordinate of the highest point into the head of list L2, and insert the coordinate of the lowest point into the tail of list L2, and obtain a total of 80 intermediate coordinate points that constitute the ridge features of the flower bud outline, and then calculate the distances between adjacent coordinate points in L2 in turn and accumulate them to obtain the pixel length of the flower bud.

6. The method for measuring daylily phenotype based on machine vision according to claim 1, characterized in that: The method of finding the feature points on the two edge lines that make the pixel distance between the intersection coordinates of the multiple line segments and the edge line of the flower bud the largest according to the intersection points of the multiple line segments and the edge line of the flower bud, and calculating the maximum pixel width of the flower bud according to the two feature points comprises the following steps: S5.1: Calculate the pixel distances between the coordinates of the intersections of the line segment and the edge of the flower bud contour in sequence using the Euclidean distance calculation formula, and find the two coordinate points (x1, y1) and (x2, y2) that maximize the pixel distance; S5.2: Calculate the tangents of the two coordinate points on the edge line according to the bilateral difference method, draw a perpendicular line through the two coordinate points and the tangents, and generate two other intersection coordinates (x1`, y1`) and (x2`, y2`) of the perpendicular line and the contour edge line. Use the Euclidean distance calculation formula to calculate the distance D1 between (x1, y1) and (x1`, y1`), and the distance D2 between (x2, y2) and (x2`, y2`); S5.3: Finally, the two pixel distances D1 and D2 are averaged to obtain an approximate estimate of the maximum width of the flower bud.

7. The method for measuring daylily phenotype based on machine vision according to claim 1, characterized in that: The step of converting the mapping relationship into physical distance to obtain the length and maximum width of the flower bud comprises the following steps: S6.1: According to the mapping relationship between the pixel size and the physical size described in S1, the pixel length and the maximum width of the daylily bud are converted into the actual physical length in mm to complete the phenotypic measurement of the daylily bud. The calculation formula of the mapping relationship described in S1 is: Where X is the width resolution of the camera in pixels, W is the width scene range seen through the camera lens in mm, and R represents the mapping relationship between the actual object size in the image and the corresponding pixel size in the image.

8. The method for measuring daylily phenotype based on machine vision according to claim 1, characterized in that: The method of predicting the fresh weight of flower buds by fitting a linear function using the least square method based on the three features of the flower bud pixel area, the flower bud length and the maximum width and the true value of the fresh weight of the flower buds comprises the following steps: S7.1: Given n sets of data, each set of data contains three independent variables and one dependent variable. The independent variables are the pixel area a1 of the flower bud, the length a2 of the flower bud and the maximum width a3 of the flower bud, and the dependent variable is the fresh weight b of the flower bud; S7.2: Normalize the data of independent and dependent variables; S7.3: Based on the data normalization results, the least squares method is used to minimize the sum of squares of the residuals between the n response values ​​of b and the values ​​calculated by its fitting model, and the independent variable coefficients and constant terms are solved.

Images