Computer Vision-Based Whole-Plant Measurement Calibration Method, System, and Device

Abstract

The present invention discloses a method, system and device for plant full-view measurement and calibration based on computer vision. The method includes: obtaining relevant parameters of a calibration board and the positions of identifiers; obtaining a side calibration image collected by a first image acquisition device of the side of the calibration board; obtaining a top calibration image collected by a second image acquisition device of the top of the calibration board at a plurality of calibration positions; respectively preprocessing the side calibration image and the top calibration image to obtain the original positions of the identifiers, and determining the positions of the identifiers in the standard state in combination with the actual physical distance ratio relationship of the identifiers, so as to obtain a first transformation matrix, a first scale relationship, a second transformation matrix and a second scale relationship; performing phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain the actual measurement information of the object to be measured. The present invention can realize two-dimensional full-view measurement of an object, and is applicable to the rapid, accurate and efficient measurement of the two-dimensional phenotypes of objects in medium and small application scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method, system and device for measuring and calibrating the whole plant based on computer vision. Background Art

[0002] Currently, in the application of object phenotype measurement technology, it mainly includes object height measurement, object organ phenotype measurement, object canopy coverage measurement, etc. The applications have the following characteristics: when measuring height, the camera is far from the object, the object depth is small, and the top canopy of the object is relatively flat and dense; when measuring organ phenotypes, it is required to cut and lay the organs flat; the canopy coverage can only calculate the proportional relationship and cannot achieve physical measurement.

[0003] In the application of object height measurement, currently there are: collecting maize images at close range through an RGB-D camera, and measuring the height information of a single maize plant through image analysis methods; measuring the height of cucumber seedlings in a greenhouse based on the comprehensive method of an RGB-D device imaging device and a calibration object reference, and through the setting of a reference object and point cloud analysis, the measured measurement error is 7.6% on average; a plant height measurement system based on laser vision, which projects a laser line emitted by a laser onto the plant body for height measurement. In the application of object top surface phenotype measurement, there are few cases. Currently, plant leaf images are collected through a camera with a certain resolution, and the number of pixels of the measured leaf of the standard reference object is obtained, so as to perform equivalent substitution to obtain the leaf area; the main existing scheme for measuring the object canopy coverage is based on a semi-automatic measurement method, which requires manual customization of the analysis area, measures and statistically analyzes the object area, and finally obtains the canopy coverage information, with strong subjectivity and low efficiency. Summary of the Invention

[0004] A method for measuring and calibrating the whole plant based on computer vision includes the following steps:

[0005] Obtain the relevant parameters of the calibration board and the positions of the identifiers, where the calibration board is rectangular and at least four identifiers are preset, and the central positions of the identifiers are respectively located at the four corners of the calibration board and form a rectangle, and the relevant parameters include the physical information of the calibration board;

[0006] Obtain a side calibration image collected by a first image acquisition device of the side of the calibration board, where the first image acquisition device is located on the side of the calibration board and at the center of the calibration board;

[0007] Obtain a top calibration image collected by a second image acquisition device of the top surface of the calibration board at several calibration positions, where the calibration board is arranged at several preset heights, and the second image acquisition device is collected above the calibration board and at the center of the calibration board;

[0008] Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier positions, determine the identifier positions under the standard state in combination with the actual physical distance ratio relationship of the identifiers, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix, and the second scale relationship;

[0009] Construct the current second scale relationship through several heights of the calibration plate and each second scale relationship;

[0010] Perform phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain the actual measurement information of the object to be measured.

[0011] As an implementable manner, the preprocessing of the side calibration image and the top calibration image respectively to obtain the original identifier positions includes the following steps:

[0012] Perform grayscale processing on the side calibration image and the top calibration image respectively to obtain the side calibration grayscale image and the top calibration grayscale image;

[0013] Based on the side calibration grayscale image and the top calibration grayscale image, obtain the average grayscale value of the side image and the average grayscale value of the top image;

[0014] Perform threshold segmentation on the side grayscale image through the average grayscale value of the side image to obtain the side binary image; perform threshold segmentation on the top grayscale image through the average grayscale value of the top image to obtain the top binary image;

[0015] Perform feature screening of the identifier on the side binary image and the top binary image to obtain the identifier region, and obtain the circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier;

[0016] Based on the area of the identifier region and the area of the circumscribed circle of the region, obtain the regional circularity. Based on the regional circularity and the area of the identifier region, analyze the identifier region obtained by the identifier feature screening to obtain the original identifier positions;

[0017] The regional circularity is expressed as follows:

[0018]

[0019] Among them, represents the regional circularity, represents the area of the identifier region, represents the circumscribed circle of the region.

[0020] As an implementable manner, the physical information of the calibration plate includes the height of the calibration plate and the width of the calibration plate, and the actual physical distance ratio relationship of the identifiers is the ratio relationship between the height of the calibration plate and the width of the calibration plate.

[0021] As an implementable manner, the first transformation matrix, the first scale relationship, the second transformation matrix, and the second scale relationship are obtained through the following steps:

[0022] Based on the original identifier positions and the identifier positions in the standard state, the pixel widths of the centers of two identifiers in the same row in the upright state and the pixel heights of the centers of two identifiers in the same column in the upright state are obtained respectively, which are expressed as follows:

[0023]

[0024]

[0025] Through curve fitting based on the position relationship between the original identifier positions and the identifier positions in the standard state, the original standard fitting relationship is obtained, which is expressed as follows:

[0026]

[0027] Through the original identifier positions, the identifier positions in the standard state, and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows:

[0028]

[0029]

[0030] Among them, represents the pixel width of the center of the identifier after being upright, represents the distance between standard identifiers in the same row, represents the distance between the upper right coordinate and the lower right coordinate in the standard identifier, represents the pixel height of the center of the identifier after being upright, represents the set of identifier position points in the side standard state or the set of identifier position points in the top surface standard state, represents the set of original identifier position points on the side or the set of original identifier position points on the top surface, represents the first transformation matrix or the second transformation matrix, represents the first scale relationship or the second scale relationship, represents the width of the calibration plate, represents the height of the calibration plate.

[0031] As an implementable manner, constructing the current second scale relationship through several heights of the calibration plate and each second scale relationship includes the following steps:

[0032] Fitting is performed based on the height of the calibration plate and the second scale relationship to obtain a fitting relationship, which is expressed as follows:

[0033]

[0034] Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows:

[0035]

[0036] Among them, represents the second scale relationship, represents the height of the calibration plate, represents the number of calibration plates, 、 respectively represent parameters.

[0037] As an implementable manner, before obtaining the actual measurement information of the object to be measured, the following steps are further included:

[0038] According to the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, the current second scale relationship of the object is obtained, which is expressed as follows:

[0039]

[0040] Among them, represents the side scale, represents the current second scale relationship of the object, represents the actual width of the object, represents a parameter.

[0041] As an implementable manner, the actual measurement information of the object to be measured is obtained through the following steps:

[0042] The side calibration image of the object to be measured is processed based on the first transformation matrix to obtain the side phenotype information of the object. The side phenotype information of the object at least includes the measured height of the object. Based on the side scale, the measured height of the object is converted to obtain the actual height of the object, which is expressed as follows:

[0043]

[0044] Based on the current second scale relationship of the object, scale transformation is performed on the top surface phenotype information of the object to obtain the actual top surface phenotype information of the object. Among them, the actual top surface phenotype information of the object at least includes the actual width, the actual spread, and the actual area index of the object, which is expressed as follows:

[0045]

[0046]

[0047]

[0048] Among them, represents the actual height of the object, represents the measured height of the object, represents the side scale, represents the current second scale relationship of the object, represents the measured width of the object, represents the actual width of the object, represents the measured spread of the object, represents the actual spread of the object, represents the measured area index of the object, represents the actual area index of the object.

[0049] A computer vision-based plant full-view measurement and calibration system, including an image acquisition module, a preprocessing and analysis module, a scale fitting module, and a phenotype information calculation module;

[0050] The image acquisition module obtains the relevant parameters of the calibration board and the positions of the identifiers. Among them, the calibration board is rectangular and has at least four preset identifiers. The central positions of the identifiers are located at the four corners of the calibration board respectively and form a rectangle. The relevant parameters include the physical information of the calibration board; obtain the side calibration image collected by the first image acquisition device for the side of the calibration board. Among them, the first image acquisition device is located on the side of the calibration board and at the center of the calibration board; obtain the top calibration image collected by the second image acquisition device for the top surface of the calibration board at several calibration positions. Among them, the calibration board is arranged at several preset heights, and the second image acquisition device is collected above the calibration board and at the center of the calibration board;

[0051] The preprocessing and analysis module preprocesses the side calibration image and the top calibration image respectively to obtain the original identifier positions, determines the identifier positions in the standard state in combination with the actual physical distance ratio relationship of the identifiers, and further obtains the first transformation matrix, the first scale relationship, the second transformation matrix, and the second scale relationship;

[0052] The scale fitting module constructs the current second scale relationship through several heights of the calibration board and each second scale relationship;

[0053] The phenotype information calculation module performs phenotype measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain the actual measurement information of the object to be measured.

[0054] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method is implemented:

[0055] Obtain the relevant parameters of the calibration board and the positions of the identifiers, where the calibration board is rectangular and at least four identifiers are preset, and the central positions of the identifiers are located at the four corners of the calibration board respectively and form a rectangle, and the relevant parameters include the physical information of the calibration board;

[0056] Obtain the side calibration image collected by the first image acquisition device for the side of the calibration board, where the first image acquisition device is located on the side of the calibration board and at the center of the calibration board;

[0057] Obtain the top calibration image collected by the second image acquisition device for the top of the calibration board at several calibration positions, where the calibration board is arranged at several preset heights, and the second image acquisition device is collected above the calibration board and at the center of the calibration board;

[0058] Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier positions, and determine the identifier positions in the standard state in combination with the actual physical distance proportional relationship of the identifiers, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship;

[0059] Construct the current second scale relationship through several heights of the calibration board and each second scale relationship;

[0060] Perform phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain the actual measurement information of the object to be measured.

[0061] A plant full-view measurement calibration device based on computer vision, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the following method is implemented:

[0062] Obtain the relevant parameters of the calibration board and the positions of the identifiers, where the calibration board is rectangular and at least four identifiers are preset, and the central positions of the identifiers are located at the four corners of the calibration board respectively and form a rectangle, and the relevant parameters include the physical information of the calibration board;

[0063] Obtain the side calibration image collected by the first image acquisition device for the side of the calibration board, where the first image acquisition device is located on the side of the calibration board and at the center of the calibration board;

[0064] Obtain the top calibration image collected by the second image acquisition device for the top of the calibration board at several calibration positions, where the calibration board is arranged at several preset heights, and the second image acquisition device is collected above the calibration board and at the center of the calibration board;

[0065] Preprocess the side calibration image and the top calibration image respectively to obtain the positions of the original identifiers. Combine the actual physical distance ratio relationship of the identifiers to determine the positions of the identifiers under standard conditions, and then obtain the first transformation matrix, the first scale ratio relationship, the second transformation matrix, and the second scale ratio relationship;

[0066] Construct the current second scale ratio relationship through several heights of the calibration board and each second scale ratio relationship;

[0067] Perform phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale ratio relationship to obtain the actual measurement information of the object to be measured.

[0068] Due to the adoption of the above technical solutions, the present invention has significant technical effects:

[0069] The calibration method for two-dimensional full-view measurement provided by the present invention can flexibly adjust the scale and shape characteristics of the calibration object, and uses a color-based segmentation algorithm and a shape-based feature selection algorithm to extract the calibration object; realizes the positioning and scale calculation of the calibration object according to the physical characteristics of the calibration object; fits the relationship formula according to the height sequence of the calibration object and the corresponding scale sequence, and combines the actual height information of the object during the actual measurement process to obtain the scale corresponding to the top camera, adaptively calculates the scale during the top phenotypic analysis of objects of different heights, and is used for the two-dimensional full-view measurement of the object. The calibration scheme and scale calculation method of the present invention process and analyze the side and top images of the object, realizing the automation and simplification of the two-dimensional full-view measurement of the object, improving the generality, robustness, and operation convenience of the application and the method. Based on the designed calibration scheme, the scale transformation relationship can adapt to objects of different heights, and the function of being calibrated once and applicable for life can be realized without changing the system structure, providing a reference basis for two-dimensional full-view measurement analysis. The method of the present invention can be applied to the rapid, accurate, and efficient measurement of the two-dimensional phenotype of objects in medium and small application scenarios. Brief Description of the Drawings

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0071] Figure 1 is the overall flow schematic diagram of the method of the present invention;

[0072] Figure 2 is the overall structure schematic diagram of the system of the present invention;

[0073] Figure 3 is the structure schematic diagram of the two-dimensional phenotype platform of the present invention;

[0074] Figure 4 is a calibration board example diagram;

[0075] Figures 5 - 6 is an example diagram for collecting calibration images of the side camera and the top camera;

[0076] Figure 7 is an example diagram for collecting calibration board images;

[0077] Explanation of the reference numerals in the accompanying drawings: 101, top RGB camera; 102, side RGB camera; 103, bottom platform of the box body; 104, object; 201, computer; 202, cloud platform; 100, image acquisition module; 200, preprocessing and analysis module; 300, scale fitting module; 400, phenotype information calculation module. Specific implementation manners

[0078] The following further describes the present invention in detail with reference to embodiments. The following embodiments are explanations of the present invention, and the present invention is not limited to the following embodiments.

[0079] Embodiment 1:

[0080] A plant full-view measurement calibration method based on computer vision, as Figure 1 shown, includes the following steps:

[0081] S100. Obtain the relevant parameters of the calibration board and the positions of the identifiers. Among them, the calibration board is rectangular and at least four identifiers are preset. The central positions of the identifiers are respectively located at the four corners of the calibration board and form a rectangle. The relevant parameters include the physical information of the calibration board;

[0082] S200. Obtain the side calibration image collected by the first image acquisition device for the side of the calibration board. Among them, the first image acquisition device is located on the side of the calibration board and at the center of the calibration board;

[0083] S300. Obtain the top calibration image collected by the second image acquisition device for the top of the calibration board at several calibration positions. Among them, the calibration board is arranged at several preset heights, and the second image acquisition device is collected above the calibration board and at the center of the calibration board;

[0084] S400. Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier positions. Combine the actual physical distance ratio relationship of the identifiers to determine the identifier positions in the standard state, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix, and the second scale relationship;

[0085] S500. Construct the current second scale relationship through several heights of the calibration board and each second scale relationship;

[0086] S600 performs phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale relationship, and obtains the actual measurement information of the object to be measured.

[0087] In one embodiment, the schematic diagram of the calibration board is shown in FIG. 7. The size of the calibration board should be larger than that of the measured object or the object to be measured to ensure the accuracy of scale conversion. The color contrast between the calibration board and the background color should be high to facilitate segmentation. In this embodiment, a rectangular calibration board with 4 dark circular identifiers and a bright background is set. The 4 identifiers form a rectangle. The physical diameter of the circular identifier is R, the physical distance of the center positions of two identifiers in the same row in the width direction is Width, and the physical distance of the center positions of two identifiers in the same column in the height direction is Height. Assume the center coordinates of the identifiers on the calibration board are C1, C2, C3, C4, where C1 is the center coordinate of the upper left identifier, C2 is the center coordinate of the upper right identifier, C3 is the center coordinate of the lower left identifier, and C4 is the center coordinate of the lower right identifier. According to this distribution, the center coordinates of the identifiers in the initial standard state are fitted, that is, a rectangle with C1 and C4 as the diagonal vertices is fitted, and thus the upper right center coordinate C2' and the lower left center coordinate C3' in the standard state are obtained. According to [C1, C2, C3, C4] and [C1, C2', C3', C4], the transformation matrix transformMatrix is obtained, and the pixel distances of each center are pixelWidth and pixelHeight respectively. Thus, an affine transformation is performed on the original image src to obtain the corrected result image dest. The initial scale is scaleSrc = (Width / pixelWidth + Height / pixelHeight) / 2.0, and the unit is mm / pixel.

[0088] In addition, during the top surface calibration process, multiple calibration positions are set and the scale fitting relationship is based on the position information. The scale of the top surface camera is calculated according to the actual object height, reducing the error of the top surface camera measurement caused by the uncertainty of the object height. In this embodiment, 3 calibration positions are set, which are 100 mm, 200 mm, and 300 mm away from the bottom surface respectively, as Figure 6As shown, the corresponding transformation matrices and scales verticalTransformMatrix1, verticalTransformMatrix2, verticalTransformMatrix1, verticalScaleSrc1, verticalScaleSrc2, verticalScaleSrc3 are calculated according to the calibration algorithm. Based on the physical heights of the calibration objects heights = [100, 200, 300] as independent variables and the scales verticalScaleSrcs = [verticalScaleSrc1, verticalScaleSrc2, verticalScaleSrc3] as dependent variables, a linear equation of one variable is obtained. Subsequently, the actual height of the object measured by the side camera can be used as the independent variable to calculate the corresponding scale relationship of the top-view camera at the current object height. The overall solution has low cost, is easy to verify, and is easy to expand. The size of the calibration object and the selection of the calibration position can be adjusted according to actual needs.

[0089] In one embodiment, the acquisition and analysis process implemented depending on the Figure 3 device specifically includes a top RGB camera 101; a side RGB camera 102; a platform at the bottom of the box 103; an object 104; a computer 201; and a cloud platform 202. The schematic diagram of the calibration board used is as Figure 7 shown. For the actual measurement of the object to be measured, first, side calibration images and a sequence of top calibration images are collected respectively, and the corresponding transformation matrix and initial scale are calculated. According to the height sequence of the calibration board and the fitting relationship of the corresponding scale during the calibration of the top camera; collect the side and top images of the object. First, correct the side image and analyze it to obtain the side phenotypic parameters containing the actual height information. According to the relationship between the object height and the scale of the top camera, the scale of the top camera of the current object is obtained. Correct and analyze the object image of the top camera and calculate the top phenotypic parameters of the object in combination with the obtained scale. Therefore, the calibration object can be accurately located and the scale can be calculated to realize the full-view measurement of the side and top of the object.

[0090] In one embodiment, the preprocessing of the side calibration image and the top calibration image respectively to obtain the position of the original identifier includes the following steps:

[0091] Perform gray processing on the side calibration image and the top calibration image respectively to obtain a side calibration gray image and a top calibration gray image;

[0092] Based on the side calibration gray image and the top calibration gray image, obtain the average gray value of the side image and the average gray value of the top image;

[0093] Threshold segmentation is performed on the side grayscale image according to the average grayscale value of the side image to obtain a side binary image; threshold segmentation is performed on the top grayscale image according to the average grayscale value of the top image to obtain a top binary image;

[0094] Performing feature screening of the identifier on the side binary image and the top binary image to obtain an identifier region, and obtaining a circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier;

[0095] Based on the area of ​​the identifier region and the area of ​​the circumscribed circle of the region, the circularity of the region is obtained; based on the circularity of the region and the area of ​​the identifier region, the identifier region obtained by the identifier feature screening is analyzed to obtain the original identifier position;

[0096] The circularity of the region is expressed as follows:

[0097]

[0098] in, Indicates the circularity of the region, Represents the area of ​​the identifier region, Represents the circumcircle of a region.

[0099] In one embodiment, the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship are obtained by the following steps:

[0100] Based on the original identifier position and the identifier position in the standard state, the pixel width of the center of two identifiers in the same row in the normal state and the pixel height of the center of two identifiers in the same column in the normal state are obtained, which are expressed as follows:

[0101]

[0102]

[0103] The original standard fitting relationship is obtained by fitting the positional relationship between the original identifier position and the identifier position under the standard state, which is expressed as follows:

[0104]

[0105] Through the original identifier position, the identifier position under the standard state and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows:

[0106]

[0107]

[0108] Among them, represents the pixel width of the center of the identifier after rotation, represents the distance between standard identifiers in the same row, represents the distance between the upper right coordinate and the lower right coordinate in the standard identifier, represents the pixel height of the center of the identifier after rotation, represents the set of identifier position points in the standard state on the side or the set of identifier position points in the standard state on the top surface, represents the set of original identifier position points on the side or the set of original identifier position points on the top surface, represents the first transformation matrix or the second transformation matrix, represents the first scale relationship or the second scale relationship, represents the width of the calibration plate, represents the height of the calibration plate.

[0109] This process can be understood as the process of calibrating the side camera and the top camera. As Figure 5 shown, this is the placement method of the calibration plate in the calibration process of the side camera. The plane of the calibration plate faces the side camera directly, and the calibration plate is located at the center position where the object to be measured is placed, and then the side calibration image image1 is acquired. Since the larger the size of the calibration plate is compared with the actual object size, the higher the phenotypic measurement accuracy is, provided that the size of the calibration plate is within the effective imaging size range of the camera.

[0110] Therefore, it is necessary to calculate the calibration parameters as follows:

[0111] The diameter of the identifier on the calibration plate is R, and the width, height and physical distance of the distance relationship between the centers are Width and Height respectively, with the unit of mm;

[0112] The algorithm process is as follows:

[0113] Input: side calibration image image1, physical information of the calibration plate Width, Height;

[0114] Algorithm flow:

[0115] Step 1: Perform image processing on the calibration image to obtain the positions C1, C2, C3, C4 of the centers of each identifier. These positions of the centers of the identifiers (centers of the circles) correspond to the upper left corner, upper right corner, lower left corner, and lower right corner positions of the calibration plate respectively. Then the identifier detection algorithm on the calibration plate is as follows:

[0116] The calibration plate identifier has a relatively high contrast. Therefore, first, the grayscale conversion of the side calibration image image1 is performed to obtain image2; the average grayscale value aveGray of image2 is calculated; image2 is thresholded with aveGray / 2.0 as the threshold to obtain a binary image image3 that only contains the area less than aveGray / 2.0; feature screening based on circularity is performed on image3. Assuming the actual area of a single region is area, the minimum circumscribed circle fitting is performed on this region to obtain the circumscribed circle area and the center position distribution as area' and C. Then the circularity of the region is circlePor = area / area'. The region is retained only when circlePor > thre1, area < thre2, and area > thre3. Finally, the identifier position coordinates C1, C2, C3, and C4 are obtained.

[0117] Step 2: If the actual physical distance ratio of the center points of the identifier is Width / Height, as Figure 4 shown, for the rectangle with C1 and C4 as the diagonal vertices, the upper-right corner center coordinate C2' and the lower-left corner center coordinate C3' in the standard state are obtained, and the original identifier center position and the identifier center position in the standard state [C1, C2, C3, C4], [C1, C2', C3', C4] are obtained. The pixel distance between C1 and C2' is pixelDis1, and the pixel distance between C3' and C4 is pixelDis2.

[0118] Step 3: According to [C1, C2, C3, C4] and [C1, C2', C3', C4], the pixel distances between the center points of the calibration plate identifier after rotation are calculated as pixelWidth = max(pixelDis1, pixelDis2) and pixelHeight = min(pixelDis1, pixelDis2). According to the formula , where corresponds to the point set coordinates of C1, C2', C3', and C4, corresponds to the point set coordinates of C1, C2, C3, and C4, and the matrix is the first transformation matrix or the side transformation matrix. sideScale = (Width / pixelWidth + Height / pixelHeight) / 2.0 is the side camera scale relationship, that is, the first scale relationship, with the unit of mm / pixel.

[0119] Finally, what the algorithm outputs is: the side camera image correction matrix sideTransformMatrix and the scale sideScale.

[0120] This step also includes the process of calibrating the top camera, in order to be able to adaptively measure the top phenotypes of objects at different heights. The specific process is as follows:

[0121] As Figure 6 shown, Figure 6 For the placement method of the calibration board during the calibration process of the top camera, the plane of the calibration board faces the top camera directly, and the calibration board is located at the center position of the platform. In this embodiment, 3 calibration positions are set, and the distances between the calibration board and the bottom surface are 100mm, 200mm, and 300mm respectively. The obtained top calibration images corresponding to the heights are the top calibration image image4, the top calibration image image5, and the top calibration image image6 respectively;

[0122] According to the transformation matrix and scale calculation process obtained by the side calibration image processing method, the second transformation matrix and the second scale relationship corresponding to each height are obtained respectively, which are verticalTransformMatrix1, verticalTransformMatrix2, verticalTransformMatrix3, verticalScaleSrc1, verticalScaleSrc2, and verticalScaleSrc3.

[0123] In a specific embodiment, constructing the current second scale relationship through the heights of the calibration board and each second scale relationship includes the following steps:

[0124] Based on the height of the calibration board and the second scale relationship for fitting, the fitting relationship is obtained, which is expressed as follows:

[0125]

[0126] Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows:

[0127]

[0128] Among them, represents the second scale relationship, represents the height of the calibration board, represents the number of calibration boards, 、 respectively represent parameters.

[0129] That is to say, because the calibration board is at different heights, in order to obtain more accurate height information of the object to be measured or the measured object and other information related to the height information, a fitting method for the top camera scale relationship formula is also designed. Specifically:

[0130] Input: the height of the calibration board [heihgt1, height2, height3,...] and the scale matrix [verticalScaleSrc1, verticalScaleSrc2, verticalScaleSrc3,...];

[0131] According to the formula , obtain

[0132] This relationship can be understood as the current second scale relationship, which is the relationship between the second scale relationship and the height of the calibration board and the number of calibration boards.

[0133] If it is used to measure the information of the actual object, the following transformation can also be performed first during the measurement, or directly during the measurement process, that is, this step of operation can be performed before obtaining the actual measurement information of the object to be measured. Specifically:

[0134] Based on the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, obtain the current second scale relationship of the object, which is expressed as follows:

[0135]

[0136] Among them, represents the side scale, represents the current second scale relationship of the object, represents the actual width of the object, represents the parameter.

[0137] In one embodiment, the actual measurement information of the object to be measured is obtained through the following steps:

[0138] Process the side calibration image of the object to be measured based on the first transformation matrix to obtain the side phenotype information of the object. The side phenotype information of the object at least includes the measured height of the object. Based on the side scale, convert the measured height of the object to obtain the actual height of the object, which is expressed as follows:

[0139]

[0140] Perform scale transformation on the top surface phenotype information of the object based on the current second scale relationship of the object to obtain the actual top surface phenotype information of the object. Among them, the actual top surface phenotype information of the object at least includes the actual width, the actual spread, and the actual area index of the object, which is expressed as follows:

[0141]

[0142]

[0143]

[0144] Among them, represents the actual height of the object, represents the measured height of the object, represents the side scale, represents the current second scale relationship of the object, represents the measured width of the object, represents the actual width of the object, represents the measured spread of the object, represents the actual spread of the object, represents the measured area index of the object, represents the actual area index of the object.

[0145] Specifically: the first transformation matrix sideTransformMatrix (side camera transformation matrix), the first scale relationship or the second scale relationship sideScale, the second transformation matrices verticalTransformMatrix1, verticalTransformMatrix2, verticalTransformMatrix3 (top camera transformation matrix) and the current second scale relationship , perform phenotypic measurement on the object to be measured. Here, the object to be measured can be a crop, and of course it can be other objects. The specific process is as follows:

[0146] First, analyze the image collected by the side camera, and use the analyzed object height information for the selection of the top camera transformation matrix and the calculation of the actual scale;

[0147] The side camera and the top camera respectively collect object information to obtain the corresponding side calibration image image7 and the top calibration image8;

[0148] Perform image processing on the side calibration image image7 to obtain the phenotypic information of the object. According to the side camera scale sideScale, convert the phenotypic information to obtain the actual side phenotypic information including the object height. The image processing process includes correcting the side image. During the correction process, the transformation matrix sideTransformMatrix is used, and the measured actual height of the object is plantHeight;

[0149] According to the object height plantHeight, calculate the position information of the nearest calibration board during the calibration process of the top camera. The transformation matrix corresponding to this position is the matrix verticalTransformMatrix for correcting the image collected by the top camera during this phenotypic analysis process, and the corresponding scale ;

[0150] Perform image processing on the top surface calibration image image8 to obtain the object phenotype information, and convert the phenotype information according to the top surface camera scale verticalScale to obtain the actual top surface phenotype information of the object. The image processing process includes correcting the top surface image, and the transformation matrix verticalTransformMatrix is used during the correction process;

[0151] The parameters for object phenotype analysis include but are not limited to the following: object height plantHeight, object width plantWide, object spread plantStenter, object leaf area index plantArea. Among them, plantHeight is side information, and plantWide, plantStenter, and plantArea are top surface information. Convert based on the calibration scale to obtain the actual measurement information:

[0152]

[0153]

[0154]

[0155]

[0156] In this way, the actual measurement information of the object to be measured is finally obtained.

[0157] Example 2:

[0158] A computer vision-based plant full-view measurement and calibration system, as Figure 2 shown, includes an image acquisition module 100, a preprocessing and analysis module 200, a scale fitting module 300, and a phenotype information calculation module 400;

[0159] The image acquisition module 100 obtains the relevant parameters of the calibration board and the positions of the identifiers. Among them, the calibration board is rectangular and at least four identifiers are preset. The central positions of the identifiers are located at the four corners of the calibration board respectively and form a rectangle. The relevant parameters include the physical information of the calibration board; obtain the side calibration image collected by the first image acquisition device on the side of the calibration board. Among them, the first image acquisition device is located on the side of the calibration board and at the center of the calibration board; obtain the top surface calibration image collected by the second image acquisition device on the top surface of the calibration board at several calibration positions. Among them, the calibration board is arranged at several preset heights, and the second image acquisition device is collected above the calibration board and at the center of the calibration board;

[0160] The preprocessing and analysis module 200 preprocesses the side calibration image and the top calibration image respectively to obtain the positions of the original identifiers, determines the positions of the identifiers in the standard state in combination with the actual physical distance ratio relationship of the identifiers, and further obtains the first transformation matrix, the first scale relationship, the second transformation matrix, and the second scale relationship.

[0161] The scale fitting module 300 constructs the current second scale relationship based on several heights of the calibration plate and each second scale relationship.

[0162] The phenotypic information calculation module 400 performs phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain the actual measurement information of the object to be measured.

[0163] All changes and modifications made without departing from the spirit and scope of the present invention, and all equivalent technical solutions also fall within the scope of the present invention.

[0164] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same and similar parts among the embodiments, reference may be made to each other.

[0165] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present invention is described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks specified in the functions.

[0167] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 specified in one block or multiple blocks.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operational steps are performed on the computer or other programmable terminal device to produce a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 specified in one block or multiple blocks.

[0169] It should be noted that:

[0170] As used in the specification, "one embodiment" or "an embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "an embodiment" that appear throughout the specification do not necessarily all refer to the same embodiment.

[0171] In addition, it should be noted that for the specific embodiments described in this specification, the shapes, names, etc. of their components can be different. Any equivalent or simple changes made to the structures, features, and principles described according to the inventive concept of the present invention are included within the protection scope of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the specific embodiments described, or use similar means for substitution, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, and they should all fall within the protection scope of the present invention.

Claims

1. A computer vision-based plant panorama measurement and calibration method, characterized in that: The following steps are involved: Obtain relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and has at least four preset identifiers, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; Acquire a side calibration image acquired by a first image acquisition device from a side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; Acquire a top surface calibration image captured by a second image acquisition device of the top surface of the calibration plate at a plurality of calibration positions, wherein the calibration plate is arranged at a plurality of preset heights, and the second image acquisition device captures an image located above the calibration plate and at the center of the calibration plate; Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier position, determine the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; The side calibration image and the top calibration image are preprocessed respectively to obtain the original identifier position, including the following steps: Performing grayscale processing on the side calibration image and the top calibration image respectively to obtain a side calibration grayscale image and a top calibration grayscale image; Based on the side calibration grayscale image and the top calibration grayscale image, an average grayscale value of the side image and an average grayscale value of the top image are obtained; Threshold segmentation is performed on the side grayscale image according to the average grayscale value of the side image to obtain a side binary image; threshold segmentation is performed on the top grayscale image according to the average grayscale value of the top image to obtain a top binary image; Performing feature screening of the identifier on the side binary image and the top binary image to obtain an identifier region, and obtaining a circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier; Based on the area of ​​the identifier region and the area of ​​the circumscribed circle of the region, the circularity of the region is obtained, and based on the circularity of the region and the area of ​​the identifier region, the identifier region obtained by the identifier feature screening is analyzed to obtain the original identifier position; The circularity of the region is expressed as follows: in, Indicates the circularity of the region, Represents the area of ​​the identifier region, represents the circumcircle of a region; The first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship are obtained by the following steps: Based on the original identifier position and the identifier position in the standard state, the pixel width of the center of two identifiers in the same row in the normal state and the pixel height of the center of two identifiers in the same column in the normal state are obtained, which are expressed as follows: The original standard fitting relationship is obtained by fitting the positional relationship between the original identifier position and the identifier position under the standard state, which is expressed as follows: Through the original identifier position, the identifier position under the standard state and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows: in, Indicates the pixel width of the center of the identifier circle after normalization. Indicates the distance between standard identifiers in the same row. Represents the distance between the upper right corner coordinates and the lower right corner coordinates in the standard identifier. Indicates the pixel height of the center of the identifier circle after normalization. It represents the position point set of the identifier under the standard state of the side surface or the position point set of the identifier under the standard state of the top surface. represents the side original identifier position point set or the top original identifier position point set, represents the first transformation matrix or the second transformation matrix, Indicates the first scale relationship or the second scale relationship, Indicates the width of the calibration plate. Indicates the height of the calibration plate; Constructing the current second scale relationship by calibrating several heights of the plate and each second scale relationship; Based on the calibration plate height and the second scale relationship, the fitting relationship is obtained, which is expressed as follows: Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows: in, Indicates the second scale relationship, Indicates the height of the calibration plate, Indicates the number of calibration plates, , Respectively represent parameters; Before obtaining the actual measurement information of the object to be measured, the following steps are also included: According to the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, the current second scale relationship of the object is obtained, which is expressed as follows: in, Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the actual width of the object. Represents parameters; Based on the first transformation matrix and the current second scale relationship, a phenotype measurement is performed on the object to be measured to obtain actual measurement information of the object to be measured.

2. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The physical information of the calibration plate includes the height and width of the calibration plate, and the actual physical distance ratio of the identifier is the ratio of the height and width of the calibration plate.

3. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The actual measurement information of the object to be measured is obtained by the following steps: The side calibration image of the object to be measured is processed based on the first transformation matrix to obtain the side phenotype information of the object. The side phenotype information of the object at least includes the measured height of the object. The measured height of the object is converted based on the side scale to obtain the actual height of the object, which is expressed as follows: The object top surface phenotype information is scale-transformed based on the current second scale relationship of the object to obtain the object's actual top surface phenotype information, wherein the object's actual top surface phenotype information at least includes the object's actual width, the object's actual width, and the object's actual area index, which is expressed as follows: in, Indicates the actual height of the object. Indicates the measured height of an object. Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the measured width of the object, Indicates the actual width of the object. Indicates the object measurement spread, Indicates the actual extent of the object. Indicates the measured area index of an object, Indicates the actual area index of an object.

4. A plant panorama measurement and calibration system based on computer vision, characterized in that: It includes image acquisition module, preprocessing and analysis module, scale fitting module and phenotypic information calculation module; The image acquisition module acquires relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and at least four identifiers are preset, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; acquires a side calibration image acquired by a first image acquisition device from the side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; acquires a top surface calibration image acquired by a second image acquisition device from the top surface of the calibration plate at several calibration positions, wherein the calibration plate is arranged at several preset heights, and the second image acquisition device acquires images located above the calibration plate and at the center of the calibration plate; The preprocessing and analysis module preprocesses the side calibration image and the top calibration image respectively to obtain the original identifier position, determines the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtains the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; The side calibration image and the top calibration image are preprocessed respectively to obtain the original identifier position, including the following steps: Performing grayscale processing on the side calibration image and the top calibration image respectively to obtain a side calibration grayscale image and a top calibration grayscale image; Based on the side calibration grayscale image and the top calibration grayscale image, an average grayscale value of the side image and an average grayscale value of the top image are obtained; Threshold segmentation is performed on the side grayscale image according to the average grayscale value of the side image to obtain a side binary image; threshold segmentation is performed on the top grayscale image according to the average grayscale value of the top image to obtain a top binary image; Performing feature screening of the identifier on the side binary image and the top binary image to obtain an identifier region, and obtaining a circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier; Based on the area of ​​the identifier region and the area of ​​the circumscribed circle of the region, the circularity of the region is obtained, and based on the circularity of the region and the area of ​​the identifier region, the identifier region obtained by the identifier feature screening is analyzed to obtain the original identifier position; The circularity of the region is expressed as follows: in, Indicates the circularity of the region, Represents the area of ​​the identifier region, represents the circumcircle of a region; The first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship are obtained by the following steps: Based on the original identifier position and the identifier position in the standard state, the pixel width of the center of two identifiers in the same row in the normal state and the pixel height of the center of two identifiers in the same column in the normal state are obtained, which are expressed as follows: The original standard fitting relationship is obtained by fitting the positional relationship between the original identifier position and the identifier position under the standard state, which is expressed as follows: Through the original identifier position, the identifier position under the standard state and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows: in, Indicates the pixel width of the center of the identifier circle after normalization. Indicates the distance between standard identifiers in the same row. Represents the distance between the upper right corner coordinates and the lower right corner coordinates in the standard identifier. Indicates the pixel height of the center of the identifier circle after normalization. It represents the position point set of the identifier under the standard state of the side surface or the position point set of the identifier under the standard state of the top surface. represents the side original identifier position point set or the top original identifier position point set, represents the first transformation matrix or the second transformation matrix, Indicates the first scale relationship or the second scale relationship, Indicates the width of the calibration plate. Indicates the height of the calibration plate; The scale fitting module constructs the current second scale relationship by calibrating a plurality of heights of the plate and each second scale relationship; Based on the calibration plate height and the second scale relationship, the fitting relationship is obtained, which is expressed as follows: Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows: in, Indicates the second scale relationship, Indicates the height of the calibration plate, Indicates the number of calibration plates, , Respectively represent parameters; Before obtaining the actual measurement information of the object to be measured, the following steps are also included: According to the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, the current second scale relationship of the object is obtained, which is expressed as follows: in, Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the actual width of the object. Represents parameters; The phenotype information calculation module performs phenotype measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain actual measurement information of the object to be measured.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

6. A plant panorama measurement and calibration device based on computer vision, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

Images