Plant full view measurement calibration method, system and device based on computer vision

Abstract

The invention discloses a plant full view measurement and calibration method, system and device based on computer vision. The method comprises the following steps: acquiring related parameters of a calibration plate and the position of an identifier; acquiring a side surface calibration image acquired by a first image acquisition device on the side surface of the calibration plate; acquiring a top surface calibration image acquired by a second image acquisition device on the top surface of the calibration plate positioned by a plurality of marks; respectively preprocessing the side calibration image and the top calibration image to obtain an original identifier position, and determining the identifier position in a standard state in combination with an actual physical distance proportional relation of the identifier to obtain a first transformation matrix, a first proportional scale relation, a second transformation matrix and a second proportional scale relation; and performing phenotype measurement on the to-be-measured object based on the first transformation matrix and the current second proportional scale relation to obtain actual measurement information of the to-be-measured object. The method can realize two-dimensional full view measurement of the object, and is suitable for rapid, accurate and efficient measurement of the two-dimensional phenotype of the object in small and medium-sized application scenes.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a computer vision-based plant overall measurement calibration method, system and device. Background Art

[0002] At present, the object phenotype measurement technology applications mainly include object height measurement, object organ phenotype measurement, object canopy coverage measurement, etc. The applications have the following characteristics: when measuring height, the camera is far away from the object, the object depth is small, and the top canopy of the object is flat and dense; when measuring organ phenotype, the organ needs to be cut and flattened; canopy coverage can only calculate the proportional relationship, and physical measurement cannot be achieved.

[0003] In the application of object height measurement, there are currently: using RGB-D cameras to collect corn images at close range, and using image analysis methods to measure the height information of individual corn plants; using RGB-D equipment imaging devices and calibration reference comprehensive methods to measure the height of cucumber seedlings in greenhouses, and through reference object setting and point cloud analysis, the average measurement error is 7.6% in practice; the plant height measurement system of agricultural objects based on laser vision measures the height by projecting the laser line emitted by the laser onto the agricultural object. However, there are fewer cases in the application of object top surface phenotype measurement. At present, plant leaf images are collected through cameras with a certain resolution to obtain the number of pixels of the measured leaves of the standard reference object, and then the leaf area is obtained by equivalent substitution; the existing main solution for measuring the canopy coverage of objects is based on semi-automatic measurement methods, which requires manual customization of analysis areas, measurement and statistical analysis of object areas, and finally obtaining canopy coverage information, which is highly subjective and inefficient. Summary of the invention

[0004] A computer vision-based plant panorama measurement and calibration method comprises the following steps: Obtain relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and has at least four preset identifiers, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; Acquire a side calibration image acquired by a first image acquisition device from a side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; Acquire a top surface calibration image captured by a second image acquisition device of the top surface of the calibration plate at a plurality of calibration positions, wherein the calibration plate is arranged at a plurality of preset heights, and the second image acquisition device captures an image located above the calibration plate and at the center of the calibration plate; Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier position, determine the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; Constructing the current second scale relationship by calibrating several heights of the plate and each second scale relationship; Based on the first transformation matrix and the current second scale relationship, a phenotype measurement is performed on the object to be measured to obtain actual measurement information of the object to be measured.

[0005] As an implementable method, the side calibration image and the top calibration image are preprocessed respectively to obtain the original identifier position, including the following steps: Performing grayscale processing on the side calibration image and the top calibration image respectively to obtain a side calibration grayscale image and a top calibration grayscale image; Based on the side calibration grayscale image and the top calibration grayscale image, an average grayscale value of the side image and an average grayscale value of the top image are obtained; Threshold segmentation is performed on the side grayscale image according to the average grayscale value of the side image to obtain a side binary image; threshold segmentation is performed on the top grayscale image according to the average grayscale value of the top image to obtain a top binary image; Performing feature screening of the identifier on the side binary image and the top binary image to obtain an identifier region, and obtaining a circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier; Based on the area of ​​the identifier region and the area of ​​the circumscribed circle of the region, the circularity of the region is obtained; based on the circularity of the region and the area of ​​the identifier region, the identifier region obtained by the identifier feature screening is analyzed to obtain the original identifier position; The circularity of the region is expressed as follows:

[0006] in, Indicates the circularity of the region, Represents the area of ​​the identifier region, Represents the circumcircle of a region.

[0007] As an implementable embodiment, the physical information of the calibration plate includes the height and width of the calibration plate, and the actual physical distance ratio of the identifier is the ratio of the height and width of the calibration plate.

[0008] As an implementable embodiment, the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship are obtained by the following steps: Based on the original identifier position and the identifier position in the standard state, the pixel width of the center of two identifiers in the same row in the normalized state and the pixel height of the center of two identifiers in the same column in the normalized state are obtained, which are expressed as follows:

[0009]

[0010] The original standard fitting relationship is obtained by fitting the positional relationship between the original identifier position and the identifier position under the standard state, which is expressed as follows:

[0011] Through the original identifier position, the identifier position under the standard state and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows:

[0012]

[0013] in, Indicates the pixel width of the center of the identifier circle after normalization. Indicates the distance between standard identifiers in the same row. Represents the distance between the upper right corner coordinates and the lower right corner coordinates in the standard identifier. Indicates the pixel height of the center of the identifier circle after normalization. It represents the position point set of the identifier under the standard state of the side surface or the position point set of the identifier under the standard state of the top surface. represents the side original identifier position point set or the top original identifier position point set, represents the first transformation matrix or the second transformation matrix, Indicates the first scale relationship or the second scale relationship, Indicates the width of the calibration plate. Indicates the height of the calibration plate.

[0014] As an implementable method, the current second scale relationship is constructed by calibrating a plurality of heights of the plate and each second scale relationship, comprising the following steps: Based on the calibration plate height and the second scale relationship, the fitting relationship is obtained, which is expressed as follows:

[0015] Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows:

[0016] in, Indicates the second scale relationship, Indicates the height of the calibration plate, Indicates the number of calibration plates, , Represent parameters respectively.

[0017] As an implementable method, before obtaining the actual measurement information of the object to be measured, the following steps are also included: According to the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, the current second scale relationship of the object is obtained, which is expressed as follows:

[0018] in, Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the actual width of the object. Indicates a parameter.

[0019] As an implementable method, the actual measurement information of the object to be measured is obtained by the following steps: The side calibration image of the object to be measured is processed based on the first transformation matrix to obtain the side phenotype information of the object. The side phenotype information of the object at least includes the measured height of the object. The measured height of the object is converted based on the side scale to obtain the actual height of the object, which is expressed as follows:

[0020] The object top surface phenotype information is scale-transformed based on the current second scale relationship of the object to obtain the object's actual top surface phenotype information, wherein the object's actual top surface phenotype information at least includes the object's actual width, the object's actual width, and the object's actual area index, which is expressed as follows:

[0021]

[0022]

[0023] in, Indicates the actual height of the object. Indicates the measured height of an object. Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the measured width of the object, Indicates the actual width of the object. Indicates the object measurement spread, Indicates the actual width of the object. Indicates the measured area index of an object, Indicates the actual area index of an object.

[0024] A plant panorama measurement and calibration system based on computer vision, including an image acquisition module, a preprocessing and analysis module, a scale fitting module and a phenotypic information calculation module; The image acquisition module acquires relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and at least four identifiers are preset, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; acquires a side calibration image acquired by a first image acquisition device from the side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; acquires a top surface calibration image acquired by a second image acquisition device from the top surface of the calibration plate at several calibration positions, wherein the calibration plate is arranged at several preset heights, and the second image acquisition device acquires images located above the calibration plate and at the center of the calibration plate; The preprocessing and analysis module preprocesses the side calibration image and the top calibration image respectively to obtain the original identifier position, determines the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtains the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; The scale fitting module constructs the current second scale relationship by calibrating a plurality of heights of the plate and each second scale relationship; The phenotype information calculation module performs phenotype measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain actual measurement information of the object to be measured.

[0025] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method is implemented: Obtain relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and has at least four preset identifiers, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; Acquire a side calibration image acquired by a first image acquisition device from a side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; Acquire a top surface calibration image captured by a second image acquisition device of the top surface of the calibration plate at a plurality of calibration positions, wherein the calibration plate is arranged at a plurality of preset heights, and the second image acquisition device captures an image located above the calibration plate and at the center of the calibration plate; Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier position, determine the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; Constructing the current second scale relationship by calibrating several heights of the plate and each second scale relationship; Based on the first transformation matrix and the current second scale relationship, a phenotype measurement is performed on the object to be measured to obtain actual measurement information of the object to be measured.

[0026] A plant panorama measurement and calibration device based on computer vision comprises a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the following method is implemented: Obtain relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and has at least four preset identifiers, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; Acquire a side calibration image acquired by a first image acquisition device from a side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; Acquire a top surface calibration image captured by a second image acquisition device of the top surface of the calibration plate at a plurality of calibration positions, wherein the calibration plate is arranged at a plurality of preset heights, and the second image acquisition device captures an image located above the calibration plate and at the center of the calibration plate; Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier position, determine the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; Constructing the current second scale relationship by calibrating several heights of the plate and each second scale relationship; Based on the first transformation matrix and the current second scale relationship, a phenotype measurement is performed on the object to be measured to obtain actual measurement information of the object to be measured.

[0027] The present invention has significant technical effects due to the adoption of the above technical solution: The calibration method for two-dimensional full-view measurement provided by the present invention can flexibly adjust the scale and shape characteristics of the calibration object, and extract the calibration object by using a color-based segmentation algorithm and a shape-based feature selection algorithm; realize the positioning and scale calculation of the calibration object according to the physical characteristics of the calibration object; obtain the corresponding scale of the top camera according to the fitting relationship between the calibration object height sequence and the corresponding scale sequence, and combine the actual height information of the object in the actual measurement process, and adaptively calculate the scale in the process of top surface phenotype analysis of objects at different heights, which is used for the two-dimensional full-view measurement of the object. The calibration scheme and scale calculation method of the present invention process and analyze the side and top surface images of the object, realize the automation and simplification of the two-dimensional full-view measurement of the object, improve the versatility, robustness and convenience of operation of the application and method, and based on the designed calibration scheme, the scale transformation relationship can adapt to objects of different heights, and when the system structure remains unchanged, the function of one-time calibration and lifelong applicability can be realized, which provides a reference basis for the two-dimensional full-view measurement analysis. The method of the present invention can be applied to the rapid, accurate and efficient measurement of the two-dimensional phenotype of objects in small and medium-sized application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 It is a schematic diagram of the overall process of the method of the present invention; Figure 2 It is a schematic diagram of the overall structure of the system of the present invention; Figure 3 is a schematic diagram of the structure of the two-dimensional phenotyping platform of the present invention; Figure 4 This is an example picture of the calibration plate; Figure 5-Figure 6 This is an example of image acquisition for calibration of side cameras and top cameras; Figure 7 This is an example of calibration plate image acquisition; Explanation of the numbers in the accompanying drawings: 101, top RGB camera; 102, side RGB camera; 103, box bottom platform; 104, object; 201, computer; 202, cloud platform; 100, image acquisition module; 200, preprocessing and analysis module; 300, scale fitting module; 400, phenotypic information calculation module. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.

[0031] Embodiment 1: A computer vision-based plant panorama measurement calibration method, such as Figure 1 As shown, the following steps are included: S100, obtaining relevant parameters of the calibration plate and positions of identifiers, wherein the calibration plate is rectangular and has at least four preset identifiers, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; S200, acquiring a side calibration image acquired by a first image acquisition device from a side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; S300, obtaining a top surface calibration image captured by a second image acquisition device of the top surface of the calibration plate at a plurality of calibration positions, wherein the calibration plate is arranged at a plurality of preset heights, and the second image acquisition device captures an image located above the calibration plate and at the center of the calibration plate; S400, preprocessing the side calibration image and the top calibration image respectively to obtain the original identifier position, and determining the identifier position under the standard state in combination with the actual physical distance ratio relationship of the identifier, and then obtaining the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; S500, constructing a current second scale relationship by calibrating a plurality of heights of the plate and each second scale relationship; S600: Perform phenotypic measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain actual measurement information of the object to be measured.

[0032] In one embodiment, the schematic diagram of the calibration plate is shown in Figure 7. The size of the calibration plate must be larger than the size of the measured object or the object to be measured to ensure the accuracy of the scale conversion. The color of the calibration plate must have a high contrast with the background color to facilitate segmentation. In this embodiment, a rectangular calibration plate with four dark circular identifiers and a light background is set. The four identifiers form a rectangle. The physical diameter of the circular identifier is R, the wide physical distance of the center position relationship of two identifiers in the same row is Width, and the high physical distance of the center position relationship of two identifiers in the same column is Height. Assuming that the center coordinates of the identifiers on the calibration plate are C1, C2, C3, and C4, C1 is the center coordinate of the upper left corner identifier, C2 is the center coordinate of the upper right corner identifier, C3 is the center coordinate of the lower left corner identifier, and C4 is the center coordinate of the lower right corner identifier, the center coordinates of the identifiers under the initial standard state are fitted according to this distribution, that is, a rectangle with C1 and C4 as diagonal vertices is fitted, thereby obtaining the center coordinates of the upper right corner C2' and the center coordinates of the lower left corner C3' under the standard state. According to [C1, C2, C3, C4], [C1, C2', C3', C4] get the transformation matrix transformMatrix, the pixel distances of each circle center are pixelWidth and pixelHeight respectively, and thus perform affine transformation on the original image src to get the corrected result image dest. The initial scale is scaleSrc=(Width / pixelWidth+Height / pixelHeight) / 2.0, and the unit is mm / pixel.

[0033] In addition, during the top surface calibration process, multiple calibration positions are set and the scale of the corresponding top surface camera is calculated according to the relationship between the position information and the scale fitting formula according to the actual object height, so as to reduce the error of the top surface camera measurement caused by the uncertainty of the object height. In this embodiment, three calibration positions are set at 100 mm, 200 mm, and 300 mm from the bottom surface respectively. Figure 6 As shown, according to the calibration algorithm, the corresponding transformation matrix and scale are verticalTransformMatrix1, verticalTransformMatrix2, verticalTransformMatrix1, verticalScaleSrc1, verticalScaleSrc2, and verticalScaleSrc3. The physical height of the calibration object heights=[100,200,300] is taken as the independent variable and the scale verticalScaleSrcs=[verticalScaleSrc1,verticalScaleSrc2,verticalScaleSrc3] is taken as the dependent variable. A linear equation of one variable can be solved. Subsequently, the actual height of the object measured by the side camera can be used as the independent variable to calculate the scale relationship of the top view camera at the current object height. The overall solution has low cost, is easy to verify, and is easy to expand. The size of the calibration object and the selection of the calibration position can be adjusted according to actual needs.

[0034] In one embodiment, depending on the Figure 3 The collection and analysis process is realized by the device, which specifically includes the top RGB camera 101; the side RGB camera 102; the bottom platform 103; the object 104; the computer 201; and the cloud platform 202. The schematic diagram of the calibration board used is as follows Figure 7 As shown in the figure, for the actual measurement of the object to be measured, first collect the side calibration image and the top calibration image sequence respectively, and calculate the corresponding transformation matrix and initial scale, according to the calibration plate height sequence and the corresponding scale fitting relationship when calibrating the top camera; collect the side and top images of the object, first correct and analyze the side image to obtain the side phenotype parameters containing the actual height information, according to the relationship between the object height and the top camera scale to obtain the current object top camera scale, correct and analyze the top camera object image and calculate the object top phenotype parameters in combination with the required scale. Therefore, the calibration object can be accurately located and the scale can be calculated to achieve full-view measurement of the side and top surfaces of the object.

[0035] In one embodiment, the preprocessing of the side calibration image and the top calibration image to obtain the original identifier position comprises the following steps: Performing grayscale processing on the side calibration image and the top calibration image respectively to obtain a side calibration grayscale image and a top calibration grayscale image; Based on the side calibration grayscale image and the top calibration grayscale image, an average grayscale value of the side image and an average grayscale value of the top image are obtained; Threshold segmentation is performed on the side grayscale image according to the average grayscale value of the side image to obtain a side binary image; threshold segmentation is performed on the top grayscale image according to the average grayscale value of the top image to obtain a top binary image; Performing feature screening of the identifier on the side binary image and the top binary image to obtain an identifier region, and obtaining a circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier; Based on the area of ​​the identifier region and the area of ​​the circumscribed circle of the region, the circularity of the region is obtained; based on the circularity of the region and the area of ​​the identifier region, the identifier region obtained by the identifier feature screening is analyzed to obtain the original identifier position; The circularity of the region is expressed as follows:

[0036] in, Indicates the circularity of the region, Represents the area of ​​the identifier region, Represents the circumcircle of a region.

[0037] In one embodiment, the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship are obtained by the following steps: Based on the original identifier position and the identifier position in the standard state, the pixel width of the center of two identifiers in the same row in the normal state and the pixel height of the center of two identifiers in the same column in the normal state are obtained, which are expressed as follows:

[0038]

[0039] The original standard fitting relationship is obtained by fitting the positional relationship between the original identifier position and the identifier position under the standard state, which is expressed as follows:

[0040] Through the original identifier position, the identifier position under the standard state and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows:

[0041]

[0042] in, Indicates the pixel width of the center of the identifier circle after normalization. Indicates the distance between standard identifiers in the same row. Represents the distance between the upper right corner coordinates and the lower right corner coordinates in the standard identifier. Indicates the pixel height of the center of the identifier circle after normalization. It represents the position point set of the identifier under the standard state of the side surface or the position point set of the identifier under the standard state of the top surface. represents the side original identifier position point set or the top original identifier position point set, represents the first transformation matrix or the second transformation matrix, Indicates the first scale relationship or the second scale relationship, Indicates the width of the calibration plate. Indicates the height of the calibration plate.

[0043] This process can be understood as the process of calibrating the side camera and the top camera, such as Figure 5As shown in the figure, this is the way to place the calibration plate during the side camera calibration process. The calibration plate plane faces the side camera, and the calibration plate is located at the center of the object to be measured, and then the side calibration image image1 is acquired. Since the larger the calibration plate size is than the actual object size, the higher the phenotypic measurement accuracy, provided that the calibration plate size is within the effective imaging size range of the camera.

[0044] Therefore, the calibration parameter calculation principle is as follows: The identifier diameter of the calibration plate is R, and the width and height physical distances of the center distances of each circle are Width and Height respectively, in mm; The algorithm process is as follows: Input: side calibration image image1, physical information of the calibration plate Width, Height; Algorithm flow: Step 1: Process the calibration image to obtain the center points of the identifiers C1, C2, C3, and C4. The center points (centers) of these identifiers correspond to the upper left corner, upper right corner, lower left corner, and lower right corner of the calibration plate. The algorithm for detecting the identifiers on the calibration plate is as follows: The contrast of the calibration plate identifier is high, so firstly, the side calibration image image1 is grayed to obtain image2; the average gray value aveGray of image2 is calculated; image2 is threshold segmented with aveGray / 2.0 as the threshold to obtain a binary image image3 containing only the area less than aveGray / 2.0; image3 is feature screened based on circularity, assuming that the actual area of ​​a single area is area, and the minimum circumscribed circle is fitted to the area to obtain the circumscribed circle area and the center position distribution as area', C, then the area circularity is circlePor=area / area', if and only if circlePor>thre1, area<thre2、area> When thre3 is used, the area is retained, and finally the identifier position coordinates C1, C2, C3, and C4 are obtained; Step 2: If the actual physical distance ratio of the center point of the identifier is Width / Height, such as Figure 4 As shown, a rectangle with C1 and C4 as diagonal vertices is formed, thereby obtaining the center coordinates C2' of the upper right corner and C3' of the lower left corner under the standard state, and obtaining the center position of the original identifier and the center position of the identifier under the standard state [C1, C2, C3, C4] and [C1, C2', C3', C4]. The pixel distance between C1 and C2' is pixelDis1, and the pixel distance between C3' and C4 is pixelDis2. Step 3: According to [C1, C2, C3, C4], [C1, C2', C3', C4], the pixel distances of the center of the calibration plate identifier after normalization are calculated as pixelWidth=max(pixelDis1, pixelDis2) and pixelHeight=min(pixelDis1, pixelDis2). According to the formula ,in Corresponding to the coordinates of the C1, C2', C3', and C4 point sets, Corresponding to the coordinates of the C1, C2, C3, and C4 point sets, we get the matrix That is the first transformation matrix or the side transformation matrix, sideScale=(Width / pixelWidth+Height / pixelHeight) / 2.0 is the side camera scale relationship, that is, the first scale relationship, and the unit is mm / pixel.

[0045] The final output of the algorithm is: side camera image correction matrix sideTransformMatrix and scale sideScale.

[0046] This step also involves calibrating the top camera in order to be able to adaptively measure the top-view phenotype of objects at different heights. The specific process is as follows: like Figure 6 As shown, Figure 6 This is the placement of the calibration plate during the top surface camera calibration process. The plane of the calibration plate faces the top surface camera and is located at the center of the platform. In this embodiment, three calibration positions are set. The distances between the calibration plate and the bottom surface are 100 mm, 200 mm, and 300 mm, respectively. The top surface calibration images of the corresponding heights collected are top surface calibration image image4, top surface calibration image image5, and top surface calibration image image6, respectively. According to the transformation matrix and scale calculation process obtained by the side calibration image processing method, the second transformation matrix and the second scale relationship corresponding to each height are obtained, which are verticalTransformMatrix1, verticalTransformMatrix2, verticalTransformMatrix3, verticalScaleSrc1, verticalScaleSrc2, and verticalScaleSrc3 respectively.

[0047] In a specific embodiment, the construction of the current second scale relationship by calibrating a plurality of heights of the plate and each second scale relationship comprises the following steps: Based on the calibration plate height and the second scale relationship, the fitting relationship is obtained, which is expressed as follows:

[0048] Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows:

[0049] in, Indicates the second scale relationship, Indicates the height of the calibration plate, Indicates the number of calibration plates, , Represent parameters respectively.

[0050] That is to say, since the calibration plates are at different heights, in order to obtain more accurate height information of the object to be measured or the measured object and other information related to the height information, a top camera scale relationship fitting method is also designed, specifically: Input: the height of the calibration plate [heihgt1, height2, height3, ...] and the scale matrix [verticalScaleSrc1, verticalScaleSrc2, verticalScaleSrc3, ...]; According to the formula ,get

[0051] This relationship can be understood as the current second scale relationship, which is the relationship between the second scale relationship and the height of the calibration plate and the number of calibration plates.

[0052] If it is used to measure the information of the actual object, the following transformation can be performed before the measurement, or the transformation can be performed directly during the measurement process, that is, this step can be performed before obtaining the actual measurement information of the object to be measured, specifically: According to the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, the current second scale relationship of the object is obtained, which is expressed as follows:

[0053] in, Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the actual width of the object. Indicates a parameter.

[0054] In one embodiment, the actual measurement information of the object to be measured is obtained by the following steps: The side calibration image of the object to be measured is processed based on the first transformation matrix to obtain the side phenotype information of the object. The side phenotype information of the object at least includes the measured height of the object. The measured height of the object is converted based on the side scale to obtain the actual height of the object, which is expressed as follows:

[0055] The object top surface phenotype information is scale-transformed based on the current second scale relationship of the object to obtain the object's actual top surface phenotype information, wherein the object's actual top surface phenotype information at least includes the object's actual width, the object's actual width, and the object's actual area index, which is expressed as follows:

[0056]

[0057]

[0058] in, Indicates the actual height of the object. Indicates the measured height of an object. Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the measured width of the object, Indicates the actual width of the object. Indicates the object measurement spread, Indicates the actual width of the object. Indicates the measured area index of an object, Indicates the actual area index of an object.

[0059] Specifically: the first transformation matrix sideTransformMatrix (side camera transformation matrix), the first scale relationship or the second scale relationship sideScale, the second transformation matrix verticalTransformMatrix1, verticalTransformMatrix2, verticalTransformMatrix3 (top camera transformation matrix) and the current second scale relationship , perform phenotypic measurement on the object to be tested. Here, the object to be tested can be crops, but of course it can be other objects. The specific process is as follows: First, the images collected by the side camera are analyzed, and the analyzed object height information is used to select the top camera transformation matrix and calculate the actual scale; The side camera and the top camera respectively collect object information to obtain the corresponding side calibration image image7 and top calibration image8; Perform image processing on the side calibration image image7 to obtain object phenotypic information, and convert the phenotypic information according to the side camera scale sideScale to obtain actual side phenotypic information including the height of the object. The image processing process includes correcting the side image. The transformation matrix sideTransformMatrix is ​​used in the correction process to measure the actual height of the object as plantHeight. According to the object height plantHeight, calculate the position information of the nearest calibration plate in the top camera calibration process. The transformation matrix corresponding to this position is the matrix verticalTransformMatrix for image correction of the top camera acquisition in this phenotypic analysis process. ; Perform image processing on the top surface calibration image image8 to obtain the object phenotype information, and convert the phenotype information according to the top surface camera scale verticalScale to obtain the actual top surface phenotype information of the object. The image processing process includes correcting the top surface image, and the transformation matrix verticalTransformMatrix is ​​used in the correction process; The object phenotypic analysis parameters include but are not limited to the following: object height plantHeight, object width plantWide, object spread plantStenter, object leaf area index plantArea, where plantHeight is the side information, plantWide, plantStenter, plantArea are the top surface information, and the actual measurement information is obtained by conversion based on the calibration scale:

[0060]

[0061]

[0062]

[0063] In this way, the actual measurement information of the object to be measured is finally obtained.

[0064] Embodiment 2: A plant panorama measurement and calibration system based on computer vision, such as Figure 2 As shown, it includes an image acquisition module 100, a preprocessing and analysis module 200, a scale fitting module 300 and a phenotype information calculation module 400; The image acquisition module 100 acquires relevant parameters of the calibration plate and positions of identifiers, wherein the calibration plate is rectangular and at least four identifiers are preset, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; acquires a side calibration image acquired by a first image acquisition device from the side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; acquires a top surface calibration image acquired by a second image acquisition device from the top surface of the calibration plate at several calibration positions, wherein the calibration plate is arranged at several preset heights, and the second image acquisition device acquires images located above the calibration plate and at the center of the calibration plate; The preprocessing and analysis module 200 preprocesses the side calibration image and the top calibration image respectively to obtain the original identifier position, and determines the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtains the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; The scale fitting module 300 constructs the current second scale relationship by calibrating a plurality of heights of the plate and each second scale relationship; The phenotype information calculation module 400 performs phenotype measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain actual measurement information of the object to be measured.

[0065] Various changes and modifications can be made without departing from the spirit and scope of the present invention, and all equivalent technical solutions also belong to the scope of the present invention.

[0066] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0067] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0071] It should be noted that: The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0072] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A computer vision-based plant panorama measurement and calibration method, characterized in that: The following steps are involved: Obtain relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and has at least four preset identifiers, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; Acquire a side calibration image acquired by a first image acquisition device from a side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; Acquire a top surface calibration image captured by a second image acquisition device of the top surface of the calibration plate at a plurality of calibration positions, wherein the calibration plate is arranged at a plurality of preset heights, and the second image acquisition device captures an image located above the calibration plate and at the center of the calibration plate; Preprocess the side calibration image and the top calibration image respectively to obtain the original identifier position, determine the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtain the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; Constructing the current second scale relationship by calibrating several heights of the plate and each second scale relationship; Based on the first transformation matrix and the current second scale relationship, a phenotype measurement is performed on the object to be measured to obtain actual measurement information of the object to be measured.

2. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The side calibration image and the top calibration image are preprocessed respectively to obtain the original identifier position, including the following steps: Performing grayscale processing on the side calibration image and the top calibration image respectively to obtain a side calibration grayscale image and a top calibration grayscale image; Based on the side calibration grayscale image and the top calibration grayscale image, an average grayscale value of the side image and an average grayscale value of the top image are obtained; Threshold segmentation is performed on the side grayscale image according to the average grayscale value of the side image to obtain a side binary image; threshold segmentation is performed on the top grayscale image according to the average grayscale value of the top image to obtain a top binary image; Performing feature screening of the identifier on the side binary image and the top binary image to obtain an identifier region, and obtaining a circumscribed circle of the region by fitting the identifier region, wherein the identifier is a circular identifier; Based on the area of ​​the identifier region and the area of ​​the circumscribed circle of the region, the circularity of the region is obtained, and based on the circularity of the region and the area of ​​the identifier region, the identifier region obtained by the identifier feature screening is analyzed to obtain the original identifier position; The circularity of the region is expressed as follows: in, Indicates the circularity of the region, Represents the area of ​​the identifier region, Represents the circumcircle of a region.

3. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The physical information of the calibration plate includes the height and width of the calibration plate, and the actual physical distance ratio of the identifier is the ratio of the height and width of the calibration plate.

4. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship are obtained by the following steps: Based on the original identifier position and the identifier position in the standard state, the pixel width of the center of two identifiers in the same row in the normal state and the pixel height of the center of two identifiers in the same column in the normal state are obtained, which are expressed as follows: The original standard fitting relationship is obtained by fitting the positional relationship between the original identifier position and the identifier position under the standard state, which is expressed as follows: Through the original identifier position, the identifier position under the standard state and the original standard fitting relationship, the first transformation matrix or the second transformation matrix and the first scale relationship or the second scale relationship are obtained, which are expressed as follows: in, Indicates the pixel width of the center of the identifier circle after normalization. Indicates the distance between standard identifiers in the same row. Represents the distance between the upper right corner coordinates and the lower right corner coordinates in the standard identifier. Indicates the pixel height of the center of the identifier circle after normalization. It represents the position point set of the identifier under the standard state of the side surface or the position point set of the identifier under the standard state of the top surface. represents the side original identifier position point set or the top original identifier position point set, represents the first transformation matrix or the second transformation matrix, Indicates the first scale relationship or the second scale relationship, Indicates the width of the calibration plate. Indicates the height of the calibration plate.

5. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The method of constructing the current second scale relationship by calibrating a plurality of heights of the calibration plate and each second scale relationship comprises the following steps: Based on the calibration plate height and the second scale relationship, the fitting relationship is obtained, which is expressed as follows: Based on the fitting relationship, the current second scale relationship is obtained, which is expressed as follows: in, Indicates the second scale relationship, Indicates the height of the calibration plate, Indicates the number of calibration plates, , Represent parameters respectively.

6. The computer vision-based plant panorama measurement and calibration method according to claim 5, characterized in that: Before obtaining the actual measurement information of the object to be measured, the following steps are also included: According to the measured height of the object, combined with the second transformation matrix and the scale fitting matrix, the current second scale relationship of the object is obtained, which is expressed as follows: in, Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the actual width of the object. Indicates a parameter.

7. The computer vision-based plant panorama measurement and calibration method according to claim 1, characterized in that: The actual measurement information of the object to be measured is obtained by the following steps: The side calibration image of the object to be measured is processed based on the first transformation matrix to obtain the side phenotype information of the object. The side phenotype information of the object at least includes the measured height of the object. The measured height of the object is converted based on the side scale to obtain the actual height of the object, which is expressed as follows: The object top surface phenotype information is scale-transformed based on the current second scale relationship of the object to obtain the object's actual top surface phenotype information, wherein the object's actual top surface phenotype information at least includes the object's actual width, the object's actual width, and the object's actual area index, which is expressed as follows: in, Indicates the actual height of the object. Indicates the measured height of an object. Indicates the side scale, Indicates the current second scale relationship of the object. Indicates the measured width of the object, Indicates the actual width of the object. Indicates the object measurement spread, Indicates the actual extent of the object. Indicates the measured area index of an object, Indicates the actual area index of an object.

8. A plant panorama measurement and calibration system based on computer vision, characterized in that: It includes image acquisition module, preprocessing and analysis module, scale fitting module and phenotypic information calculation module; The image acquisition module acquires relevant parameters of the calibration plate and the positions of the identifiers, wherein the calibration plate is rectangular and at least four identifiers are preset, the center positions of the identifiers are respectively located at the four corners of the calibration plate to form a rectangle, and the relevant parameters include physical information of the calibration plate; acquires a side calibration image acquired by a first image acquisition device from the side of the calibration plate, wherein the first image acquisition device is located on the side of the calibration plate and at the center of the calibration plate; acquires a top surface calibration image acquired by a second image acquisition device from the top surface of the calibration plate at several calibration positions, wherein the calibration plate is arranged at several preset heights, and the second image acquisition device acquires images located above the calibration plate and at the center of the calibration plate; The preprocessing and analysis module preprocesses the side calibration image and the top calibration image respectively to obtain the original identifier position, determines the identifier position under the standard state in combination with the actual physical distance ratio of the identifier, and then obtains the first transformation matrix, the first scale relationship, the second transformation matrix and the second scale relationship; The scale fitting module constructs the current second scale relationship by calibrating a plurality of heights of the plate and each second scale relationship; The phenotype information calculation module performs phenotype measurement on the object to be measured based on the first transformation matrix and the current second scale relationship to obtain actual measurement information of the object to be measured.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A plant overall measurement and calibration device based on computer vision, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images