A rice plant row spacing measurement method based on optimization inverse perspective transformation

Abstract

The present application is directed to the problem of large manual selection error of feature points in the inverse perspective transformation method based on point-to-correspondence transformation, and proposes a rice plant row spacing measurement method based on optimal inverse perspective transformation, belonging to the technical field of rice plant row spacing measurement, comprising: S1, setting a calibration part; S2, establishing a target function of optimal inverse perspective transformation; collecting an original image containing the calibration part, determining and substituting the initial value, iteratively solving the target function to obtain the optimal inverse perspective transformation matrix; S3, collecting the rice plant image, preprocessing the image; extracting the rice plant contour, analyzing the center coordinates of the rice plant; substituting the inverse perspective transformation formula composed of the optimal inverse perspective transformation matrix to obtain the physical coordinates of the center point of the rice plant; S4, fitting the center point of each row of rice plants according to the least square method to obtain the center line of each row of rice plants, calculating the distance between adjacent center lines, and recording as the rice plant row spacing. The accuracy of the model parameters is improved, and the measurement accuracy of the rice plant row spacing is improved.

Description

Technical Field

[0001] This invention relates to the field of rice row spacing measurement technology, specifically to a method for measuring rice row spacing based on optimized inverse perspective transformation. Background Technology

[0002] With the development of high-speed computers and the continuous reduction in the cost of large-capacity storage devices, machine vision technology has made significant progress in recent years. Among them, monocular vision ranging technology has been widely used in robotics, intelligent driving, and unmanned equipment.

[0003] Measuring rice row spacing using monocular vision is an effective technical solution for monitoring rice planting quality. Monocular vision methods for measuring rice row spacing fall into two main categories: First, by calibrating the camera's intrinsic and extrinsic parameters to recover the scene's 3D spatial information, the row spacing is measured. This method has advantages such as good versatility; however, it is not suitable for measuring row spacing when the camera is moving. Second, inverse perspective transformation transforms the front view to a bird's-eye view, simultaneously achieving a transformation from pixel scale to physical scale, making it suitable for measuring row spacing in moving scenes. Inverse perspective transformation mainly includes three methods: inverse perspective transformation based on multi-view geometry, inverse perspective transformation based on simplified camera models, and inverse perspective transformation based on point-to-homography. Inverse perspective transformation based on multi-view geometry has advantages such as good modeling quality and high measurement accuracy, but it requires multiple 2D images for 3D information recovery, resulting in high computational cost and low efficiency. Inverse perspective transformation based on simplified camera models has advantages such as simple form and fast computation speed, but the camera's pitch and yaw angles have a significant impact on the inverse perspective transformation results. Compared to the previous two methods, the inverse perspective transformation method based on point-to-point homography does not require information such as camera intrinsic and extrinsic parameters or ranging planes, making it simple to implement and computationally efficient. However, the commonly used methods still have issues with accuracy that need improvement. This is attributed to errors in point-to-point information. For example, when manually selecting point-to-point information, the error between the selected point and the target point is large, resulting in low accuracy of the inverse perspective transformation. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies, specifically the problem that the manual selection of feature points in the inverse perspective transformation method based on point-to-homography transformation results in large errors, leading to a need to improve the accuracy of rice row spacing measurement. It provides a method for measuring rice row spacing based on optimized inverse perspective transformation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for measuring rice row spacing based on optimized inverse perspective transformation includes the following steps:

[0007] S1. Set the calibration components;

[0008] S2. Establish the objective function for optimizing the inverse perspective transformation; acquire the original image containing the calibration component, determine and substitute the initial values, iteratively solve the objective function, and obtain the optimal inverse perspective transformation matrix;

[0009] S3. Collect images of rice plants and preprocess them; extract the outline of the rice plants and analyze the center coordinates of the rice plants; substitute the inverse perspective transformation formula composed of the optimal inverse perspective transformation matrix to obtain the physical coordinates of the center point of the rice plants.

[0010] S4. Fit the center point of each row of rice plants according to the least squares method to obtain the center line of each row of rice plants, calculate the distance between adjacent center lines, and record it as the rice plant row spacing.

[0011] To optimize the above technical solution, the specific measures also include:

[0012] Further, in step S1, the calibration component includes n nested squares, the side lengths of the n squares being d1, d2, ..., d... n And at each of the four vertices of each square, there is a circle with radius r centered at that vertex, satisfying d1 > 2r and d i+1 ≥d i +4r; where d i and d i+1 Let represent the side lengths of the i-th and (i+1)-th squares, respectively, and 1 ≤ i ≤ n-1.

[0013] Further, in step S2, the inverse perspective transformation specifically includes:

[0014] To construct a new image, each pixel in the original image is mapped to a new location. The mathematical model is as follows:

[0015]

[0016] In the formula, (x, y) represents any point on the new image; (u, v) represents any point on the original image; s represents the scale parameter; M is the inverse perspective transformation matrix.

[0017] Furthermore, in step S2, the objective function for establishing the optimal inverse perspective transformation is specifically:

[0018] The objective function is established with the goal of minimizing the total fitting error of the 4n circles of the calibration component. The constraints are that each of the n squares has two pairs of parallel opposite sides, two pairs of perpendicular adjacent sides, and the absolute value of the difference in adjacent side lengths should be less than ε. ε is a small positive constant. The objective function is as follows:

[0019]

[0020] In the formula, f(M) is the sum of the fitting errors of 4n circles:

[0021]

[0022] In the formula, This indicates that on the original image, at square A... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Ath i The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, at square B... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Bth i The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, at the Cth position of the square... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Cth i The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, at square D... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Dth i The homogeneous coordinates of the vertex on the new image; Let i represent the scale parameter, where i = 1, 2, ..., n, j = 1, 2, ..., m, m ∈ N and m ≥ 50; n represents the number of squares in the calibration piece, and m represents the number of pixels on the circumference at the vertex of the square;

[0023] Equality constraints Where i = 1, 2, ..., n; the details are as follows:

[0024]

[0025] Equality constraints Where i = 1, 2, ..., n; the details are as follows:

[0026]

[0027] Equality constraints Where i = 1, 2, ..., n, as follows:

[0028]

[0029] Equality constraints Where i = 1, 2, ..., n; the details are as follows:

[0030]

[0031] Inequality constraints Where i = 1, 2, ..., n; the details are as follows:

[0032]

[0033] In the formula, Xx = [1 0 0], Yy = [0 1 0]; Indicates the Ath i The homogeneous coordinates of the vertex in the original image. Indicates the Bth i The homogeneous coordinates of the vertex in the original image. Indicates the Cth i The homogeneous coordinates of the vertex in the original image. Indicates the Dth i The homogeneous coordinates of the vertex in the original image; Indicates the scale parameter;

[0034] Based on Lagrange multiplier theory, the aforementioned constrained objective function can be rewritten as an unconstrained objective function, as follows:

[0035]

[0036] In the formula, λ1, λ2, λ3, λ4, and μ represent Lagrange multipliers.

[0037] Further, in step S2, the acquisition of the original image containing the calibration component, the determination and substitution of initial values, and the iterative solution of the objective function to obtain the optimal inverse perspective transformation matrix are specifically as follows:

[0038] S2.1 According to the requirements for measuring the row spacing of rice plants, set up calibration components in the scene and collect on-site images as the original images;

[0039] S2.2 Manually select four vertices of any square on the original image, and perform sub-pixel detection of the vertex within its Q×Q pixel neighborhood, with each vertex as the center, where Q∈N and Q≥5. Using the sub-pixel coordinates and physical coordinates of the four vertices, construct four sets of initial point pairs, and use the coordinates of the four sets of initial point pairs as initial values.

[0040] S2.3, The first inverse perspective transformation matrix is ​​calculated using the initial values ​​as M. (1) Transform the original image into the first new image;

[0041] S2.4 After k iterations, the inverse perspective transformation matrix is ​​M. (k) This yields the k-th new image of the original image;

[0042] S2.5. Process the original image and the k-th new image respectively to obtain the homogeneous coordinates required for calculating the objective function, and calculate the result R of the objective function. (k) ;

[0043] S2.6 Update the parameters of the objective function using gradient descent;

[0044] S2.7, Based on the inverse perspective transformation matrix M (k+1) This yields the (k+1)th new image of the original image;

[0045] S2.8. Process the original image and the (k+1)th new image respectively to obtain the homogeneous coordinates required for calculating the objective function, and calculate the result R of the objective function. (k+1) ;

[0046] S2.9, If R (k) and R (k+1) When the difference is less than a set value, the iteration stops, and the optimal inverse perspective transformation matrix M is obtained. * Otherwise, repeat steps S2.4 to S2.9.

[0047] Furthermore, step S2.6 specifically includes:

[0048] Update the parameters of the objective function using the following formula:

[0049]

[0050] In the formula, α is the learning rate when updating parameters. This indicates that for the objective function L(M) (k) Find the gradient.

[0051] Furthermore, step S3 specifically includes:

[0052] S3.1. According to the requirements for rice plant measurement, collect images of rice paddies in the scene as the images to be measured;

[0053] S3.2 Preprocess the image to be tested to separate the rice plants from the background;

[0054] S3.3. The preprocessed image is used for contour detection to extract the contour of each rice plant. Based on the position of the rice plant contour, the sub-pixel coordinates of the center point of each rice plant are extracted.

[0055] S3.4, Substitute the subpixel coordinates Using the inverse perspective transformation formula composed of the optimal inverse perspective transformation matrix Obtain the physical coordinates of the center point of the rice plant.

[0056] Furthermore, in step S3.2, the preprocessing includes converting the image to grayscale, filtering, and binarizing it.

[0057] The beneficial effects of this invention are:

[0058] This invention proposes a method for measuring rice row spacing based on optimized inverse perspective transformation, which optimizes the inverse perspective transformation effect and improves the accuracy of rice row spacing measurement using a monocular vision camera. The key to optimized inverse perspective transformation lies in optimizing the parameters of the inverse perspective transformation model by improving the fit between the calibration elements in the transformed image and the ideal contour. Before the optimization task begins, parameters are proactively assigned as initial values. Through iterative updates, the parameters that achieve the required fit are used as the optimized inverse perspective transformation parameters. The optimized inverse perspective transformation model is used to perform inverse perspective transformation on images containing rice plants, obtaining a bird's-eye view of the image, thus achieving high-precision measurement of rice row spacing. Compared to manually selecting point pairs for inverse perspective transformation, the optimized inverse perspective transformation method can optimize model parameters, improving measurement accuracy and robustness. Attached Figure Description

[0059] Figure 1 A schematic diagram of the calibration component for the design;

[0060] Figure 2 This is a schematic diagram of the label circle fitting error;

[0061] Figure 3 This is a flowchart of a method for measuring rice row spacing based on optimized inverse perspective transformation. Detailed Implementation

[0062] The invention will now be described in further detail with reference to the accompanying drawings.

[0063] To address the issue of large errors in manually selecting feature points, leading to insufficient accuracy in rice row spacing measurement, in inverse perspective transformation methods based on point-to-homography transformation, this invention designs a multi-layered nested calibration component. By minimizing the roundness error of the feature circle (containing feature points) and combining it with geometric constraints, an objective function for optimizing inverse perspective transformation parameters is established. A rice row spacing measurement process based on inverse perspective transformation is developed, thus proposing a method for measuring rice row spacing based on optimized inverse perspective transformation. Compared to inverse perspective transformation methods based on multi-view geometry, this method has higher computational efficiency. Compared to inverse perspective transformation based on simplified camera models, it eliminates the need to calibrate the camera's intrinsic and extrinsic parameters. Compared to existing inverse perspective transformation methods based on manual point selection, it can semi-automatically detect feature points and achieves higher measurement accuracy.

[0064] See Figures 1-3 In one embodiment, the present invention proposes a method for measuring the spacing between rice rows based on optimized inverse perspective transformation, specifically including the following steps:

[0065] 1. Make calibration parts: The main body of the calibration parts consists of two nested squares A1B1C1D1 and A2B2C2D2 with side lengths of 50cm and 80cm respectively. Each of the four vertices of each square has a circle with a radius of 5cm centered at that vertex.

[0066] 2. Image Acquisition: According to the requirements for measuring the row spacing of rice plants, a camera is fixed in the scene, and calibration components are set up. The camera is used to acquire two-dimensional images of the calibration components, which are the original images.

[0067] In the original image, four vertices of a square A1B1C1D1 are manually selected. Sub-pixel detection is performed within a 5×5 pixel neighborhood of each vertex. Based on the sub-pixel detection results, the pixel coordinates of the four vertices are obtained. Using these pixel coordinates and physical coordinates, four initial point pairs are constructed, with the coordinates of these four initial point pairs serving as initial values.

[0068] 3. Substitute the above initial values ​​into the mathematical model of inverse perspective transformation. The first inverse perspective transformation matrix M is obtained. (1) The original image is transformed into the first new image.

[0069] Image processing is performed on both the original and new images. The images are converted to grayscale, filtered, and a contour detection algorithm is used. In the original image, sub-pixel detection is performed on the circular region at each vertex of the square, extracting 50 pixel coordinates at equal angles on each circle. This indicates that on the original image, at square A... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Ath i The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, at square B... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Bth i The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, at the Cth position of the square... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Cth i The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, at square D... i The homogeneous coordinates of the j-th pixel on the circumference at the vertex. Indicates the Dth i The homogeneous coordinates of the vertex on the new image; Represents the scale parameter (where i = 1, 2, j = 1, 2, ..., 50).

[0070] 4. Using the data obtained above, calculate the Lagrange equation for the objective function:

[0071]

[0072] In the formula, f(M) (1) R represents the fitting error of the eight circles in the first new image, i.e., the result of the objective function is R. (1) :

[0073]

[0074] 5. Using gradient descent, update the parameters of the objective function in the direction of the negative gradient to obtain the second inverse perspective transformation matrix M. (2) :

[0075]

[0076] After each parameter update, check whether the constraints are met. If not, the parameter values ​​need to be recalculated to ensure that the constraints are satisfied.

[0077] 6. After k iterations, the inverse perspective transformation matrix is ​​M. (k) This yields the k-th new image from the original image. Image processing is then performed on both the original image and the k-th new image to obtain the data required for the above calculations. Finally, the objective function R is calculated. (k) The parameters updated for the kth time are as follows:

[0078]

[0079] According to the inverse perspective transformation matrix M (k+1) This yields the (k+1)th new image of the original image. Image processing is then performed on both the original image and the (k+1)th new image to obtain the data required for the above calculations. The result R of the objective function is then calculated. (k+1) .

[0080] 7. If R (k) and R (k+ 1 ) If the difference is small enough, stop the iteration; otherwise, repeat step 6 above.

[0081] Let M be the inverse perspective transformation matrix at the point of stopping iteration. * That is, the optimal inverse perspective transformation matrix.

[0082] 8. According to the requirements for rice plant measurement, images of rice paddies without calibration devices are collected in the scene as the images to be measured. The images are converted to grayscale and filtered to separate the rice plants from the background. Contour detection is used on the preprocessed images to extract the contour of each rice plant. Based on the position of the rice plant contour, the sub-pixel coordinates of the center point of each rice plant are extracted.

[0083] Use the coordinates obtained above as Substitution Obtain the physical coordinates of the center point of each rice plant. By fitting the center point of each row of rice plants using the least squares method, the center line of that row is obtained. The distance between adjacent center lines is calculated and denoted as the row spacing.

[0084] 9. Before measuring the row spacing of rice plants based on optimized inverse perspective transformation, the above-mentioned camera image acquisition and parameter optimization process is first performed. Next, the image containing rice plants acquired by this camera is used as the image to be measured. An inverse perspective transformation is performed using the optimized inverse perspective model. Based on the spacing between the center lines of the rice plant rows containing the target of interest in the bird's-eye view image, the row spacing of the rice plants can be obtained.

[0085] Through the above process, by optimizing the parameters of the inverse perspective transformation model, the fitting error of the new image is made sufficiently small, meaning that the new image after the inverse perspective transformation is close to the ideal situation. Compared with the inverse perspective transformation based on the homography matrix calculated in one step using four pairs of coordinate points, this method improves the accuracy of the model parameters and the accuracy of rice row spacing measurement.

[0086] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for measuring rice row spacing based on optimized inverse perspective transformation, characterized in that, Includes the following steps: S1. Set calibration components; the calibration components include... n A series of nested squares, with each of the four vertices containing a circle centered at that vertex and with a radius of [missing information]. r A circle; S2. Establish the objective function for optimizing the inverse perspective transformation; acquire the original image containing the calibration component, determine and substitute the initial values, iteratively solve the objective function, and obtain the optimal inverse perspective transformation matrix; in step S2, establishing the objective function for optimizing the inverse perspective transformation specifically involves: With calibration parts The objective is to minimize the sum of the fitting errors of each circle. In each of the squares, two pairs of opposite sides are parallel and two pairs of adjacent sides are perpendicular, and the absolute value of the difference in the length of the adjacent sides should be less than 1 / 3. Establish an objective function to define the constraints. Let the objective function be a small positive constant, as follows: In the formula, It is the inverse perspective transformation matrix. for The sum of fitting errors for each circle: In the formula, This indicates that on the original image, in the square... The first circle at the vertex homogeneous coordinates of each pixel Indicates the first The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, in the square... The first circle at the vertex homogeneous coordinates of each pixel Indicates the first The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, in the square... The first circle at the vertex homogeneous coordinates of each pixel Indicates the first The homogeneous coordinates of the vertex on the new image; This indicates that on the original image, in the square... The first circle at the vertex homogeneous coordinates of each pixel Indicates the first The homogeneous coordinates of the vertex on the new image; , , , Denotes the scale parameter, where, , , and ; n This indicates the number of squares in the calibration component. m This represents the number of pixels on the circumference at the vertex of the square. Equality constraints ,in, The details are as follows: Equality constraints ,in, The details are as follows: Equality constraints ,in, The details are as follows: Equality constraints ,in, The details are as follows: Inequality constraints ,in, The details are as follows: In the formula, , ; Indicates the first The homogeneous coordinates of the vertex in the original image. Indicates the first The homogeneous coordinates of the vertex in the original image. Indicates the first The homogeneous coordinates of the vertex in the original image. Indicates the first The homogeneous coordinates of the vertex in the original image; , , , Indicates the scale parameter; Based on Lagrange multiplier theory, the aforementioned constrained objective function can be rewritten as an unconstrained objective function, as follows: In the formula, , , , , Represents the Lagrange multiplier; S3. Collect images of rice plants and preprocess them; extract the outline of the rice plants and analyze the center coordinates of the rice plants; substitute the inverse perspective transformation formula composed of the optimal inverse perspective transformation matrix to obtain the physical coordinates of the center point of the rice plants. S4. Fit the center point of each row of rice plants according to the least squares method to obtain the center line of each row of rice plants, calculate the distance between adjacent center lines, and record it as the rice plant row spacing.

2. The method for measuring rice row spacing based on optimized inverse perspective transformation as described in claim 1, characterized in that, In step S1, n The side lengths of the squares are as follows: , ... ,satisfy and ;in, and They represent the first The and the first The side length of the square, and .

3. The method for measuring rice row spacing based on optimized inverse perspective transformation as described in claim 1, characterized in that, In step S2, the inverse perspective transformation specifically includes: To construct a new image, each pixel in the original image is mapped to a new location. The mathematical model is as follows: In the formula, , Represents any point on the new image; , Represents any point on the original image; Indicates the scale parameter; It is the inverse perspective transformation matrix.

4. The method for measuring rice row spacing based on optimized inverse perspective transformation as described in claim 1, characterized in that, In step S2, the acquisition of the original image containing the calibration component, the determination and substitution of initial values, and the iterative solution of the objective function to obtain the optimal inverse perspective transformation matrix are specifically as follows: S2.1 According to the requirements for measuring the row spacing of rice plants, set up calibration components in the scene and collect on-site images as the original images; S2.2 Manually select the four vertices of any square on the original image, and center on each vertex. Sub-pixel detection of vertices is performed within the pixel neighborhood, where... and Using the sub-pixel coordinates and physical coordinates of the four vertices, four sets of initial point pairs are constructed, with the coordinates of the four sets of initial point pairs as the initial values; S2.3, The first inverse perspective transformation matrix is ​​obtained by calculating using the initial values. Transform the original image into the first new image; S2.4, after In the next iteration, the inverse perspective transformation matrix is: The first image obtained is the original image. A new image; S2.5, respectively process the original image and the... The new image is processed to obtain the homogeneous coordinates required for calculating the objective function, and the result of the objective function is calculated. ; S2.6 Update the parameters of the objective function using gradient descent; S2.7, Based on the inverse perspective transformation matrix The first image obtained is the original image. A new image; S2.8, respectively process the original image and the... The new image is processed to obtain the homogeneous coordinates required for calculating the objective function, and the result of the objective function is calculated. ; S2.9, if and When the difference is less than a set value, the iteration stops, and the optimal inverse perspective transformation matrix is ​​obtained. Otherwise, repeat steps S2.4 to S2.

9.

5. The method for measuring rice row spacing based on optimized inverse perspective transformation as described in claim 4, characterized in that, Step S2.6 specifically includes: Update the parameters of the objective function using the following formula: In the formula, It is the learning rate during parameter updates. Represents the objective function Find the gradient.

6. The method for measuring rice row spacing based on optimized inverse perspective transformation as described in claim 4, characterized in that, Step S3 specifically includes: S3.

1. According to the requirements for rice plant measurement, collect images of rice paddies in the scene as the images to be measured; S3.2 Preprocess the image to be tested to separate the rice plants from the background; S3.

3. The preprocessed image is used for contour detection to extract the contour of each rice plant. Based on the position of the rice plant contour, the sub-pixel coordinates of the center point of each rice plant are extracted. S3.4, Substitute the subpixel coordinates The inverse perspective transformation formula, constructed from the optimal inverse perspective transformation matrix, is used. The physical coordinates of the center point of the rice plant are obtained; , Represents any point on the new image; , Represents any point on the original image; Indicates the scale parameter; It is the optimal inverse perspective transformation matrix.

7. The method for measuring rice row spacing based on optimized inverse perspective transformation as described in claim 6, characterized in that, In step S3.2, the preprocessing includes converting the image to grayscale, filtering, and binarizing it.

Images