A spatial straight line fitting method based on photogrammetry
By using weighted optimization methods in photogrammetry, using camera distortion parameters to calculate confidence and establish an objective function, the problem of large fitting error in traditional photogrammetry is solved, and more accurate spatial linear fitting and edge information extraction are achieved.
Patent Information
- Application Number
- CN202210030618.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-12
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-01-12
AI Technical Summary
In the field of engineering measurement, traditional photogrammetry methods have large errors when fitting straight lines in space due to camera distortion, making it difficult to accurately extract the edge information of the object to be tested.
Weighted optimization method is adopted to obtain distortion parameters through camera calibration, calculate the confidence of spatial points, and establish an objective function to solve spatial linear equations, reduce errors, and improve fitting accuracy.
By distorting into weights, the error is significantly reduced, the accuracy of fitting the straight line is improved, and more accurate edge information is obtained.
Smart Images

Figure CN114463356B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of photogrammetry, and specifically relates to a spatial straight line fitting method based on photogrammetry, which is used for extracting spatial edge information of an object to be measured. Background Art
[0002] Close-range photogrammetry is a measurement method that can instantly obtain a large amount of physical and geometric information of the measured object. Today, it is widely used in the field of engineering measurement and has become an important part of industrial product classification, guidance, monitoring, assembly and automated production. It is a non-contact measurement method, and its basic principle is still based on analytic geometry. According to the basic principle of photogrammetry, direct calculation of the corresponding spatial points from the points on the image will produce certain errors, especially in the measurement fields such as industry and construction, where straight lines need to be detected. How to improve the fitting accuracy of the extracted straight lines is of great significance to the practical application of close-range photogrammetry in the engineering field. Summary of the invention
[0003] The present invention provides a spatial straight line fitting method based on photogrammetry, which can make the straight line fitted by the spatial points obtained through the image points more accurate through the weighted optimization method, thus providing a basis for edge extraction.
[0004] The technical solution of the present invention is: a spatial straight line fitting method based on photogrammetry, comprising the following steps:
[0005] Step 1: Obtain camera intrinsic parameters and distortion through camera calibration;
[0006] Step 2: Select appropriate angles, distances, and control points, and use the calibrated camera to shoot the object to be measured. There are n camera stations in total, and each camera station shoots one image.
[0007] Step 3, according to the basic principle of photogrammetry, use the control points to complete the rear intersection of n images;
[0008] Step 4: Manually select m spatial points on the image that form a spatial straight line, and the same-name image points q on n images 1 ,q 2 ,……,q n The coordinates of the space point Q are obtained by forward intersection, and the three-dimensional coordinates of the selected m space points can be obtained by forward intersection of n images;
[0009] Step 5, according to the camera distortion, complete the calculation of the confidence f of the m spatial points;
[0010] Step 6: Establish a spatial straight line equation for the edge to be extracted and calculate the distance d from the spatial point i to the straight line i , for the convenience of calculation, directly take d i2 Calculate the objective function;
[0011] Step 7: Establish the objective function W according to the confidence of the spatial point and the distance from the spatial point to the straight line. The formula is shown in (9);
[0012]
[0013] Where i = 1, 2, ..., m, f i Represents the confidence of the i-th spatial point;
[0014] Step 8, solving the minimum value of the objective function W. When the objective function W reaches the minimum value, the expression of the spatial straight line equation is obtained, which is the spatial edge information of the object to be measured.
[0015] Furthermore, the distortion in step 1 includes radial distortion and tangential distortion. The radial distortion direction is divided into two directions: x and y. The radial distortion formula considers the first three terms of the Taylor series and can be obtained by camera calibration. Its expression is as follows:
[0016]
[0017] Tangential distortion is also divided into two directions: x and y. The tangential distortion formula is as follows:
[0018]
[0019] Taking radial distortion and tangential distortion into consideration, the distortion expression of a single pixel is calculated by formula (1) (2):
[0020]
[0021] In the formula,
[0022] r 2 =u 2 +v 2 , where u and v are the pixel coordinates in the x and y directions respectively.
[0023] A x To consider the distortion in the x-direction after radial distortion and tangential distortion,
[0024] A y To consider the distortion in the y direction after radial distortion and tangential distortion,
[0025] μ xj To consider the distortion in the x direction in radial distortion,
[0026] μ yj To consider the distortion in the y direction in radial distortion,
[0027] μxq To consider the distortion in the x direction in the tangential distortion,
[0028] μ yq To consider the distortion in the y direction in the tangential distortion,
[0029] κ 1 , κ 2 , κ 3 is the radial distortion parameter,
[0030] p 1 、p 2 is the tangential distortion parameter.
[0031] Furthermore, in step 5, the distortion of a single pixel is calculated as follows:
[0032]
[0033] Each spatial point in the space is calculated from n images, so the sum of the distortions corresponding to each spatial point is,
[0034]
[0035] The confidence level of each spatial point can be calculated from formula (6):
[0036]
[0037] Among them, i=1, 2, ..., m.
[0038] Furthermore, the expression of the space line L in step 6 is:
[0039]
[0040] x 0 ,y 0 、z 0 , a, b, c, t are all equation parameters;
[0041] Then the distance from the spatial point to the spatial straight line L to be fitted is:
[0042]
[0043] The coordinates of the space point are (x i ,y i , z i ), i = 1, 2, ..., m.
[0044] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: in some engineering fields, when edge information of the object to be measured is required, due to the existence of camera distortion, the traditional photogrammetry method has a large error when fitting a spatial straight line. The method proposed in the present invention uses distortion as a weight to fit a spatial straight line, which can reduce the error and improve the accuracy of the fitted straight line, thereby obtaining more accurate edge information. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flow chart of the present invention.
[0046] Figure 2 The photographing station is arranged near the crane object according to the present invention.
[0047] Figure 3 These are m points of the space straight line manually selected by the present invention.
[0048] Figure 4 The figure is a schematic diagram of the coordinates of a space point obtained by front intersection of image points in the present invention.
[0049] Figure 5 It is a schematic diagram of the distance from a spatial point to a straight line in the present invention. DETAILED DESCRIPTION
[0050] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0051] like Figure 1 As shown, a spatial straight line fitting method based on photogrammetry includes the following steps:
[0052] Step 1: Obtain camera intrinsic parameters and distortion through camera calibration;
[0053] The distortion in step 1 includes radial distortion and tangential distortion. The radial distortion direction is divided into two directions: x and y. The radial distortion formula considers the first three terms of the Taylor series and can be obtained by camera calibration. Its expression is as follows:
[0054]
[0055] Tangential distortion is also divided into two directions: x and y. The tangential distortion formula is as follows:
[0056]
[0057] Taking radial distortion and tangential distortion into consideration, the distortion expression of a single pixel is calculated by formula (1) (2):
[0058]
[0059] In the formula,
[0060] r2 =u 2 +v 2 , where u and v are the pixel coordinates in the x and y directions respectively.
[0061] A x To consider the distortion in the x-direction after radial distortion and tangential distortion,
[0062] A y To consider the distortion in the y direction after radial distortion and tangential distortion,
[0063] μ xj To consider the distortion in the x direction in radial distortion,
[0064] μ yj To consider the distortion in the y direction in radial distortion,
[0065] μ xq To consider the distortion in the x direction in the tangential distortion,
[0066] μ yq To consider the distortion in the y direction in the tangential distortion,
[0067] κ 1 , κ 2 , κ 3 is the radial distortion parameter,
[0068] p 1 、p 2 is the tangential distortion parameter.
[0069] Step 2: Select appropriate angles, distances, and control points, and use the calibrated camera to shoot the object to be measured. There are n camera stations in total, and each camera station shoots one image.
[0070] Step 3, according to the basic principle of photogrammetry, use the control points to complete the rear intersection of n images;
[0071] Step 4: Manually select m spatial points on the image that form a spatial straight line, and the same-name image points q on n images 1 ,q 2 ,……,q n The coordinates of the space point Q are obtained by forward intersection, and the three-dimensional coordinates of the selected m space points can be obtained by forward intersection of n images;
[0072] Step 5, according to the camera distortion, complete the calculation of the confidence f of the m spatial points;
[0073] In step 5, the projection of a spatial point in an image is an image point. A spatial point can be projected on multiple images to form multiple image points. Multiple image points corresponding to a spatial point are points of the same name. The distortion of a single image point is calculated as follows:
[0074]
[0075] Each spatial point in the space is calculated from n images, so the sum of the distortions corresponding to each spatial point is,
[0076]
[0077] The confidence level of each spatial point can be calculated from formula (6):
[0078]
[0079] Among them, i=1, 2, ..., m.
[0080] Step 6: Establish a spatial straight line equation for the edge to be extracted and calculate the distance d from the spatial point i to the straight line i , for the convenience of calculation, directly take d i 2 Calculate the objective function;
[0081] The expression of the space line L in step 6 is:
[0082]
[0083] x 0 ,y 0 、z 0 , a, b, c, t are all equation parameters;
[0084] Then the distance from the spatial point to the spatial straight line L to be fitted is:
[0085]
[0086] The coordinates of the space point are (x i ,y i , z i ), i = 1, 2, ..., m.
[0087] Step 7: Establish the objective function W according to the confidence of the spatial point and the distance from the spatial point to the straight line. The formula is shown in (9);
[0088]
[0089] Where i = 1, 2, ..., m, f i Represents the confidence of the i-th spatial point;
[0090] The implementation process of the present invention is described below with reference to specific examples:
[0091] (1) Figure 1As shown, the camera calibration is first completed to obtain the camera intrinsic parameters and distortion.
[0092] (2) Figure 2 As shown, select appropriate angles, distances and control points, use the calibrated camera to shoot the object to be measured, and use n camera stations, each of which shoots 1 image.
[0093] (3) According to the basic principles of photogrammetry, control points are used to complete the rear intersection of n images.
[0094] (4) Artificially select m points on the image that form a spatial straight line, such as Figure 3 As shown, the points are numbered 1-6, ie m=6.
[0095] (5) Figure 4 As shown, the same-named pixels q on n images 1 ,q 2 ,……,q n The coordinates of the spatial point Q are obtained by forward intersection. The three-dimensional coordinates of the m spatial points selected in (4) can all be obtained by forward intersection of n images.
[0096] (6) According to the camera distortion, the confidence f of m spatial points is completed using formula (6): i Calculation.
[0097] (7) Figure 5 As shown, establish the equation of the space line and calculate the distance d from the space point to the line i , for the convenience of calculation, directly take d i 2 Calculate the objective function.
[0098] (8) The objective function W is established based on the confidence of the spatial point and the distance from the spatial point to the straight line, and its formula is shown in (9).
[0099] (9) Solve the minimum value of the objective function W. When the objective function W reaches the minimum value, the corresponding spatial straight line equation is the equation to be fitted, that is, the spatial edge information of the crane object.
[0100] The matters not described in detail in this specification are all known technologies to those skilled in the art.
[0101] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
Claims
1. A spatial straight line fitting method based on photogrammetry, characterized in that: The steps include: Step 1: Obtain camera intrinsic parameters and distortion through camera calibration; Step 2: Select appropriate angles, distances, and control points, and use the calibrated camera to shoot the object to be measured. There are n camera stations in total, and each camera station shoots one image. Step 3, according to the basic principle of photogrammetry, use the control points to complete the rear intersection of n images; Step 4: manually select m spatial points on the image that form a spatial straight line, and the same-name image points q1, q2, ..., q n The coordinates of the space point Q are obtained by forward intersection, and the three-dimensional coordinates of the selected m space points can be obtained by forward intersection of n images; Step 5, according to the camera distortion, complete the calculation of the confidence f of the m spatial points; Step 6: Establish a spatial straight line equation for the edge to be extracted and calculate the distance d from the spatial point i to the straight line i , for the convenience of calculation, directly take d i 2 Calculate the objective function; Step 7: Establish the objective function W according to the confidence of the spatial point and the distance from the spatial point to the straight line. The formula is shown in (9); Where i = 1, 2, ..., m, f i Represents the confidence of the i-th spatial point; Step 8, solving the minimum value of the objective function W. When the objective function W reaches the minimum value, the expression of the spatial straight line equation is obtained, which is the spatial edge information of the object to be measured.
2. The method for spatial straight line fitting based on photogrammetry according to claim 1, characterized in that: The distortion in step 1 includes radial distortion and tangential distortion. The radial distortion direction is divided into two directions: x and y. The radial distortion formula considers the first three terms of the Taylor series and can be obtained by camera calibration. Its expression is as follows: Tangential distortion is also divided into two directions: x and y. The tangential distortion formula is as follows: Taking radial distortion and tangential distortion into consideration, the distortion expression of a single pixel is calculated by formula (1) (2): In the formula, r 2 =u 2 +v 2 , where u and v are the pixel coordinates in the x and y directions respectively. A x To consider the distortion in the x-direction after radial distortion and tangential distortion, A y To consider the distortion in the y direction after radial distortion and tangential distortion, μ xj To consider the distortion in the x direction in radial distortion, μ yj To consider the distortion in the y direction in radial distortion, μ xq To consider the distortion in the x direction in the tangential distortion, μ yq To consider the distortion in the y direction in the tangential distortion, κ1, κ2, κ3 are radial distortion parameters, p1 and p2 are tangential distortion parameters.
3. A spatial straight line fitting method based on photogrammetry as claimed in claim 2, characterized in that: In step 5, the projection of a spatial point in the image is an image point. A spatial point can be projected on multiple images to form multiple image points. The distortion of a single image point is calculated as follows: Each spatial point in the space is calculated from n images, so the sum of the distortions corresponding to each spatial point is, The confidence level of each spatial point can be calculated from formula (6): Among them, i=1, 2, ..., m.
4. The method for spatial straight line fitting based on photogrammetry according to claim 1, characterized in that: The expression of the space line L in step 6 is: x0, y0, z0, a, b, c, t are all equation parameters; Then the distance from the spatial point to the spatial straight line L to be fitted is: The coordinates of the space point are (x i ,y i , z i ), i = 1, 2, ..., m.
5. The spatial straight line fitting method based on photogrammetry according to claim 1, characterized in that: The value of m is 6.
