Visual measurement cross-medium optical path parameter calibration method and system for severe construction scene based on refraction field reconstruction

Abstract

The invention discloses a refraction field reconstruction-based cross-medium optical path parameter calibration method and system for visual measurement of a severe construction scene, and relates to the field of visual measurement of the severe construction scene. The system is mainly invented for solving the problem of vision measurement failure caused by the existing vision measurement system. The method comprises the following steps: acquiring a camera parameter calibration image group and a single light path parameter calibration image; camera parameter calibration is independently completed, and camera internal parameters, a distortion coefficient and a last target attitude matrix are provided for optical path parameter calibration; constructing a refraction field reconstruction model, and establishing a pixel coordinate optimization function about an intersection point of a window medium normal passing through a lens optical center and a camera image plane; the refractive index is gradually changed in a preset search interval, the minimization solving process of the pixel coordinate optimization function is repeatedly expanded, and when the pixel coordinate optimization function reaches the minimum value in the search interval, the corresponding intersection point pixel coordinates and the refractive index are final light path parameters. The device has the advantage of solving the problems of vision measurement failure and the like caused by configuration of a protection window.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual measurement in harsh construction scenes, and particularly relates to a method and system for calibrating cross-medium optical path parameters of visual measurement in harsh construction scenes based on refractive field reconstruction. BACKGROUND

[0002] Visual measurement technology has the advantages of non-contact and continuous measurement, and can replace some dangerous and inefficient traditional construction techniques when applied to the field of engineering construction. However, due to the development of visual measurement technology, it is currently mainly applied to work scenes with stable and controllable environmental factors such as factory production lines and laboratories, and has not yet been widely used in harsh construction scenes such as extreme environments with high humidity, high pressure, dust, falling rocks and severe vibration represented by shield tunnel construction and TBM tunnel construction.

[0003] When visual measurement technology is introduced into harsh construction scenes, the visual system needs to be equipped with a protective window, and the window needs to have a certain thickness to withstand pressure and prevent breakage. However, thickening the protective window will cause cross-medium optical refraction, resulting in severe refraction distortion in the final image. After the thick protective window is installed, the traditional pinhole imaging model is invalid due to the optical refraction effect, and existing camera calibration methods represented by the most common Zhang's calibration method are based on the pinhole imaging model and cannot analyze and represent the cross-medium refracted optical path, thereby failing to guarantee the final visual measurement accuracy and making it difficult to be applied to actual engineering construction. For the optical refraction effect, domestic and foreign scholars have focused on underwater visual systems, and the refraction effect is different from that in non-underwater scenes. The corresponding calibration technology cannot meet the needs of visual measurement in harsh construction scenes. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a method and system for calibrating cross-medium optical path parameters of visual measurement in harsh construction scenes based on refractive field reconstruction, which can solve the problem of cross-medium optical path refraction and visual measurement failure caused by the need to configure a thick protective window for the visual measurement system in harsh construction scenes. Based on the independent completion of camera parameter calibration, optical path parameter calibration can be completed with only a single image, which is convenient to operate and has strong flexibility, and lays an important foundation for realizing cross-medium visual measurement in harsh construction scenes.

[0005] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0006] A method for calibrating cross-medium optical path parameters of visual measurement in harsh construction scenes based on refractive field reconstruction, comprising the following steps:

[0007] S11: complete the calibration image acquisition, and obtain the camera parameter calibration image group and the single light path parameter calibration image in sequence;

[0008] S12: extract the real pixel coordinates of all target points in the camera parameter calibration image group and the single light path parameter calibration image in sequence;

[0009] S13: independently complete the camera parameter calibration, and provide the camera internal parameter K, the distortion coefficients k1 and k2, and the last target posture matrix [R f t f ] for the light path parameter calibration;

[0010] S14: construct a refraction field reconstruction model, and establish a pixel coordinate optimization function P(u n , v n ) about the intersection point p n of the normal line of the window medium, the lens optical center and the camera image plane;

[0011] S15: change the refractive index n in the preset search interval by the search step b, repeatedly expand the minimization solving process of the pixel coordinate optimization function F(u n , v n ), and when the pixel coordinate optimization function F(u n , v n ) reaches the minimum value in the search interval, the corresponding intersection pixel coordinates (u n , v n ) and the refractive index n are the final light path parameters.

[0012] Preferably, the spatial position and posture of the last image of the camera parameter calibration image group and the single light path parameter calibration image in step S11 are kept the same. After the last camera parameter calibration image is acquired, the target and the camera are kept fixed, the protective window of the vision measurement system is installed, and the single light path parameter calibration image under the same spatial position and posture is acquired.

[0013] Preferably, the mathematical expression of the refraction field reconstruction model in step S14 is as follows:

[0014]

[0015] In the formula, s is a scale factor, (u c , v c ) is the refractive projection point calculation pixel coordinates of the target point under the condition of the window medium without lens distortion, K is the camera internal parameter matrix, P is the world coordinates of the target point, t is the thickness of the window medium, and (u, v) is the real pixel coordinates of the refractive projection point of the target point under the condition of the window medium without lens distortion.

[0016] Preferably, the mathematical expression of the pixel coordinate optimization function F(u n , v n ) in step S14 is:

[0017]

[0018] wherein k is the number of target points in the calibration image.

[0019] Further, the initial value solving process of the intersection pixel coordinates (u n , v n ) in step S15 includes the following steps:

[0020] S51: according to the camera calibration result, solving the real pixel coordinates (u s , v s ) of the direct projection points of the target points in the last camera parameter calibration image under the condition of removing lens distortion in the non-window medium;

[0021] S52: the pixel coordinates (u s , v s ) and the pixel coordinates (u, v) of the corresponding target points in the light path parameter calibration image are on the intersection line L of the refracted light path and the camera image plane, and a two-point straight line equation is constructed by using the pixel coordinates (u s , v s ) and (u, v);

[0022] S53: repeating steps S51 and S52 in sequence for all target points;

[0023] S54: constructing a linear equation group with the number of equations being the number of target points by using the two-point straight line equations of all target points;

[0024] S55: solving the linear equation group by using the least square method to obtain the initial value of the intersection pixel coordinates (u n , v n ).

[0025] The application also provides a refractive field reconstruction-based visual measurement cross-medium light path parameter calibration system for a harsh construction scene, comprising:

[0026] an image acquisition module, configured to acquire a camera parameter calibration image group and a single light path parameter calibration image;

[0027] an image processing module, configured to extract target point pixel real coordinates containing lens distortion in the camera parameter calibration image group and the single light path parameter calibration image;

[0028] a camera calibration module, configured to execute a camera parameter calibration algorithm and output camera internal parameters, distortion coefficients and a last target attitude matrix;

[0029] The optical path calibration module is used to execute the optical path parameter calibration algorithm and output the pixel coordinates (u) of the intersection point where the glass medium normal intersects the lens optical center and the camera image plane. n v n ) and refractive index n.

[0030] The beneficial effects of this invention are as follows: Based on independently calibrating camera parameters, this invention employs a cross-medium optical path parameter calibration method for visual measurement in harsh construction scenarios, which requires only a single image to complete the calibration. This solves the prominent problems of cross-medium optical path refraction and visual measurement failure caused by the need for thickened protective windows in visual measurement systems under harsh construction scenarios. This invention can analytically characterize cross-medium refractive optical paths and realizes refraction field reconstruction, laying an important foundation for visual measurement applications in harsh construction scenarios. Attached Figure Description

[0031] Figure 1 This is a flowchart of the method for calibrating cross-medium optical path parameters in harsh construction scenarios based on refraction field reconstruction, as described in this invention.

[0032] Figure 2 This is a schematic diagram of the refraction field reconstruction model of the present invention;

[0033] Figure 3 The intersection pixel coordinates (u) of this invention n v n Flowchart of initial value solution;

[0034] Figure 4 This is a schematic diagram of the structure of the visual measurement cross-medium optical path parameter calibration system for harsh construction scenarios according to the present invention; in the figure: 1 image acquisition module, 2 image processing module, 3 camera calibration module, 4 optical path calibration module. Detailed Implementation

[0035] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0036] Reference Figure 1 A method for calibrating cross-medium optical path parameters in harsh construction scenarios based on refraction field reconstruction is presented, with the specific operation steps as follows:

[0037] S11: Complete the calibration image acquisition. Obtain the camera parameter calibration image group and the single light path parameter calibration image in sequence. The number of images in the camera parameter calibration image group is 15, and the number of target points in the target is selected as 20, which can meet the accuracy requirement of calibration.

[0038] S12: Extract the real pixel coordinates containing lens distortion of all target points in the camera parameter calibration image group and the single light path parameter calibration image in sequence.

[0039] S13: Independently complete the camera parameter calibration by using the commonly used Zhang calibration method, etc., to provide the camera internal parameters K, distortion coefficients k1, k2 and the last target attitude matrix [R f t f ].

[0040] S14: Construct a refraction field reconstruction model to establish a pixel coordinate optimization function F(u n , v n ) about the intersection point p n of the normal line of the view window medium, the lens optical center and the camera image plane.

[0041] S15: Change the refractive index n in the preset search interval by b as the search step length, repeatedly expand the minimization solving process of the pixel coordinate optimization function F(u n , v n ), and when the pixel coordinate optimization function F(u n , v n ) reaches the minimum value in the search interval, the corresponding intersection pixel coordinates (u n , v n ) and the refractive index n are the final light path parameters. The search step length b is set to 0.001, and the preset search interval is the refractive index regular existence interval of the view window medium. For ordinary glass and organic glass, the preset search interval is [1.4, 1.6]. The method used in the minimization solving process is the Levenberg-Marquardt algorithm.

[0042] The spatial position and attitude of the last image of the camera parameter calibration image group and the single light path parameter calibration image in the above step S11 are kept the same. After the last camera parameter calibration image is acquired, the target and the camera are kept fixed, the protective view window of the vision measurement system is installed, and the single light path parameter calibration image under the same spatial position and attitude is acquired.

[0043] Referring to Figure 2 , the mathematical expression of the refraction field reconstruction model in the above step S14 is:

[0044]

[0045] In the formula, s is a scale factor, (uc v c ) is the pixel coordinate of the refracted projection point of the target point under the condition of a windowed medium, after removing lens distortion. K is the camera intrinsic parameter matrix, P is the world coordinate of the target point, t is the thickness of the windowed medium, and (u, v) is the true pixel coordinate of the refracted projection point of the target point under the condition of a windowed medium, after removing lens distortion.

[0046] The pixel coordinate optimization function F(u) in step S14 above n v n The mathematical expression for ) is:

[0047]

[0048] In the formula, k is the number of target points in the calibration image.

[0049] Reference Figure 3 Furthermore, the intersection point p of the normal to the aforementioned glass medium at the optical center of the lens and the image plane of the camera. n As the center of refraction distortion, it remains relatively unchanged under the refracted light path at different target points. This property can be used to obtain the pixel coordinates (u) of the aforementioned intersection point. n v n The initial value of ) is obtained, and the solution process includes the following steps:

[0050] S51: Based on the true pixel coordinates containing lens distortion in the last camera parameter calibration image, solve for the true pixel coordinates (u) of the direct projection point of the target point under windowless medium conditions, removing lens distortion, according to the camera calibration results. s v s ).

[0051] S52: Pixel coordinates (u s v s The pixel coordinates (u, v) of the corresponding target point in the optical path parameter calibration image are both located on the intersection line L of the refracted optical path and the camera image plane. Using the pixel coordinates (u... s v s Construct a two-point equation for a straight line using (u, v);

[0052] S53: Repeat steps S51 and S52 sequentially for all target points.

[0053] S54: Using the two-point linear equations of all target points, construct a system of linear equations with the number of target points.

[0054] S55: Solve the system of linear equations using the least squares method to obtain the pixel coordinates of the intersection point (u). n v n The initial value of ).

[0055] ReferenceFigure 4 The application also provides a refractive field reconstruction-based visual measurement cross-medium optical path parameter calibration system for a harsh construction scene, comprising:

[0056] An image acquisition module is configured to acquire a camera parameter calibration image group and a single optical path parameter calibration image.

[0057] An image processing module is configured to extract real pixel coordinates of target points containing lens distortion in the camera parameter calibration image group and the single optical path parameter calibration image.

[0058] A camera calibration module is configured to execute a camera parameter calibration algorithm and output camera internal parameters K, distortion coefficients k1 and k2, and a last target posture matrix [R f t f ].

[0059] An optical path calibration module is configured to execute an optical path parameter calibration algorithm and output intersection pixel coordinates (u n , v n ) of a glass medium normal passing through a lens optical center and a camera image plane and a refractive index n.

[0060] The application can complete cross-medium optical path parameter calibration by using only a single image on the basis of independently completing camera calibration, realize optical path path analysis and characterization, and solve the problems of cross-medium optical path refraction and visual measurement failure caused by the need to configure a thick protective window in a visual measurement system in a harsh construction scene, thereby laying an important foundation for visual measurement application in a harsh construction scene. The application has no limitation on the camera calibration method and can be directly combined with the existing camera calibration method for use, has good realizability and economy, and can be widely applied to a harsh construction scene such as a shield method and a TBM method tunnel.

[0061] The above describes many specific details to facilitate a full understanding of the application, but those skilled in the art can still determine or deduce many other variants or modifications conforming to the principles of the application according to the disclosed content of the application without departing from the spirit and scope of the application. Therefore, the application is not limited by the above disclosed specific embodiments, and the scope of the application should be understood and recognized as covering all these variants or modifications.

Claims

1. A method for calibrating cross-medium optical path parameters in visual measurement of a harsh construction scene based on refraction field reconstruction, characterized in that: Comprising the following steps; S11: Complete the calibration image acquisition, and obtain the camera parameter calibration image group and the single light path parameter calibration image in turn; S12: Extract all target point real pixel coordinates containing lens distortion from the camera parameter calibration image group and the single light path parameter calibration image in turn; S13: independently complete camera parameter calibration, provide camera internal parameters K, distortion coefficients k1, k2 and the last target attitude matrix [R f t f ] for light path parameter calibration; S14: Construct a refractive field reconstruction model, establish a pixel coordinate optimization function F(u n ,v n ) about the intersection point p of the normal line of the view window medium through the lens optical center and the camera image plane n ); S15: change the refractive index n in the preset search interval by b as the search step, repeat the minimization solving process of the pixel coordinate optimization function F(u n , v n ), when the pixel coordinate optimization function F(u n , v n ) reaches the minimum value in the search interval, the corresponding intersection pixel coordinates (u n , v n ) and the refractive index n are the final optical path parameters.

2. The refractive field reconstruction-based visual measurement cross-medium light path parameter calibration method for harsh construction scenes according to claim 1, characterized in that: The spatial position and attitude of the last image of the camera parameter calibration image group and the single light path parameter calibration image in step S11 are kept the same.

3. The refractive field reconstruction-based visual measurement cross-medium light path parameter calibration method for harsh construction scenes according to claim 1, characterized in that: The mathematical expression of the refractive field reconstruction model in step S14 is: where s is a scale factor, (u c , v c ) is the pixel coordinate of the target point in the refracted projection point without lens distortion under the condition of the window medium, K is the camera internal parameter matrix, P is the world coordinate of the target point, t is the thickness of the window medium, and (u, v) is the real pixel coordinate of the target point in the refracted projection point without lens distortion under the condition of the window medium.

4. The refractive field reconstruction-based visual measurement cross-medium light path parameter calibration method for harsh construction scenes according to claim 1, characterized in that: The mathematical expression of the pixel coordinate optimization function F(u n , v n ) in step S14 is as follows: In the formula, k is the number of target points in the calibration image.

5. The refractive field reconstruction-based visual measurement cross-medium light path parameter calibration method for harsh construction scenes according to claim 1, characterized in that: The initial value solving process of the intersection point pixel coordinates (u n , v n ) in step S15 includes the following steps: S51: according to the camera calibration result, solving the direct projection point real pixel coordinates (u s , v s ) of the target point in the last camera parameter calibration image under the condition of removing lens distortion of the windowless medium; S52: Pixel coordinates (u s v s The pixel coordinates (u, v) of the target point in the optical path parameter calibration image are both located on the intersection line L of the refracted optical path and the camera image plane. Using the pixel coordinates (u... s v s Construct a two-point equation for a straight line using (u, v); S53: Repeat steps S51 and S52 for all target points in turn; S54: Use the two-point linear equation of all target points to construct a linear equation group with the number of equations equal to the number of target points; S55: Solve the linear equations by least square method to obtain the initial value of the intersection pixel coordinates (u n , v n ).

6. A refractive field reconstruction-based visual measurement cross-medium light path parameter calibration system for harsh construction scenes, characterized in that: Comprising: An image acquisition module for acquiring the camera parameter calibration image group and the single light path parameter calibration image; An image processing module for extracting target point real pixel coordinates containing lens distortion from the camera parameter calibration image group and the single light path parameter calibration image; a camera calibration module configured to execute a camera parameter calibration algorithm to output the camera intrinsic parameters K, distortion coefficients k1, k2, and a last target pose matrix [R f t f ] a light path calibration module configured to execute a light path parameter calibration algorithm to output a pixel coordinate (u n , v n ) of an intersection of the glass medium normal with a camera image plane through a lens optical center and a refractive index n.

Images