A computer vision-based method for measuring three-dimensional strain fields on cylindrical surface

Abstract

This paper proposes a computer vision-based method for measuring the three-dimensional strain field on the surface of a cylindrical structure. The method calibrates a binocular camera system and captures images of the cylindrical structure before and after deformation. A region of interest on the cylindrical surface is selected and divided into measurement grid points. The three-dimensional coordinates and displacements of each grid point in the camera coordinate system are obtained through stereo matching and temporal matching. A cylindrical standard coordinate system is established using nonlinear optimization, with the center of the cylinder base as the origin and the central axis as the z-axis. The three-dimensional coordinates and displacements of each grid point in this standard cylindrical coordinate system are then obtained. A calculation window is selected with each grid point as the center, and a local coordinate system within the tangent plane is established within the standard coordinate system. The three-dimensional coordinates and displacements of each point within the calculation window are mapped to the local coordinate system. Local least-squares fitting is then performed on each window to obtain the strain at the central grid point, achieving three-dimensional strain field measurement. The method has low computational complexity and high measurement efficiency and accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical measurement experimental mechanics, and in particular relates to a method for measuring three-dimensional strain fields on the surface of a cylindrical structure based on computer vision. Background Art

[0002] As a fundamental geometric structure, cylindrical structures are widely used in a wide range of fields, including aerospace, shipbuilding, electricity, and transportation. They serve as key components in numerous large-scale core industrial equipment. For example, the attitude and orbit control propulsion system in aerospace engineering contains numerous thin-walled pipes, such as fluid loop pipes. These are cylindrical structures that are subject to harsh mechanical environments such as impact loads during flight, potentially causing structural damage and even major accidents. Therefore, rapid and high-precision measurement of the three-dimensional strain field on the surface of cylindrical structures is crucial for ensuring the safe and reliable operation of industrial equipment.

[0003] Currently, there are two main methods for strain measurement: contact and non-contact. Traditional contact measurement methods require attaching strain gauges and other components to the surface of the object being measured. This method has significant limitations, including limited information, allowing only a small number of local points to be measured; the placement of the gauges requires surface treatment before attachment, making the operation cumbersome; and the strain gauges' sensitivity to environmental changes. Among non-contact measurement methods, the three-dimensional digital correlation image algorithm is a representative example. While it achieves non-contact measurement of strain fields, it requires extensive correlation function calculations and searches, resulting in low computational efficiency. Furthermore, when measuring the three-dimensional strain field of a curved surface, it requires numerous least-squares solutions on the tangent plane, resulting in high computational complexity and hindering practical application. Summary of the Invention

[0004] To overcome the shortcomings of existing strain measurement technologies and address the high computational complexity and low precision of current non-contact methods when measuring three-dimensional strain fields on curved surfaces, the present invention focuses on cylindrical structures and provides a computer vision-based method for measuring the three-dimensional strain field on the surface of cylindrical structures. This method has the advantages of low computational complexity, high measurement efficiency, and high precision, and is widely applicable to various cylindrical structural components in industry.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A method for measuring the three-dimensional strain field on the surface of a cylindrical structure based on computer vision comprises the following steps:

[0007] Step 1: Calibrate the high-resolution binocular camera system, obtain the intrinsic and extrinsic parameter matrices of the binocular camera, and collect digital images of the cylindrical structure before and after deformation;

[0008] Step 2: Select the surface area of interest and divide it into measurement grid points. Use stereo matching and temporal matching to obtain the pixel coordinates of the grid points before and after deformation. Calculate the 3D coordinates and 3D displacement of each grid point in the camera coordinate system based on the intrinsic and extrinsic parameter matrices of the binocular camera.

[0009] Step 3: By constructing an error function and performing nonlinear optimization on it, a cylindrical standard coordinate system is established with the center of the cylinder base as the origin and the central axis of the cylinder as the z-axis;

[0010] Step 4: According to the transformation relationship between the camera coordinate system and the cylindrical standard coordinate system, obtain the three-dimensional coordinates and three-dimensional displacement of each grid point in the cylindrical standard coordinate system;

[0011] Step 5: Select a calculation window with each grid point as the center, establish a local coordinate system in the tangent plane in the cylindrical standard coordinate system, and map the 3D coordinates and 3D displacements of each point in the calculation window in the cylindrical standard coordinate system to the local coordinate system;

[0012] Step 6: Perform local least squares fitting on each calculation window to obtain the Cauchy strain at the central grid point and realize the three-dimensional strain field measurement on the surface of the cylindrical structure.

[0013] Compared with the existing non-contact three-dimensional strain measurement method, the advantages of the present invention are:

[0014] (1) The present invention is a complete three-dimensional strain field measurement method for cylindrical structure surfaces. It does not rely on the coordinate system selection of the binocular camera during calibration and three-dimensional reconstruction, and is widely applicable to the surface strain field measurement of various cylindrical structural components in industry.

[0015] (2) Low computational complexity and high measurement efficiency. The geometric features of the cylindrical surface are effectively utilized, avoiding the use of general surface strain solution methods to solve the cylindrical surface strain, thus reducing unnecessary calculations.

[0016] (3) High measurement accuracy. By establishing a cylindrical standard coordinate system, the tangent plane at each grid point has a simple analytical expression, avoiding the cumulative error caused by solving the tangent plane through point-by-point least squares fitting in the strain field measurement method of general curved surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the three-dimensional strain field measurement method for the cylindrical structure surface based on computer vision.

[0018] Figure 2 This is the mathematical model for binocular camera imaging in the present invention.

[0019] Figure 3 Schematic diagram of selecting a region of interest and dividing calculation grid points in the present invention.

[0020] Figure 4 Schematic diagram of the multi-scale optical flow algorithm used in the present invention for time series matching.

[0021] Figure 5 Schematic diagram of the cylindrical standard coordinate system established for the present invention.

[0022] Figure 6 Schematic diagram of the local coordinate system with analytical expression established for the present invention.

[0023] Figure 7 This is a schematic diagram of the local least squares strain solution algorithm used in the present invention.

[0024] Figure 8 is the ε of the finite element software x Strain field simulation results (left) and ε measured using the algorithm of this invention x Comparison of strain fields (right).

[0025] Figure 9 is the ε of the finite element software y Strain field simulation results (left) and ε measured using the algorithm of this invention y Comparison of strain fields (right). DETAILED DESCRIPTION

[0026] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0027] like Figure 1 As shown, a method for measuring the three-dimensional strain field on the surface of a cylindrical structure based on computer vision includes the following steps:

[0028] Step 1: Calibrate the high-resolution binocular camera system, obtain the intrinsic and extrinsic parameter matrices of the binocular camera, and collect digital images of the cylindrical structure before and after deformation.

[0029] In the embodiment of the present invention, the binocular camera system is placed near the cylindrical structure to be measured so that the optical axes of the two cameras form an appropriate angle, such as Figure 2 As shown in the figure, the Zhang Zhengyou calibration method is used. The binocular camera is kept stationary, and the calibration plate is positioned at different locations. A homography matrix is established for each calibration plate position. Linear solvers and LM optimization are then performed to obtain the intrinsic and extrinsic parameter matrices of the two cameras, as well as the rotation and translation matrices between the two cameras. After calibration, the binocular camera is used to capture digital images of the cylindrical structure before and after deformation.

[0030] Step 2: Select the surface area of interest and divide it into measurement grid points. Obtain the pixel coordinates of the grid points before and after deformation through stereo matching and temporal matching. Calculate the 3D coordinates and 3D displacement of each grid point in the camera coordinate system based on the intrinsic and extrinsic parameter matrices of the binocular camera.

[0031] In the embodiment of the present invention, for the image before deformation captured by the left camera, the surface area of interest is selected and the calculation grid points are divided, such as Figure 3 As shown. Stereo matching is performed by combining epipolar constraints with SIFT feature matching. The process is as follows:

[0032] Assume that the image point p in the left camera image l =(u l , v l , 1) T , and the pixel p in the right camera image r =(u r , v r , 1) T If they are corresponding image points, then they should satisfy the epipolar geometric relationship, that is, satisfy the epipolar constraint equation:

[0033]

[0034] Among them, A l is the intrinsic parameter matrix of the left camera, A r is the intrinsic parameter matrix of the right camera, R is the rotation matrix between the left and right cameras, and S is converted from the translation matrix between the left and right cameras.

[0035] For each grid point p in the left camera image l , calculate its 128-dimensional SIFT feature descriptor vector L = (l1, l2, ..., l k ,…,l 128 ), and according to the epipolar constraint equation, a straight line is obtained for the right camera image, and the corresponding image point p r Calculate the SIFT feature descriptor R of each pixel in the neighborhood of this line. j =(r j1 , r j2 ,…,r jk ,…,r i128 ), and calculate the similarity measure d j :

[0036]

[0037] where l k is the kth component of the descriptor vector, r jk is the kth component of the descriptor vector of the jth point in the neighborhood.

[0038] Compare the similarity metrics. The pixel corresponding to the minimum similarity metric value is the corresponding pixel p in the right camera image. r :

[0039] pr =arg min(d j (R j , L))

[0040] After stereo matching, a multi-scale optical flow algorithm for virtual feature point tracking is used for temporal matching. The process is as follows:

[0041] For the two images before and after deformation taken by the same camera, a multi-scale image pyramid is established, such as Figure 4 As shown. Assume that the initial value of the optical flow iteration is Starting from the top layer, by minimizing the matching error function, the optical flow residual value ε of the grid point (x, y) at each layer is calculated L :

[0042]

[0043] Among them: I L and J L are the L-th layer images of the pyramid of the two frames of images, and are the optical flow residual values of the grid points in the horizontal and vertical directions in the L-layer image, ω x and ω y are the neighborhood sizes of the grid points in the horizontal and vertical directions on the L-layer graph, and are the pixel horizontal and vertical coordinates of the grid point in the L layer image, respectively.

[0044] The optical flow calculation result of this layer is passed to the next layer as the initial value. Repeat the inter-layer iterative operation until the bottom layer, and find the corresponding image point (x+d x , y+d y ), the timing matching is completed.

[0045] Step 3: By constructing an error function and performing nonlinear optimization on it, a cylindrical standard coordinate system is established with the center of the cylinder base as the origin and the central axis of the cylinder as the z-axis.

[0046] In the embodiment of the present invention, a nonlinear optimization method is used to obtain the origin coordinates of the cylindrical standard coordinate system and the direction vectors of the three coordinate axes. The three-dimensional coordinates of the cylindrical surface grid point obtained in step 2 are P i =(x i ,y i , z i ), i = 1, 2, 3..., the radius of the cylinder is r. Assume that the coordinates of the origin of the cylindrical standard coordinate system to be solved are P0 = (x0, y0, z0), and the direction vector of the z axis (center axis) is e z =(e1, e2, e3), such as Figure 5As shown. The distance d from each point on the cylindrical surface to the central axis i The calculation formula is as follows:

[0047]

[0048] The distance between each grid point on the cylindrical surface and the central axis should be as close to the radius r as possible. The following constrained optimization problem of the nonlinear error function is constructed:

[0049]

[0050] The direction vectors of the other two axes can be calculated based on the characteristics that the three direction vectors are orthogonal to each other and satisfy the right-hand rule in the right-hand coordinate system. x 、e y The following expression can be used for calculation to make it a standard right-hand coordinate system:

[0051]

[0052] Step 4: According to the transformation relationship between the camera coordinate system and the cylindrical standard coordinate system, obtain the three-dimensional coordinates and three-dimensional displacement of each grid point in the cylindrical standard coordinate system.

[0053] In the embodiment of the present invention, the three-dimensional coordinates P of each grid point in the camera coordinate system are i =(x i ,y i , z i ) and three-dimensional displacement D i =(u i , v i , w i ), according to the transformation relationship between the camera coordinate system and the established cylindrical standard coordinate system, the following calculation method is used to obtain the three-dimensional coordinates of each grid point in the cylindrical standard coordinate system and three-dimensional displacement

[0054]

[0055]

[0056] Among them, the upper right subscript n represents the physical quantity in the cylindrical standard coordinate system, and the lower right subscript i represents the i-th grid point, and the same below.

[0057] Step 5: Select the calculation window with each grid point as the center, establish the local coordinate system in the tangent plane in the cylindrical standard coordinate system, and map the three-dimensional coordinates and three-dimensional displacements of each point in the calculation window in the cylindrical standard coordinate system to the local coordinate system.

[0058] In the embodiment of the present invention, each grid point in the cylindrical standard coordinate system As the center, according to the displacement field smoothing requirement, a rectangular calculation window is selected. For each window, at the center grid point Establish a local coordinate system in the tangent plane at Figure 6 According to the special geometric relationship between the cylindrical standard coordinate system and the cylinder, the two direction vectors of the local coordinate system are Normal vector n to the tangent plane n There is an analytical expression:

[0059]

[0060]

[0061]

[0062] The three-dimensional coordinates of each grid point in the calculation window in the cylindrical standard coordinate system and three-dimensional displacement Map to the local coordinate system and obtain the coordinates in the local coordinate system of the cutting plane and

[0063]

[0064]

[0065] Among them, the upper right subscript local represents the physical quantity in the local coordinate system. is the coordinate vector of the i-th point in the window relative to the center point of the window, for The out-of-plane component of the tangent plane, for The in-plane component of the tangent plane, is the displacement vector of the i-th point in the window, for The out-of-plane component of the tangent plane, for The in-plane component of the tangent plane, the same below.

[0066] Step 6: Perform local least squares fitting on each calculation window to obtain the Cauchy strain at the central grid point and realize the three-dimensional strain field measurement on the surface of the cylindrical structure.

[0067] In an embodiment of the present invention, Figure 7 As shown, for each calculation window, the local displacement field u in the local coordinate system is local 、v local Satisfies a two-dimensional linear polynomial:

[0068]

[0069] In the formula, a0, a1, a2, b0, b1, b2 are the coefficients of the polynomial to be fitted, x local ,y local are the grid point coordinates in the local coordinate system of the tangent plane.

[0070] Substitute the local coordinates of each point in the calculation window and perform local least squares fitting:

[0071]

[0072] Where W is the selected calculation window area.

[0073] Based on the theory of elasticity, the Cauchy strain at the central grid point is calculated as follows:

[0074]

[0075] Where, ε x is the first normal strain component, ε y is the second normal strain component, γ xy is the shear strain component.

[0076] A calculation window is established for each grid point and the Cauchy strain is calculated to achieve three-dimensional strain field measurement on the surface of the cylindrical structure.

[0077] The present invention adopts the computer vision-based three-dimensional strain field measurement method of cylindrical structure surface. An industrial aluminum cylindrical structure is selected for experiment. Under constant load conditions, the surface three-dimensional strain field is measured. At the same time, a model of the aluminum cylindrical structure is established in commercial finite element simulation software. The same loading conditions and material parameters as in the experiment are applied, and finite element simulation analysis is performed. The strain field results of the present invention are compared with the finite element simulation results. Figure 8 、 Figure 9 From the comparison results, it can be seen that the present invention can effectively reduce the computational complexity while achieving high-precision measurement of the three-dimensional strain field on the surface of the cylindrical structure.

Claims

1. A method for measuring three-dimensional strain field on the surface of a cylindrical structure based on computer vision, characterized in that: The following steps are involved: Step 1: Calibrate the high-resolution binocular camera system, obtain the intrinsic and extrinsic parameter matrices of the binocular camera, and collect digital images of the cylindrical structure before and after deformation; Step 2: Select the surface area of interest and divide it into measurement grid points. Obtain the pixel coordinates of the grid points before and after deformation through stereo matching and time matching. Calculate the 3D coordinates and 3D displacement of each grid point in the camera coordinate system based on the intrinsic and extrinsic parameter matrices of the binocular camera. Step 3: By constructing an error function and performing nonlinear optimization on it, a cylindrical standard coordinate system is established with the center of the cylinder base as the origin and the central axis of the cylinder as the z-axis; Step 4: According to the transformation relationship between the camera coordinate system and the cylindrical standard coordinate system, obtain the three-dimensional coordinates and three-dimensional displacement of each grid point in the cylindrical standard coordinate system; Step 5: Select a calculation window with each grid point as the center, establish a local coordinate system in the tangent plane in the cylindrical standard coordinate system, and map the 3D coordinates and 3D displacements of each point in the calculation window in the cylindrical standard coordinate system to the local coordinate system; Step 6: Perform local least squares fitting on each calculation window to obtain the Cauchy strain at the central grid point and realize the three-dimensional strain field measurement on the surface of the cylindrical structure; In step 2, after stereo matching, a multi-scale optical flow algorithm for virtual feature point tracking is used to perform temporal matching. The process is as follows: For the two frames of images before and after deformation taken by the same camera, a multi-scale image pyramid is established; the initial value of the optical flow iteration is set to Starting from the top layer, by minimizing the matching error function, the optical flow residual value ε of the grid point (x, y) at each layer is calculated L : Where: I L and J L are the L-th layer images of the pyramid of the two frames of images, and are the optical flow residual values of the grid points in the horizontal and vertical directions in the L-layer image, ω x and ω y are the neighborhood sizes of the grid points in the horizontal and vertical directions on the L-layer graph, and are the pixel horizontal and vertical coordinates of the grid point in the L layer image, respectively; The optical flow calculation result of this layer is passed to the next layer as the initial value, and the inter-layer iteration operation is repeated until the bottom layer, and the corresponding image point (x+d x , y+d y ), completing the timing matching.

2. The method for measuring three-dimensional strain field on the surface of a cylindrical structure based on computer vision according to claim 1, characterized in that: In step 2, stereo matching is performed by combining epipolar constraint with SIFT feature matching. The process is as follows: Assume that the image point p in the left camera image l =(u l , v l , 1) T , and the pixel p in the right camera image r =(u r , v r , 1) T If they are corresponding image points, then they satisfy the epipolar constraint equation: Among them, A l is the intrinsic parameter matrix of the left camera, A r is the intrinsic parameter matrix of the right camera, R is the rotation matrix between the left and right cameras, and S is obtained by converting the translation matrix between the left and right cameras; For each grid point p in the left camera image l , calculate its 128-dimensional SIFT feature descriptor vector L = (l1, l2, ..., l k ,…,l 128 ), l k is the kth component of the descriptor vector, and a straight line of the right camera image is obtained according to the epipolar constraint equation, p r That is, on this straight line, calculate the SIFT feature descriptor R of each pixel in the neighborhood of this straight line. j =(r j1 , r j2 ,…,r jk ,…,r i128 ), r jk Describe the kth component of the subvector of the jth point in the neighborhood and calculate the similarity measure d j : Compare the similarity metrics, and the image point corresponding to the minimum similarity metric value is p r : <h2 style=";text-align:left;direction:ltr">p<h2 style=";text-align:left;direction:ltr"> r <h2 style=";text-align:left;direction:ltr"> =argmin(d<h2 style=";text-align:left;direction:ltr"> j <h2 style=";text-align:left;direction:ltr"> (R<h2 style=";text-align:left;direction:ltr"> j <h2 style=";text-align:left;direction:ltr"> (L)) 3. The method for measuring three-dimensional strain field on the surface of a cylindrical structure based on computer vision according to claim 1, characterized in that: In step 3, a nonlinear optimization method is used to obtain the origin coordinates of the cylindrical standard coordinate system and the direction vectors of the three coordinate axes; The three-dimensional coordinates of the grid points on the cylindrical surface obtained in step 2 are P i =(x i ,y i , z i ), i = 1, 2, 3..., the radius of the cylinder is r; suppose the coordinates of the origin of the cylindrical standard coordinate system to be solved are P0 = (x0, y0, z0), and the z-axis direction vector is e z =(e1, e2, e3); distance d from each point on the cylindrical surface to the central axis i The calculation formula is as follows: The distance between each grid point on the cylindrical surface and the central axis should be as close to the radius r as possible. The following constrained optimization problem of the nonlinear error function is constructed: The direction vectors of the other two axes are calculated based on the characteristics that the three direction vectors are orthogonal to each other and satisfy the right-hand rule in the right-hand coordinate system; thus, the direction vectors e of the x and y axes of the cylindrical standard coordinate system are x 、e y Take the following expression and calculate it to make it a standard right-hand coordinate system:

4. The method for measuring three-dimensional strain field on the surface of a cylindrical structure based on computer vision according to claim 1, characterized in that: In step 4, the three-dimensional coordinates P of each grid point in the camera coordinate system are i =(x i ,y i , z i ) and three-dimensional displacement D i =(u i , v i , w i ), according to the transformation relationship between the camera coordinate system and the established cylindrical standard coordinate system, the following calculation method is used to obtain the three-dimensional coordinates of each grid point in the cylindrical standard coordinate system and three-dimensional displacement The upper right subscript n represents the physical quantity in the cylindrical standard coordinate system, and the lower right subscript i represents the i-th grid point.

5. The method for measuring three-dimensional strain field on the surface of a cylindrical structure based on computer vision according to claim 1, characterized in that: In step 5, each grid point in the cylindrical standard coordinate system As the center, a rectangular calculation window is selected according to the requirement of displacement field smoothing; for each calculation window, a local coordinate system is established in the tangent plane at the grid point at the center of the rectangular window. According to the special geometric relationship between the cylindrical standard coordinate system and the cylinder, the two-way vectors of the local coordinate system are Normal vector n to the tangent plane n There is an analytical expression: The three-dimensional coordinates of each grid point in the calculation window in the cylindrical standard coordinate system and three-dimensional displacement Map to the local coordinate system and obtain the coordinates in the local coordinate system of the cutting plane and Among them, the upper right subscript local represents the physical quantity in the local coordinate system. is the coordinate vector of the i-th point in the window relative to the center point of the window, for The out-of-plane component of the tangent plane, for The in-plane component of the tangent plane, is the displacement vector of the i-th point in the window, for The out-of-plane component of the tangent plane, for The in-plane component of the tangent plane.

6. The method for measuring three-dimensional strain field on the surface of a cylindrical structure based on computer vision according to claim 1, characterized in that: In step 6, for each calculation window, the local displacement field u in the local coordinate system is set local 、v local Satisfies a two-dimensional linear polynomial: In the formula, a0, a1, a2, b0, b1, b2 are the coefficients of the polynomial to be fitted, x local ,y local is the grid point coordinate in the local coordinate system of the tangent plane; Substitute the local coordinates of each point in the calculation window and perform local least squares fitting: Where W is the selected calculation window area; Based on the theory of elasticity, the Cauchy strain at the central grid point is calculated as follows: Where, ε x is the first normal strain component, ε y is the second normal strain component, γ xy is the shear strain component; A calculation window is established for each grid point and the Cauchy strain is calculated to achieve three-dimensional strain field measurement on the surface of the cylindrical structure.

Images