Complex deformation field measurement method for hole structure

Abstract

The invention discloses a method for measuring a complex deformation field for a hole structure. The method specifically comprises the following steps: step 1, making a random pattern with high contrast on the surface of a test piece to be tested; step 2, carrying out camera calibration by using a binocular vision system; step 3, acquiring images of the initial state and the deformation process of the tested piece, and detecting a region of interest (ROI) for the initial state of the left camera; step 4, dividing computational grids in the region of interest, and performing image matching by using a digital image correlation algorithm added with a semi-second-order shape function; 5, performing three-dimensional reconstruction on a matching result in each state based on camera calibration information, and solving a deformation field of each deformation state relative to the initial state; according to the method, the performance of the optimized structure can be quickly tested, the operation is simple, the calculation is efficient, and the measurement precision is high.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image correlation method in experimental mechanics, and particularly relates to a method for measuring complex deformation fields of hole structures. Background Art

[0002] Lightweighting based on topology optimization is in great demand in fields such as transportation or load-bearing like automobiles, bridge construction, and aerospace. For the reliability of parts during service, it is necessary to verify their mechanical properties after structural optimization. Currently, the commonly used method is to directly perform finite element analysis on the optimized CAD model. However, this method may lead to unreliable simulation results due to unreasonable mesh division or boundary conditions that do not match the actual situation.

[0003] In experimental mechanics, digital image correlation (DIC) is a simple non-contact measurement method. By using a camera to capture the speckle characteristics of the measured surface before and after being stressed, the deformation field can be obtained through image calculation. However, the structures after lightweight design are more complex, often having holes or distortions, and traditional DIC is difficult to handle such measurement objects with incomplete surfaces and complex spatial structures.

[0004] On the one hand, there are many holes in the optimized structure. In traditional DIC, by manually selecting the calculation area as a whole, in this way, the invalid background also participates in the calculation, which will not only cause redundant calculations, but also lead to large matching errors at the edge parts. Therefore, it is necessary to preprocess the image before calculation to extract the effective speckle calculation area. On the other hand, the optimized structure is irregular, which leads to instability in stereo matching of traditional correlation matching methods. Moreover, the deformation after loading may be uneven, increasing the difficulty of deformation tracking of images in the same sequence. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the present invention proposes a method for measuring complex deformation fields of hole structures. First, by using the gray information entropy of the foreground and background, the speckle calculation area in the reference image is automatically extracted, avoiding invalid calculations for non-measurement areas. Secondly, for the complex deformation of the surface of the structure with holes, a second-order deformation component is introduced to describe the complex mapping inside the sub-region, and the correlation matching algorithm is improved. In this way, while improving the matching accuracy, the calculation efficiency is ensured.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for measuring complex deformation fields of hole structures specifically includes the following steps:

[0008] Step 1, Speckle Coating: Use paint, a marker pen, or thermal transfer on the surface of the test piece to create a random pattern with strong contrast.

[0009] Step 2, Camera Calibration: Use Zhang Zhengyou's calibration method to solve the internal parameter matrix and external parameter matrix of the two cameras. The internal parameter matrix includes lens distortion, principal point deviation, and focal length. The external parameter matrix refers to the transformation relationship of the right camera coordinate system relative to the left camera coordinate system. After calibration, start collecting images.

[0010] Step 3, Image Acquisition and ROI Detection: Before the test piece is deformed under load, each of the left and right cameras collects an image as the initial state. As the test piece is deformed under load, the binocular cameras synchronously collect images during the deformation process. Select the area to be detected on the initial state of the left camera for image analysis, and automatically extract the effective area.

[0011] Step 4, Correlation Matching: Set the left image in the initial state as the reference image, and divide the calculation grid on the effective area extracted in Step 3. Each grid is a reference sub-region. Correlation matching includes two parts. One is the matching of the deformed image captured by the left camera with the reference image, called deformation tracking. The other is the matching of the left and right images in the same state, called stereo matching.

[0012] Step 5, 3D Reconstruction and Deformation Field Calculation: According to the results of camera calibration in Step 2 and the corresponding image points obtained after stereo matching in Step 4, reconstruct the 3D point set in each state. Subtract the corresponding point coordinates of each deformed state from those of the initial state to obtain the deformation at each position in different states, and complete the deformation field calculation.

[0013] Further, the specific method for image acquisition and ROI detection in Step 3 is as follows:

[0014] Step 3.1, Collect the initial image and deformed image of the test piece, select the area to be detected on the initial state of the left camera, and adjust the contrast of this area to be detected to reduce the influence of uneven illumination on ROI detection.

[0015] Step 3.2, Calculate the gradient for each pixel point in the result of Step 3.1. If the gradient of this pixel point is less than 10, it is the background, and set the gray value at this pixel point to 0. Otherwise, record the gradient value of this pixel point.

[0016] Step 3.3, For the result of Step 3.2, calculate the average value G of all non-zero pixel points on the image avg ;

[0017] Step 3.4, Select a window size of w×w pixels, calculate the window entropy, and set the threshold Th of the window entropy to G avg ×w 2 / 2. For each pixel in the area to be detected, if the window entropy is less than Th, the pixel at the window center belongs to the invalid area. Conversely, if the window entropy is greater than the threshold Th, it is considered that the window has speckle features and belongs to the valid area, and the pixel coordinates at the window center are marked.

[0018] Step 3.5. Output the marked pixels and display the ROI.

[0019] Further, the specific method of correlation matching in Step 4 is as follows:

[0020] Full-pixel matching uses small neighborhood linear search with gray-scale constraint and epipolar geometry constraint. Taking this result as the initial value for sub-pixel matching, the forward cumulative Gauss-Newton iteration method FA-GN is used to complete the matching process after several iterations.

[0021] The specific method of sub-pixel matching: After initializing the deformation and light intensity coefficients, a system of equations for the gray-scale mapping between the reference sub-region and the deformed sub-region is constructed for sub-pixel iteration. Before iteration, first judge the gray-scale of each pixel point f(x i , y i ) in the reference sub-region, and remove the equations corresponding to the background pixel points. At the edge of the ROI, some pixel points in the reference sub-region are the background; calculate the partial derivative matrix and update the iteration increment. If the difference between the unknown vector p before and after iteration is small, the calculation is completed; otherwise, calculate the partial derivative matrix and iteration increment again until the convergence condition is met.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention proposes a method for measuring complex deformation fields of hole structures. First, based on the difference in image entropy between the foreground and background, the region of interest of the reference image is automatically extracted, which simplifies the computational amount of correlation matching. Second, a semi-second-order shape function is introduced and the correlation matching algorithm is improved to make the matching of hole surface deformation more stable. In the structural design, after lightweighting, its mechanical properties need to be tested. This method can quickly perform performance tests on the optimized structure, with simple operation, high computational efficiency, and high measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of a method for measuring complex deformation fields of hole structures according to the present invention;

[0025] Figure 2 It is a schematic diagram of a measurement scheme for complex deformation fields of hole structures according to the present invention;

[0026] Figure 3 It is a schematic diagram of three-dimensional deformation field calculation of a method for measuring complex deformation fields of hole structures according to the present invention;

[0027] Figure 4 It is a flowchart of integer - pixel search based on linear intensity change for a method of measuring complex deformation fields of hole structures in the present invention;

[0028] Figure 5 It is a flowchart of ROI detection algorithm for a method of measuring complex deformation fields of hole structures in the present invention;

[0029] Figure 6 It is a processing effect diagram of each step of ROI detection for a method of measuring complex deformation fields of hole structures in the present invention;

[0030] Figure 7 It is a flowchart of sub - pixel iterative algorithm with semi - second - order shape function added in a method of measuring complex deformation fields of hole structures in the present invention;

[0031] Figure 8 It is a schematic diagram of the detection result of ROI in an embodiment of the present invention;

[0032] Figure 9 It is a schematic diagram of the matching result of left and right image sub - regions in an embodiment of the present invention;

[0033] Figure 10 It is a schematic diagram of seed point transfer in an embodiment of the present invention;

[0034] Figure 11 It is a schematic diagram of the three - dimensional deformation field of hole structures in an embodiment of the present invention. Specific embodiments

[0035] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] As Figure 1 shown, a method for measuring complex deformation fields of hole structures specifically includes the following steps:

[0037] Step 1, speckle coating: On the surface of the test piece, use paint, a marker pen, or a heat - transfer printing method to make a random pattern with strong contrast.

[0038] Step 2, camera calibration: Use Zhang Zhengyou's calibration method to solve the internal parameter matrix and external parameter matrix of the two cameras. The internal parameter matrix includes lens distortion, principal point deviation, and focal length. The external parameter matrix refers to the transformation relationship of the right - camera coordinate system relative to the left - camera coordinate system. After calibration, start collecting images.

[0039] Step 3, image acquisition and ROI detection: Before the test piece is deformed under load, each of the left and right cameras collects an image as the initial state. As the test piece is deformed under load, the binocular cameras synchronously collect images during the deformation process. Select the area to be detected on the initial state of the left - camera image for image analysis and automatically extract the effective area;

[0040] In the image of the hole structure, only the surface of the solid material attached with speckles is the object to be measured. During measurement, the DIC technique should be used to calculate only the area with speckles on the image. This part of the effective area is called ROI (Region of Interest). Since there are a large number of invalid areas on the hole image, in order to ensure calculation stability and save calculation costs, an algorithm should be used to automatically extract the effective area. The extraction of the effective area is namely ROI detection.

[0041] There may be holes on the surface of the structure after lightweight design, and it is meaningless to calculate these areas without material. The reference image can be divided into two regions according to features. One is the effective region covered with a speckle pattern on the surface, and the other is the invalid region including holes and the environment. Digital image correlation is based on the texture features of the image, and the stability of its matching depends on the rich information inside the sub-region. In the collected image, the measured solid surface is covered with speckles. The gray level of the speckle area jumps significantly, and this part of the effective area is called the foreground. While in the areas of holes and defocus, the gray level distribution is flat, the gray level difference between adjacent pixels is small, and the information contained is small. This part of the invalid area is called the background.

[0042] In image analysis, entropy is an index reflecting the information amount and complexity of the image. The larger the entropy of the image, the richer the pixel gray levels, the greater the information amount of the image, and vice versa. This is exactly the basis for distinguishing the foreground and the background. A small moving window can be set to calculate the gray level or gradient to reflect the information entropy.

[0043] The information entropy of the foreground is large, the gray level features are rich, the gray level difference is large, and the gray level distribution range is wide. The gray level variance can be used to describe the jumping degree of the gray level within the window:

[0044]

[0045] where f(x,y) is the gray level value, and f μ (x,y) is the average gray level within the window;

[0046] The information entropy of the background area is small, the gray level change is small, and the gray level gradient describes the difference between a pixel point and its neighboring pixels. The gradients in eight directions are accumulated and summed:

[0047]

[0048] The above formula can be simplified to the sum of the gradients in the horizontal and vertical directions: i, j are the pixel numbers of the window, and r, s are the numbers of the eight directions;

[0049]

[0050] where the gradients in the horizontal and vertical directions are f x(x, y) and f y (x, y) is calculated using forward differences;

[0051] As Figure 5 shown, the specific steps include:

[0052] Step 3.1: Collect the initial image and the deformed image of the test piece. Select the area to be detected in the initial state of the left camera. To reduce the influence of uneven illumination on ROI detection, adjust the contrast of this area to be detected, that is, perform a finite number of opening operations on the collected image, generally about 15 times, with a 5×5 pixel kernel, and take the result as 0.6 times the original image and subtract it from the original image;

[0053] Step 3.2: Calculate the gradient for each pixel point of the result of Step 3.1. If the gradient of this pixel point is less than 10, then it is the background, and set the gray value at this pixel point to 0. Otherwise, record the gradient value of this pixel point;

[0054] Step 3.3: For the result of Step 3.2, calculate the average value G of all non-zero pixel points on the image avg ;

[0055] Step 3.4: Select a window size of w×w pixels, calculate the window entropy, and set the threshold Th of the window entropy to G avg ×w 2 / 2. Make a judgment for each pixel point in the area to be detected. (Note that when the number of pixel points with gray value 0 in the window is greater than w 2 / 2, then the central pixel of this window obviously belongs to the invalid area). If the window entropy is less than Th, then the central pixel of the window belongs to the invalid area. Otherwise, if the window entropy is greater than the threshold Th, it is considered that this window has speckle characteristics and belongs to the valid area, and mark the pixel coordinates at the center of this window;

[0056] Step 3.5: Output the marked pixels and display the ROI.

[0057] Step 4: Correlation matching: Set the left image in the initial state as the reference image, and divide the calculation grid on the effective surface extracted in Step 3. Each grid is a reference sub-region. The correlation matching includes two parts. One is the matching between the deformed image captured by the left camera and the reference image, which is called deformation tracking; the other is the matching between the left and right images in the same state, which is called stereo matching;

[0058] As Figure 4 and Figure 7 shown, the correlation matching method includes two steps: integer-pixel matching and sub-pixel matching. Integer-pixel matching uses small neighborhood linear search with gray value constraint and epipolar geometry constraint. Taking this result as the initial value for sub-pixel matching, use the forward accumulation type Gauss-Newton iteration method (FA-GN) to complete the matching process after several iterations.

[0059] The lightweight structure has an irregular surface and exhibits non-uniform deformation during compression. In this invention, a semi-second-order shape function is used and the sub-pixel matching algorithm is improved to enhance the accuracy of deformation matching.

[0060] Before and after deformation, the illumination on the surface of the sub-region may change. The commonly used gray-scale change model for the sub-region is a linear model:

[0061] g(x',y') = af(x,y) + b (4)

[0062] Where f(x,y) and g(x',y') are the gray-scale values of the reference sub-region and the deformed sub-region respectively; a and b are the light intensity change coefficients.

[0063] The corresponding points inside the reference sub-region and the deformed sub-region can form a system of multivariate non-linear over-determined equations, denoted as:

[0064] E i (p) = g(x i ',y i ) - af(x i ,y i ) - b = 0 (5)

[0065] Where E i (p) represents any point inside the sub-region. Performing a first-order Taylor expansion on E i (p) and ignoring the higher-order terms, its iterative format is:

[0066] E i (p (k) ) + ▽E i (p (k) )(p (k+1) - p (k) ) = 0 (6)

[0067] Where ▽E i (p (k) ) is the first-order partial derivative matrix. Let ▽E i (p (k) ) = J, then the solution of the system of equations (7) is the iterative format:

[0068] p (k+1) = p (k) - (J T J) -1 J T E(p (k) ) (7)

[0069] The shape function describes the deformation of the sub-region (subset). Selecting a semi-second-order shape function can describe complex deformations:

[0070]

[0071] It can describe the evolution from a rectangle to an arbitrary straight-sided quadrilateral. At the same moment, there is only a projective transformation between the left and right images, and the semi-second-order form function can better approximate the projective transformation.

[0072] Then the unknown vector p = [u, u x , u y , u xy , v, v x , v y , v xy , a, b] T , According to the derivative rule of multivariate composite functions, the partial derivative components in Equation (6) are:

[0073]

[0074] In particular, when j = 9, 10,

[0075] Denote The partial derivative matrix can be obtained:

[0076]

[0077] In Equation (10), the bicubic interpolation function is used to calculate the sub-pixel gray level and gradient.

[0078] In the above calculation process, Equation (5) can also be denoted as:

[0079]

[0080] The initial translation value (u 0 , v 0 ) of the deformed sub-region relative to the reference sub-region is obtained through integer-pixel search. After that, the sub-pixel matching process based on Newton iteration is as Figure 7 shown.

[0081] Specific approach:

[0082] After initializing the deformation and light intensity coefficients, a system of equations for the gray level mapping between the reference sub-region and the deformed sub-region, that is, Equation (11), is constructed for sub-pixel iteration;

[0083] Before iteration, first judge the gray level of each pixel point f(x i , y i ) in the reference sub-region, and remove the equations corresponding to the background pixel points. As Figure 4 shown, at the edge of the ROI, some pixel points in the reference sub-region are the background. Therefore, before iteration, first judge the gray level of each pixel point f(x i , y i ) in the reference sub-region, and remove the equations corresponding to the background pixel points;

[0084] Calculate the partial derivative matrix and update the iteration increment. If the difference between the unknown vector p before and after the iteration is small, the calculation is completed. Otherwise, calculate the partial derivative matrix and the iteration increment again until the convergence condition is met.

[0085] Step 5, 3D reconstruction and deformation field calculation: Based on the camera calibration results in step 2 and the corresponding image points obtained after stereo matching in step 4, reconstruct the 3D point set in each state, and subtract the coordinates of the corresponding points in each deformation state from those in the initial state to obtain the deformation of each position in different states, complete the deformation field calculation, and the measurement is now complete. Figure 3 shown.

[0086] After ROI detection, the speckle calculation area can be well segmented. The processing effects of each step are as follows: Figure 6 After the ROI detection is completed, the effective area is divided into calculation grids, and the division results (such as Figure 9 (a)). Then, relevant matching is performed, and the matching scheme is as follows Figure 3 The whole pixel search part in the correlation matching uses a small area linear search with grayscale constraints and epipolar geometry constraints. The search method is as follows Figure 4 In sub-pixel matching, the present invention uses a semi-second-order shape function and improves the matching algorithm, which can improve the accuracy of deformation matching when facing the non-uniform deformation of the workpiece during the compression process. The iterative process is as follows: Figure 7 As shown, the specific processing steps are as follows:

[0087] The application of the present invention is explained by taking the measurement of the deformation field of a hollow thin-walled cylinder filled with meshes under pressure as an example. The measurement scheme is as follows: Figure 2 As shown. A tensile testing machine was used to apply pressure to the test piece, and binocular stereo vision and digital image correlation were used to measure the deformation field distribution of the test piece surface at different times. The deformation field of the mesh structure under pressure at different times was measured using two Point Grey cameras (model GS3-U3-41C6M, resolution 2048×2048 pixels), with a lens focal length of 25 mm and a measurement field of 100×100 mm. 2 , the camera optical axis angle is 24°, the distance between the two lens focal points is 124mm, and the distance from the binocular camera baseline to the test piece surface is 290mm. After the speckle is sprayed on the surface of the test piece, a piece of white paper is placed on the inner wall of the model to prevent the rear hole from interfering with the front. The stretching machine applies pressure at a speed of 2mm / min, and the camera acquisition frequency is 1Hz.

[0088] After camera calibration, a load is applied to the test piece and its deformation process is collected. Due to the observation angle of the camera, the speckle features on both sides of the measured cylinder in the image degenerate severely. To avoid affecting the deformation calculation, a speckle calculation area is roughly selected in the image artificially, such as Figure 8 (a) shows. Using ROI detection, the calculation result of window entropy is as shown in Figure 8 (b), and the calculated foreground is as shown in Figure 8 (c). Calculate the grid for the ROI. The size of the sub-region in the correlation matching is selected as 31×31 pixels 2 , and the step size is 10 pixels. The grid divided according to the foreground on the reference image is as shown in Figure 9 (a). Perform correlation matching on the grid. The matching result of the right camera is as shown in 9(b).

[0089] To ensure the stability of the matching, two seed points are added. Regarding the selection of seed points, in the image matching process, the seed points are calculated first, and the calculation results of the seed points are passed to the four-neighboring regions as initial values for iterative calculation, so as to ensure the robustness of the matching algorithm. As shown in Figure 10 , the seed points should preferably be selected in the central region with high speckle quality within a single connected region. The mesh structure has multiple strips intertwined, and the transfer efficiency of a single point is relatively low. However, setting too many seed points may encounter regions with severe speckle decorrelation during the transfer process, resulting in matching failure and making the algorithm unstable. Therefore, in this embodiment, two seed points are set according to the characteristics of the experimental object.

[0090] After the correlation matching is completed, according to the results of camera calibration and the corresponding image points obtained after matching, a three-dimensional point set for each state is reconstructed. The differences between the corresponding point coordinates of each deformation state and the initial state are calculated to obtain the deformations at various positions in different states, and the deformation field calculation is completed. The surface deformations at four moments during the compression process of the test piece are selected for display, as shown in Figure 11 .

[0091] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific embodiments of the present invention are limited to this. For those of ordinary skill in the technical field to which the present invention belongs, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the patent protection scope determined by the claims submitted by the present invention.

Claims

1. A method for measuring complex deformation fields of hole structures, characterized in that: The specific steps include: Step 1, speckle coating: Use paint, marker or thermal transfer to create a high-contrast random pattern on the surface of the test piece; Step 2, camera calibration: Use Zhang Zhengyou calibration method to solve the intrinsic parameter matrix and extrinsic parameter matrix of the two cameras. The intrinsic parameter matrix includes lens distortion, principal point deviation and focal length. The extrinsic parameter matrix refers to the transformation relationship between the right camera coordinate system and the left camera coordinate system. After calibration, start collecting images; Step 3, image acquisition and ROI detection: Before the test piece is deformed by the load, the left and right cameras each capture an image as the initial state. As the test piece begins to deform by the load, the binocular camera synchronously captures images of the deformation process, selects the area to be detected in the initial state of the left camera for image analysis, and automatically extracts the effective format; Step 4, correlation matching: Set the left image in the initial state as the reference image, and divide the calculation grid on the effective format extracted in step 3. Each grid is a reference sub-area. The correlation matching includes two parts. One is the matching of the deformed image taken by the left camera with the reference image, which is called deformation tracking; the other is the matching of the left and right images in the same state, which is called stereo matching. Step 5, 3D reconstruction and deformation field calculation: Based on the camera calibration results in step 2 and the corresponding image points obtained after stereo matching in step 4, reconstruct the 3D point set in each state, and subtract the coordinates of the corresponding points in each deformed state from those in the initial state to obtain the deformation of each position in different states, and complete the deformation field calculation.

2. A method for measuring complex deformation fields of hole structures according to claim 1, characterized in that: The specific steps for image acquisition and ROI detection in step 3 are as follows: Step 3.1, collect the initial image and the deformed image of the test piece, select the area to be detected in the initial state of the left camera, and adjust the contrast of the area to be detected in order to reduce the influence of uneven illumination on ROI detection; Step 3.2, calculate the gradient of each pixel of the result of step 3.

1. If the gradient of the pixel is less than 10, it is the background and the gray value of the pixel is set to 0. Otherwise, record the gradient value of the pixel. Step 3.3, for the result of step 3.2, calculate the average value G of all non-zero pixels on the image avg ; Step 3.4, select the window size as w×w pixels, calculate the window entropy, and set the threshold Th of the window entropy to G avg × 2 / 2, judge pixel by pixel in the area to be detected. If the window entropy is less than Th, the central pixel of the window belongs to the invalid area. On the contrary, if the window entropy is greater than the threshold Th, the window is considered to have speckle characteristics and belongs to the valid area, and the pixel coordinates at the center of the window are marked; Step 3.5: Output the labeled pixels and display the ROI.

3. The complex deformation field measurement method for hole structure according to claim 1 is characterized in that: The specific steps for the relevant matching in step 4 are as follows: Integer pixel matching uses a small neighborhood linear search with grayscale constraints and epipolar geometry constraints, and uses this result as the initial value for sub-pixel matching. The forward accumulation Gauss-Newton iteration method FA-GN is used to complete the matching process after several iterations. The specific method of sub-pixel matching is as follows: after initializing the deformation and light intensity coefficients, a set of equations for grayscale mapping between the reference sub-region and the deformed sub-region is constructed to perform sub-pixel iteration; before iteration, the value of each pixel point f(x i ,y i ) grayscale, remove the equation corresponding to the background pixels, at the edge of the ROI, a part of the pixels in the reference sub-area are the background; calculate the partial derivative matrix and update the iteration increment. If the difference between the unknown vector p before and after the iteration is small, the calculation is completed; otherwise, calculate the partial derivative matrix and the iteration increment again until the convergence condition is met.

Images