Strain vibration field measurement system and method based on optical multi-view three-dimensional imaging

Abstract

The invention relates to a strain vibration field measurement system and method based on optical multi-view three-dimensional imaging. The system comprises a camera imaging parameter calibration module for calibrating relative imaging parameters of other cameras by taking a middle camera as a reference, and outputting camera parameters and independent calibration results of each visual angle under a unified coordinate system; the feature point matching module is used for completing accurate matching of same-name feature points at different visual angles and different moments and outputting a standardized feature point matching result; the feature point three-dimensional coordinate reconstruction module is used for converting feature point information in the two-dimensional image into three-dimensional space coordinates to obtain three-dimensional position data of feature points; the three-dimensional displacement and strain resolving module is used for calculating the three-dimensional displacement and strain of the feature points of the region to be measured by taking the initial moment as a reference and generating three-dimensional displacement field and strain field data; and the result visualization module is used for converting the numerical three-dimensional coordinates, displacement and strain data into visual visualization graphs. According to the invention, surface strain monitoring of an object in a running state is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision measurement technology, specifically to a strain vibration field measurement system and method based on optical multi-view three-dimensional imaging. Background Technology

[0002] With the development of the industrial sector, the demand and complexity of strain measurement are constantly increasing. Industrial structural health monitoring places higher demands on the efficiency, accuracy, and dynamic monitoring capabilities of surface strain measurement. Camera vision deformation measurement technology, with its advantages of being non-contact, non-destructive, and high-precision, is widely used in the three-dimensional deformation measurement of object surfaces. It obtains positional information by identifying structural features in images and calculates mechanical parameters such as deformation and vibration by combining them with time information.

[0003] Visual measurement techniques are divided into active and passive measurement methods. Active measurement includes structured light measurement and time-of-flight methods, with structured light measurement further categorized into temporal / spatial structured light coding based on the encoding method. Passive measurement includes monocular vision, binocular (multi-view) stereo vision, and digital speckle correlation methods (DSCM). Among these, DSCM, as a typical application of digital vision in strain measurement, measures the strain field by comparing speckle images before and after deformation, and has become an important means of optical mechanical deformation measurement.

[0004] Currently, researchers both domestically and internationally have conducted extensive research and accumulated rich results on two-dimensional digital speckle correlation methods (2D-DSCM). However, existing technologies still have significant limitations: traditional 2D-DSCM is insufficient to meet the measurement requirements of three-dimensional strain vibration fields; monocular / binocular vision measurement can only achieve centimeter-level position measurement accuracy in large-scale, complex conditions, and even m-level accuracy in some scenarios, failing to meet high-precision measurement needs. Furthermore, existing vision measurement technologies have not yet formed mature system solutions in areas such as multi-view camera collaborative calibration and feature point matching from different viewpoints, making it impossible to efficiently achieve accurate measurement of strain vibration fields using optical multi-view three-dimensional imaging.

[0005] Therefore, there is an urgent need to develop a strain vibration field measurement system and method based on optical multi-view three-dimensional imaging to solve the technical problem of deformation measurement of large-scale, high-precision objects and improve the efficiency and accuracy of strain vibration field measurement under complex working conditions. Summary of the Invention

[0006] To address the aforementioned technical challenges, this invention proposes a strain vibration field measurement system and method based on optical multi-view three-dimensional imaging, aiming to solve the problem of deformation measurement for large-scale, high-precision objects. Developing vision-based methods for object deformation measurement is of great significance.

[0007] To address the aforementioned technical problems, one objective of this invention is to provide a strain vibration field measurement system based on optical multi-view three-dimensional imaging, which, based on the image data processing requirements acquired by the camera, includes: The camera imaging parameter calibration module uses the intermediate camera as a reference to calibrate the relative imaging parameters of the other cameras, and finally outputs the camera parameters in a unified coordinate system and the independent calibration results of each viewpoint. The feature point matching module completes the accurate matching of the same feature points from different perspectives and at different times, and outputs standardized feature point matching results. The feature point 3D coordinate reconstruction module converts feature point information in a 2D image into 3D spatial coordinates to obtain the 3D position data of the feature points. The three-dimensional displacement and strain calculation module, which calculates the three-dimensional displacement field and strain field calculation functions, realizes the calculation of the three-dimensional displacement and strain of the feature points of the area to be measured with the initial time as a reference, and generates three-dimensional displacement field and strain field data. The results visualization module converts numerical three-dimensional coordinates, displacement, and strain data into intuitive visual graphics, facilitating the observation of three-dimensional deformation and strain distribution of the structure.

[0008] Furthermore, the camera imaging parameter calibration module specifically integrates Zhang Zhengyou's checkerboard calibration method and the speckle self-calibration method. The Zhang Zhengyou checkerboard calibration method completes the calculation by simultaneously acquiring checkerboard images of approximately 21 poses using three cameras. The speckle self-calibration method solves the problem by acquiring speckle images from multiple perspectives and utilizing multi-view geometric constraints.

[0009] Furthermore, the feature point matching module operates as follows: the input is speckle (sequence) images captured by three cameras. First, cross-viewpoint feature point matching is completed through an inter-viewpoint matching algorithm, and then optimized by combining an intra-viewpoint matching algorithm. Two-dimensional digital image correlation algorithm and SIFT algorithm are used for collaborative processing to solve the problem of feature point matching between intra-viewpoints and inter-viewpoints, and finally output the matching results of the same-name feature points at different viewpoints at various times.

[0010] Furthermore, the feature point 3D coordinate reconstruction module specifically comprises: the input of camera imaging parameter calibration results and feature point matching results; based on the principle of triangulation, combining the calibrated camera intrinsic and extrinsic parameters and the correspondence of matched feature points, completing the reconstruction of the feature point 3D coordinates through spatial geometric calculations, and outputting the feature point 3D coordinate data.

[0011] Furthermore, the specific operation of the three-dimensional displacement field and strain field calculation module is as follows: the input of this module is the three-dimensional coordinates of the feature point. First, the three-dimensional plane is fitted using the least squares method around the target point, and the plane equation is solved. A local coordinate system is established with the target point as the origin. The coordinates in the world coordinate system are transformed to the local coordinate system through Euclidean transformation. The displacement data is obtained by performing time differentiation on the local coordinates, and then fitted using a quadratic surface. Ue, Ve, We Three displacement field functions: ; in, These are the coefficients of the displacement field function. The strain tensor is calculated using the displacement field function as follows: ; The three-dimensional displacement and strain field data can be obtained through the above process.

[0012] Furthermore, the result visualization function module specifically includes: inputting the three-dimensional coordinates, three-dimensional displacement, and three-dimensional strain data of feature points, converting the data into a three-dimensional cloud map through color mapping, three-dimensional modeling, etc. The color gradient of the cloud map intuitively reflects the distribution characteristics of structural deformation and strain, and finally outputting a three-dimensional displacement field and strain field visualization cloud map of the area to be measured.

[0013] Based on the same concept, this invention also proposes a method for measuring strain vibration fields based on optical multi-view three-dimensional imaging, the specific steps of which are as follows: S1. Feature Point Extraction and Matching First, a multi-view camera array is controlled to synchronously acquire speckle feature images of the surface of the structure under test. Then, the local geometric features of each image feature point are extracted using the Scale Invariant Feature Transform (SIFT) algorithm. g j Compared with appearance description d j ;pass d j Initial matching is performed based on similarity, and then through... g j A second screening is performed based on geometric consistency, followed by geometric verification using the Random Sample Consensus (RANSAC) algorithm to eliminate incorrect matching point pairs caused by perspective transformations and nonlinear transformations; finally, the epipolar constraint formula is applied. Solve for the fundamental matrix F; select the solution with the minimum bidirectional reprojection error as the optimal solution, and verify the remaining matching point pairs by applying an epipolar constraint threshold. P 1 Let be the homogeneous coordinates of the feature points on the imaging plane of the first camera. P 2 These are the homogeneous coordinates of the corresponding feature points on the imaging plane of the second camera; S2. Solving for initial values ​​of imaging parameters First, the images are sorted by the number of matching feature points. The image with the most matching pairs is selected for initialization. Combining the estimated lens focal length and the initial value of the principal point position of the image center, the initial value of the distortion coefficients is set to 0. This is then determined by the fundamental matrix F and the corresponding intrinsic parameter matrix. K 1 and K 2 The essential matrix E is obtained by solving the correspondence relationship, and the relationship is shown below: ; in, t ^ 21 Let be the rotation matrix of camera 2 relative to camera 1. K 1 Let be the intrinsic parameter matrix of the first camera. K 2 Let E be the intrinsic parameter matrix of the second camera. Then, singular value decomposition is performed on E to obtain candidate solutions for the camera pose. The correct rotation matrix is ​​selected by judging the positive depth of the 3D point P in the camera coordinate system. R 21 With translation vector t 2 If the scene points are coplanar or the camera is in pure rotational motion, the camera extrinsic parameters are recovered using the identity matrix H. S3. Imaging Parameter Optimization and 3D Reconstruction First, at least one additional camera is introduced. The extrinsic parameters of the new camera are estimated using the perspective n-point method (PnP) combined with the coordinates of the reconstructed 3D points. Then, triangulation is performed on the new camera, and the positive depth of the 3D points is verified by the triangulation angle α, thus completing the reconstruction of the new 3D points. Simultaneously, a target function is constructed using bundle adjustment (BA) combined with the Levenberg-Marquardt method, as shown below: ; in, P c The middle part represents the set of camera parameters to be optimized, including intrinsic parameters. f, cx, cy, k1, k2 ]( f For camera focal length, cx , cy The coordinates of the principal point on the imaging plane. k1 , k2 (radial distortion coefficients) and extrinsic parameters (rotation matrix and translation vector). x j Let j be the spatial coordinates of the j-th 3D point. π(P c ,x j ) The projection function refers to the function that projects three-dimensional points. xj Through camera parameters Pc The pixel coordinates obtained after projection onto the imaging plane are calculated. Based on this objective function, the camera's intrinsic and extrinsic parameters and 3D point coordinates are optimized. After each optimization, large error observation results are removed by extrinsic filtering. Finally, known size / control points are introduced to supplement the self-calibrated absolute physical scale information to complete the 3D reconstruction. S4. Scale Information Supplement By introducing standard objects or ground control points of known size, the self-calibration and 3D reconstruction results are scaled to obtain a 3D reconstruction model with absolute physical scale.

[0014] The above-described one or more technical solutions of the present invention have at least one or more of the following technical effects: Based on the multi-view self-calibration method, in the process of solving camera parameters, the optimization objective is to minimize the reprojection error of feature points in each camera. The world coordinates of feature points and the intrinsic, distortion coefficients and extrinsic parameters of each camera are continuously adjusted, and finally the self-calibration of camera parameters is achieved. The test results show that the calibration results obtained by this invention meet the requirements of strain measurement.

[0015] To obtain the optimal camera spatial arrangement, this scheme pre-sets the camera array to be in a ring shape based on the 3D scene of the test site, and dynamically adjusts the number of cameras and the angle between them. The number of cameras is set to 3, 4, and 5, and the camera angles are set to the range of 5°, 10°, 15°, 20°, 25°, and 30°. Experimental results show that the best arrangement is 5 cameras with an angle of 10°.

[0016] This scheme obtains a one-to-one correspondence between each pixel within a common area across multiple viewpoints using Subset-DIC and performs same-viewpoint image matching based on digital image correlation algorithms. Simultaneously, it comprehensively utilizes the imaging parameters of all cameras, performing distortion correction processing on the pixel coordinates of all images and feature points using distortion parameters obtained from self-calibration before spatial displacement calculation. Then, using the information from self-calibration and image matching, a multi-view triangulation method is executed to calculate the 3D coordinates of each measurement point in the deformed image sequence. This yields the strain information of the feature points. Attached Figure Description

[0017] Figure 1 Schematic diagram of a multi-view stereo vision image fusion measurement system; Figure 2 Three-dimensional strain calculation; Figure 3 Example of cloud map visualization; Figure 4 : The principle of polar geometry; Figure 5Histogram (left) and time variation (right) of the equivalent focal length f of a camera obtained through self-calibration in a dynamic experiment; Figure 6 Histogram (left) and time-varying values ​​(right) of a camera principal point cx and cy obtained through self-calibration in a dynamic experiment; Figure 7 Histogram (left) and time-varying values ​​(right) of a certain camera distortion parameters k1 and k2 obtained through self-calibration in a dynamic experiment. Detailed Implementation

[0018] This invention proposes a high-precision synchronous triggering system for acquiring images using a multi-view camera array. This system ensures time synchronization between the triggering of multiple cameras and data acquisition, avoiding measurement accuracy degradation or even measurement failure caused by time errors between cameras. A self-calibration method based on a multi-view camera array is proposed. This method utilizes the texture information of the object being measured and achieves self-calibration of the intrinsic and extrinsic parameters and coordinate system of multiple cameras through motion reconstruction structures, avoiding the cumbersome steps of using specific calibration objects. Through multi-camera common-view constraints and temporal matching, the two-dimensional coordinates of feature points in deformed images are obtained. Then, a multi-view triangulation method is used to calculate the three-dimensional coordinates of the feature points in each set of deformed images, obtaining the three-dimensional displacement field. This avoids errors caused by data stitching in traditional methods and improves the accuracy of three-dimensional strain calculation.

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments obtained. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0020] This invention employs a multi-view (n≥3 cameras) array optical measurement system to achieve high-precision measurement of the three-dimensional deformation of objects. For this system, an array calibration algorithm and an image fusion algorithm were developed. The measurement system mainly consists of system tooling, an imaging unit, and an image analysis and processing system.

[0021] 1. System tooling The system fixture provides a stable installation and positioning reference for the entire measurement system. It includes a multi-camera array bracket, a workpiece positioning fixture, and an optical calibration plate fixing assembly, which can ensure the relative positional accuracy between the camera, the workpiece, and the calibration plate to adapt to the measurement needs of workpieces of different sizes.

[0022] During testing, the system fixture ensures that the spatial position of the imaging system remains stable. The camera bracket is constructed from vertical and horizontal aluminum profiles. The horizontal aluminum profile can be adjusted along the extension direction of the vertical aluminum profile to adapt to the height measurement requirements of different test objects. The horizontal and vertical aluminum profiles are connected by a detachable rigid connection, which ensures the strength of the bracket structure and facilitates quick assembly and disassembly and adjustment according to the test scenario.

[0023] 2. Imaging Unit The imaging unit of this system is a multi-view synchronous acquisition module. Its core function is to acquire high-resolution, high-frame-rate multi-view image data of the workpiece during deformation, providing a reliable data source for subsequent three-dimensional deformation calculation.

[0024] The hardware configuration of this unit is based on multiple high-speed cameras with a resolution of 1920×1080@3000 fps, which meets the project's requirement that the image acquisition frame rate be no less than 1000Hz. After verification by the image analysis algorithm, this resolution can support a strain measurement accuracy requirement of no less than 400με.

[0025] In terms of layout design, the imaging unit needs to ensure that multiple cameras form a common field of view for the measured area. Based on the spatial dimensions of the test site, the maximum values ​​of the object distance *u* and the camera spacing *d* are determined, and then the maximum angle between the cameras is derived using a formula: ; In the formula, t represents the characteristic dimension of the measured part in the camera imaging direction. Within the allowable range of the site, the included angle between adjacent cameras should be controlled at around 30° to ensure the accuracy of three-dimensional deformation measurement.

[0026] The selection of a camera lens needs to be determined by comprehensively measuring the field of view H, object distance u, and camera target surface size h. The specific selection formula is as follows: ; This ensures that the lens's field of view completely covers the area being measured, while avoiding interference from image distortion with the measurement results.

[0027] 3. Image Analysis and Processing System Based on the image data processing requirements acquired by the camera, this system breaks down the image analysis software into five major functional modules: camera imaging parameter calibration module, feature point matching module, feature point 3D coordinate reconstruction module, 3D displacement and strain calculation module, and result visualization module.

[0028] The camera imaging parameter calibration module solves for the intrinsic parameters of multiple cameras and the extrinsic parameters between cameras, achieving a unified coordinate system for the multiple cameras and independently calibrating the camera imaging parameters from each viewpoint. This module integrates Zhang Zhengyou's checkerboard calibration method and the speckle self-calibration method. Zhang Zhengyou's checkerboard calibration method calculates the parameters by simultaneously acquiring checkerboard images of approximately 21 poses from three cameras. The speckle self-calibration method acquires speckle images from multiple views and solves the problem using multi-view geometric constraints. In the actual calibration process, the intermediate camera is used as a reference to calibrate the relative imaging parameters of the other cameras, ultimately outputting the camera parameters in a unified coordinate system and the independent calibration results for each viewpoint.

[0029] The feature point matching module accurately matches corresponding feature points from different viewpoints and at different times, outputting standardized feature point matching results. The module takes speckle (sequence) images captured by three cameras as input. It first performs initial cross-viewpoint feature point matching using an inter-viewpoint matching algorithm, then optimizes this by combining it with a same-viewpoint matching algorithm. Two-dimensional digital image correlation algorithms and SIFT algorithms are used in conjunction to solve the challenges of feature point matching within and between viewpoints, ultimately outputting corresponding feature point matching results from different viewpoints at various times.

[0030] The feature point 3D coordinate reconstruction module converts feature point information in a 2D image into 3D spatial coordinates, obtaining the 3D position data of the feature points. The module takes camera imaging parameter calibration results and feature point matching results as inputs. Based on the principle of triangulation, and combining the calibrated camera intrinsic and extrinsic parameters with the correspondence of matched feature points, it reconstructs the 3D coordinates of the feature points through spatial geometric calculations and outputs the 3D coordinate data of the feature points.

[0031] The 3D displacement and strain field calculation module calculates the 3D displacement and strain of feature points in the test area with the initial time as a reference, generating 3D displacement and strain field data. The module takes the 3D coordinates of the feature points as input. First, it fits a 3D plane around the target point using the least squares method and solves the plane equations, establishing a local coordinate system with the target point as the origin. The coordinates in the world coordinate system are then transformed to the local coordinate system using Euclidean transformation. Time differentiation is performed on the local coordinates to obtain the displacement data. Finally, three displacement field functions, Ue, Ve, and We, are fitted using a quadratic surface. ; in, These are the coefficients of the displacement field function. The strain tensor is calculated using the displacement field function as follows: ; The three-dimensional displacement and strain field data can be obtained through the above process.

[0032] The results visualization module converts numerical 3D coordinates, displacement, and strain data into intuitive visualizations, facilitating the observation of 3D deformation and strain distribution of the structure. The module takes 3D coordinates of feature points, 3D displacement, and 3D strain data as input. Through color mapping and 3D modeling, the data is converted into 3D cloud maps. The color gradient of the cloud map intuitively reflects the distribution characteristics of structural deformation and strain, and the final output is a visualization cloud map of the 3D displacement and strain fields of the measured area.

[0033] 1. Feature point extraction and matching First, a multi-view camera array is used to synchronously acquire speckle feature images of the surface of the structure under test. The local geometric features gj and appearance description dj of each image feature point are extracted using the Scale Invariant Feature Transform (SIFT) algorithm. Then, by matching feature points from different images, the Random Sample Consensus (RANSAC) algorithm is used for geometric verification to eliminate incorrect matching point pairs caused by perspective transformations and nonlinear transformations. Finally, the fundamental matrix F is solved based on the epipolar constraint formula, as shown below. The solution with the minimum bidirectional reprojection error is selected as the optimal solution, and the remaining matching point pairs are checked using the epipolar constraint threshold. .

[0034] 2. Solving for initial values ​​of imaging parameters First, the images are sorted according to the number of feature point matches. The image with the most matching pairs is selected for initialization. Combining the estimated lens focal length and the initial value of the principal point position of the image center, the initial value of the distortion coefficient is set to 0. The essential matrix E is solved by the correspondence between the fundamental matrix F and the corresponding intrinsic parameter matrices K1 and K2, as shown in the following formula: ; Then, singular value decomposition is performed on E to obtain candidate solutions for camera pose. The correct rotation matrix R21 and translation vector t2 are selected by judging the positive depth of the 3D point P in the camera coordinate system. If the scene points are coplanar or the camera is in pure rotational motion, the camera extrinsic parameters are recovered by the identity matrix H.

[0035] 3. Imaging parameter optimization and 3D reconstruction First, at least one additional camera is introduced. The extrinsic parameters of the new camera are estimated using the perspective n-point method (PnP) combined with the coordinates of the reconstructed 3D points. Then, triangulation is performed on the new camera, and the positive depth of the 3D points is verified by the triangulation angle α, thus completing the reconstruction of the new 3D points. Simultaneously, a target function is constructed using bundle adjustment (BA) combined with the Levenberg-Marquardt method, as shown below: ; Based on the objective function, the camera intrinsic parameters [f, cx, cy, k1, k2], extrinsic parameters, and 3D point coordinates xj are optimized. After each optimization, large error observation results are removed by extrinsic point filtering. Finally, known size / control points are introduced to supplement the self-calibrated absolute physical scale information to complete the 3D reconstruction.

[0036] 4. Supplementing Scale Information By introducing standard objects or ground control points of known size, the self-calibration and 3D reconstruction results are scaled to obtain a 3D reconstruction model with absolute physical scale.

[0037] The invention has been proven feasible through experiments, simulations, and usage. It can be applied to the batch processing of radiation imaging features of solid rocket motor exhaust plumes. The 3D surface images of radiation at different time periods are intuitive and easy to understand. Furthermore, outliers can be effectively removed through sliding window and thresholding methods. The entire invention is feasible.

[0038] 1. Static Experiment Table 1. Comparison of equivalent focal length f calculated using the self-calibration method with different numbers of cameras (Zhang Zhengyou method) (Unit: pixels)

[0039] Table 1 shows a comparison of the equivalent focal length *f* obtained using Zhang Zhengyou's calibration method and the self-calibration method using different numbers of cameras. As can be seen from the table, the equivalent focal length *f* gradually increases with the number of cameras involved in the self-calibration. When the number of cameras exceeds five, the equivalent focal length *f* gradually stabilizes at approximately 1210 pixels. In contrast, the equivalent focal length *f* obtained using Zhang Zhengyou's calibration method for all cameras is generally around 1200 pixels.

[0040] Table 2. Comparison of principal point cx obtained by the self-calibration method using different numbers of cameras with Zhang Zhengyou's method. (Unit: pixels)

[0041] Table 3. Comparison of principal point cy obtained by the self-calibration method using different numbers of cameras with Zhang Zhengyou's method. (Unit: pixels)

[0042] Tables 2 and 3 present comparative data on the principal point coordinates cx and cy obtained using Zhang Zhengyou's calibration method and self-calibration with different numbers of cameras. Analysis of these data shows that as the number of cameras involved in the self-calibration increases, the values ​​of cx and cy gradually decrease and tend to stabilize. The results obtained using a self-calibration method with four cameras are closest to those obtained using Zhang Zhengyou's method.

[0043] Table 4. Comparison of radial distortion coefficient k1 obtained by the self-calibration method using different numbers of cameras with Zhang Zhengyou's method.

[0044] Table 5. Comparison of radial distortion coefficient k2 obtained by the self-calibration method using different numbers of cameras with Zhang Zhengyou's method.

[0045] Tables 4 and 5 show the comparison of distortion coefficients k1 and k2 obtained by self-calibration using Zhang Zhengyou's calibration method with different numbers of cameras. Unlike the two intrinsic parameters mentioned above, which vary with the number of cameras, the distortion coefficient calibration results are not sensitive to the number of cameras.

[0046] Table 6. Comparison of results from Zhang Zhengyou's method regarding external parameters between pairs of cameras obtained through self-calibration.

[0047] Table 6 compares the results obtained from self-calibration using external parameters and Zhang Zhengyou's method, including the rotation angles (turn angles) between each pair of cameras and the translation vectors between cameras. The table shows that the difference between the rotation angle (yaw angle) along the y-axis obtained by the self-calibration method and Zhang Zhengyou's method is more significant relative to the x-axis and z-axis. This may be due to the large difference in yaw angles between cameras under experimental conditions (the directions of the xyz axes in the camera coordinate system). Furthermore, the translational deviation along the z-axis is also quite prominent, possibly because measuring the distance from the surface is more difficult, leading to increased data uncertainty. These observations reveal that the estimation of specific degrees of freedom may be influenced by more factors in different calibration methods.

[0048] 2. Dynamic Experiment Statistical methods were used to investigate how to efficiently utilize self-calibration data at each time point during the measurement of object deformation in order to optimize and ultimately determine the intrinsic and extrinsic parameters of the camera. In the experiment, six cameras captured 50 images of the same object under different deformation states. During the self-calibration process, five intrinsic parameters were extracted from each camera: focal length (f), principal point coordinates (cx, cy), and two radial distortion coefficients (k1, k2).

[0049] Figure 5This paper presents histogram statistics of 50 sets of data on the equivalent focal length *f* of a certain camera, along with statistical analysis of the mean and standard deviation. The data shows characteristics of a normal distribution, but also exhibits random variations over time. Analysis reveals the following reasons: the camera's imaging sensor and related electronic components may generate electronic noise, which can interfere with focal length measurement. Furthermore, changes in lighting conditions in the shooting environment may affect the camera's autofocus system, indirectly influencing the measured focal length. Additionally, the camera's internal software may introduce errors when processing image data, and these errors may vary with time and shooting conditions. Therefore, the combined effect of these factors may cause the focal length measurement to exhibit random variations over time.

[0050] Figure 6 Histogram statistics of 50 sets of data for the camera principal point coordinates cx and cy, along with corresponding mean and standard deviation analyses, are presented. Similar to the analysis results for the equivalent focal length f, these data exhibit a normal distribution with random fluctuations over time. However, compared to the equivalent focal length f, the variation in the principal point coordinates is much smaller. This is mainly attributed to the highly stable design of modern camera imaging sensors; in the absence of significant external disturbances, the position of the principal point remains relatively fixed with minimal change.

[0051] Figure 7 Histogram statistics of 50 sets of data for camera distortion parameters k1 and k2 are presented, along with corresponding mean and standard deviation analyses. Similar to previous results, these data also exhibit a normal distribution and show random fluctuations over time.

[0052] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations to the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A strain vibration field measurement system based on optical multi-view three-dimensional imaging, characterized in that, Image data processing requirements based on camera acquisition include: The camera imaging parameter calibration module uses the intermediate camera as a reference to calibrate the relative imaging parameters of the other cameras, and finally outputs the camera parameters in a unified coordinate system and the independent calibration results of each vision. The feature point matching module completes accurate matching of the same feature points under different visions and time periods, and outputs standardized feature point matching results. The feature point 3D coordinate reconstruction module converts feature point information in a 2D image into 3D spatial coordinates to obtain the 3D position data of the feature points. The three-dimensional displacement and strain calculation module realizes the calculation of three-dimensional displacement and strain of feature points in the area to be measured with the initial time as a reference, and generates three-dimensional displacement field and strain field data. The results visualization module converts numerical three-dimensional coordinates, displacement, and strain data into intuitive visual graphics, facilitating the observation of three-dimensional deformation and strain distribution of the structure.

2. The strain vibration field measurement system based on optical multi-view three-dimensional imaging according to claim 1, characterized in that, The camera imaging parameter calibration module specifically integrates Zhang Zhengyou's checkerboard calibration method and speckle self-calibration method. Zhang Zhengyou's checkerboard calibration method completes the calculation by simultaneously acquiring checkerboard images of about 21 poses using 3 cameras. The speckle self-calibration method solves the problem by acquiring speckle images from multiple perspectives and using multi-view geometric constraints.

3. The strain vibration field measurement system based on optical multi-view three-dimensional imaging according to claim 1, characterized in that, The specific operation of the feature point matching module is as follows: the input is speckle (sequence) images captured by 3 cameras. First, the cross-view feature point matching is completed through the inter-view matching algorithm, and then the same-view matching algorithm is combined for optimization. The two-dimensional digital image correlation algorithm and SIFT algorithm are used to process the feature points in a coordinated manner to solve the problem of feature point matching between the same view and the view. Finally, the matching results of the same-name feature points at different views at each time are output.

4. The strain vibration field measurement system based on optical multi-view three-dimensional imaging according to claim 1, characterized in that, The feature point 3D coordinate reconstruction module is specifically designed as follows: the input of the module is the camera imaging parameter calibration result and the feature point matching result. Based on the principle of triangulation, combined with the calibrated camera intrinsic and extrinsic parameters and the correspondence of the matched feature points, the module completes the reconstruction of the feature point 3D coordinates through spatial geometric calculations and outputs the feature point 3D coordinate data.

5. The strain vibration field measurement system based on optical multi-view three-dimensional imaging according to claim 1, characterized in that, The specific operation of the three-dimensional displacement and strain field calculation module is as follows: The input of this module is the three-dimensional coordinates of the feature point. First, the three-dimensional plane is fitted using the least squares method around the target point, and the plane equation is solved. A local coordinate system is established with the target point as the origin. The coordinates in the world coordinate system are transformed to the local coordinate system through Euclidean transformation. The displacement data is obtained by performing time differentiation on the local coordinates, and then fitted with a quadratic surface. Ue, Ve, We Three displacement field functions: ； in, x e , y e , z e These are the three-dimensional coordinate components of the feature points in the local coordinate system. These are the coefficients of the displacement field function. The strain tensor is calculated using the displacement field function as follows: ； in, ε xx , ε yy , ε zz For normal strain components, ε xy , ε yz , ε zx These are the shear strain components. The three-dimensional displacement and strain field data can be obtained through the above process.

6. The strain vibration field measurement system based on optical multi-view three-dimensional imaging according to claim 1, characterized in that, The result visualization module specifically works as follows: the input is the three-dimensional coordinates, three-dimensional displacement, and three-dimensional strain data of the feature points. The data is converted into a three-dimensional cloud map through color mapping, three-dimensional modeling, and other methods. The color gradient of the cloud map intuitively reflects the distribution characteristics of structural deformation and strain. Finally, the three-dimensional displacement field and strain field visualization cloud map of the area to be measured are output.

7. The measurement method of the strain vibration field measurement system based on optical multi-view three-dimensional imaging according to any one of claims 1-6, characterized in that, The specific steps are as follows: S1. Feature Point Extraction and Matching First, a multi-view camera array is controlled to synchronously acquire speckle feature images of the surface of the structure under test. Then, the local geometric features of each image feature point are extracted using the Scale Invariant Feature Transform (SIFT) algorithm. g j Compared with appearance description d j ;pass d j Initial matching is performed based on similarity, and then through... g j A second screening is performed based on geometric consistency, followed by geometric verification using the Random Sample Consensus (RANSAC) algorithm to eliminate incorrect matching point pairs caused by perspective transformations and nonlinear transformations; finally, the epipolar constraint formula is applied. Solve for the fundamental matrix F; select the solution with the minimum bidirectional reprojection error as the optimal solution, and verify the remaining matching point pairs by applying an epipolar constraint threshold. P 1 Let be the homogeneous coordinates of the feature points on the imaging plane of the first camera. P 2 These are the homogeneous coordinates of the corresponding feature points on the imaging plane of the second camera; S2. Solving for initial values ​​of imaging parameters First, the images are sorted by the number of matching feature points. The image with the most matching pairs is selected for initialization. Combining the estimated lens focal length and the initial value of the principal point position of the image center, the initial value of the distortion coefficients is set to 0. This is then determined by the fundamental matrix F and the corresponding intrinsic parameter matrix. K 1 and K 2 The essential matrix E is obtained by solving the correspondence relationship, and the relationship is shown below: ； in, t ^ 21 Let be the rotation matrix of camera 2 relative to camera 1. K 1 Let be the intrinsic parameter matrix of the first camera. K 2 Let E be the intrinsic parameter matrix of the second camera. Then, singular value decomposition is performed on E to obtain candidate solutions for the camera pose. The correct rotation matrix is ​​selected by judging the positive depth of the 3D point P in the camera coordinate system. R 21 With translation vector t 2 If the scene points are coplanar or the camera is in pure rotational motion, the camera extrinsic parameters are recovered using the identity matrix H. S3. Imaging Parameter Optimization and 3D Reconstruction First, at least one additional camera is introduced. The extrinsic parameters of the new camera are estimated using the perspective n-point method (PnP) combined with the coordinates of the reconstructed 3D points. Then, triangulation is performed on the new camera, and the positive depth of the 3D points is verified by the triangulation angle α, thus completing the reconstruction of the new 3D points. Simultaneously, a target function is constructed using bundle adjustment (BA) combined with the Levenberg-Marquardt method, as shown below: ； in, P c The middle part represents the set of camera parameters to be optimized, including intrinsic parameters. f, cx, cy, k1, k2 ]( f For camera focal length, cx , cy The coordinates of the principal point on the imaging plane. k1 , k2 (radial distortion coefficients) and extrinsic parameters (rotation matrix and translation vector). x j Let j be the spatial coordinates of the j-th 3D point. π(P c ,x j ) The projection function refers to the function that projects three-dimensional points. xj Through camera parameters P c The pixel coordinates obtained after projection onto the imaging plane are calculated. Based on this objective function, the camera's intrinsic and extrinsic parameters and 3D point coordinates are optimized. After each optimization, large error observation results are removed by extrinsic filtering. Finally, known size / control points are introduced to supplement the self-calibrated absolute physical scale information to complete the 3D reconstruction. S4. Scale Information Supplement By introducing standard objects or ground control points of known size, the self-calibration and 3D reconstruction results are scaled to obtain a 3D reconstruction model with absolute physical scale.

Images