Three-dimensional deformation information acquisition method and system for large deformation measurement of material

Abstract

The invention provides a three-dimensional deformation information acquisition method and system for large deformation measurement of a material, and relates to the technical field of material manufacturing and testing, and the method comprises the steps: carrying out the continuous shooting of the deformation state of the surface of a to-be-measured material through a binocular camera, so as to obtain a to-be-measured speckle image sequence of the surface of the to-be-measured material; taking a first to-be-measured speckle image in the to-be-measured speckle image sequence as a first reference image, taking a second to-be-measured speckle image in the to-be-measured speckle image sequence as a first deformed image, and calculating correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points; dividing the to-be-detected speckle image sequence into a plurality of to-be-detected speckle image groups based on the plurality of reference image update points, taking a first to-be-detected speckle image in each to-be-detected speckle image group as a second reference image, and performing deformation calculation on each to-be-detected speckle image group to obtain three-dimensional deformation information of each to-be-detected speckle image group, and integrating all the three-dimensional deformation information to obtain the three-dimensional deformation information at each moment. According to the invention, the accuracy of acquiring the three-dimensional deformation information can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of material manufacturing and testing, and in particular to a method and system for acquiring three-dimensional deformation information for large deformation measurement of materials. Background Art

[0002] As an emerging non-contact deformation measurement method, digital image correlation method can measure the full-field deformation of the surface of the measured object only by relying on the camera itself. Compared with other methods, it has the advantages of low equipment cost, simple operation, and global measurement, making it widely used in aerospace, shipbuilding, automobile and other fields.

[0003] The Chinese patent with publication number CN118347423A discloses a method and device for obtaining three-dimensional deformation field information of a component, and the method for obtaining three-dimensional deformation field information of the component includes: obtaining deformation process data of a component to be measured carrying a speckle pattern from different viewing angles to extract local deformation data corresponding to each viewing angle, and splicing them to obtain an initial splicing result; judging whether there is an overlapping part in the initial splicing result; if there is an overlapping part, iteratively optimizing the initial splicing result until the optimal rigid body transformation matrix between each local deformation data is found, and taking the currently optimized splicing result as the precision splicing result; performing continuity verification on the precision splicing result; homogenizing the precision splicing result that has passed the verification to obtain the three-dimensional deformation field information of the component to be measured. However, the above scheme cannot avoid the problem of discontinuous deformation in the splicing part, which easily leads to deformation measurement errors in the precision splicing result. Therefore, it is very necessary to provide a method and system for obtaining three-dimensional deformation information for large deformation measurement of materials to improve the accuracy of collecting three-dimensional deformation information. Summary of the invention

[0004] In view of this, the present invention proposes a method and system for acquiring three-dimensional deformation information for large deformation measurement of materials. The speckle image sequence to be measured is divided into multiple image groups, and an updated reference image is used for analysis in each image group, which can effectively deal with the image decorrelation problem caused by excessive deformation, so as to improve the accuracy of collecting three-dimensional deformation information.

[0005] The present invention provides a method for obtaining three-dimensional deformation information for large deformation measurement of materials, the method comprising:

[0006] Continuously photographing the deformation state of the surface of the material to be tested by a binocular camera to obtain a speckle image sequence of the surface of the material to be tested, wherein the surface of the material to be tested has a pre-prepared speckle pattern;

[0007] taking a first speckle image to be measured in the sequence of speckle images to be measured as a first reference image, taking a second speckle image to be measured in the sequence of speckle images to be measured as a first deformed image, and calculating a correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points, wherein the first speckle image to be measured represents the first speckle image to be measured in the sequence of speckle images to be measured, and the second speckle image to be measured represents any speckle image to be measured in the sequence of speckle images to be measured except the first speckle image to be measured;

[0008] The speckle image sequence to be measured is divided into a plurality of speckle image groups to be measured based on a plurality of reference image update points, the first speckle image to be measured in each speckle image group to be measured is used as the second reference image, deformation calculation is performed on each speckle image group to be measured, three-dimensional deformation information of each speckle image group to be measured is obtained, and all the three-dimensional deformation information is integrated to obtain the three-dimensional deformation information at each moment.

[0009] On the basis of the above technical solution, preferably, the calculating of the weighted correlation coefficient between the reference image and the deformed image to obtain a plurality of reference image update points specifically includes:

[0010] determining whether a weighted correlation coefficient between the first reference image and the first deformed image is within a preset threshold range;

[0011] If the weighted correlation coefficient between the first reference image and the first deformed image is not within a preset threshold range, using an intermediate image between the first reference image and the first deformed image as a transition deformed image;

[0012] The weighted correlation coefficient between the first reference image and the transition deformed image is repeatedly calculated until the weighted correlation coefficient is within a preset threshold range, the transition deformed image is used as a transition reference image, and deformation calculation is performed on the speckle image to be tested in the speckle image sequence to be tested, so as to obtain a plurality of reference image update points and a deformed speckle image.

[0013] On the basis of the above technical solution, preferably, the expression of the weighted correlation coefficient is:

[0014] C n =aC1+bC2+cC3+dC4

[0015] Among them, C nrepresents the total weighted correlation coefficient of the first reference image and the first deformed image, C1 represents the weighted correlation coefficient located in the first preset threshold interval, C2 represents the weighted correlation coefficient located in the second preset threshold interval, C3 represents the weighted correlation coefficient located in the third preset threshold interval, C4 represents the weighted correlation coefficient located in the fourth preset threshold interval, a represents the weight coefficient corresponding to the weighted correlation coefficient located in the first preset threshold interval, b represents the weight coefficient corresponding to the weighted correlation coefficient located in the second preset threshold interval, c represents the weight coefficient corresponding to the weighted correlation coefficient located in the third preset threshold interval, and d represents the weight coefficient corresponding to the weighted correlation coefficient located in the fourth preset threshold interval.

[0016] More preferably, the method further comprises:

[0017] Correct and match the left camera reference image and the right camera reference image of the binocular camera at the same time by epipolar geometry and parallax constraints, and obtain left and right image matching points corresponding to the left camera reference image and the right camera reference image;

[0018] If the sub-region correlation coefficients of the left camera reference image and the right camera reference image are greater than the preset sub-region correlation coefficients, the left and right image matching points are substituted into the inverse synthetic Gauss-Newton method for iterative optimization to obtain sub-pixel matching parameters.

[0019] More preferably, the method further comprises:

[0020] performing image feature point extraction and matching operations on the speckle image to be measured and the deformed speckle image based on the AKAZE image feature algorithm to obtain left and right camera matching values, deformation matching values, and left and right deformation matching points corresponding to the image feature points;

[0021] If the camera correlation coefficients of the right camera reference image and the right camera target image at different times are greater than the preset camera correlation coefficient, the left and right camera matching values ​​and the deformation matching values ​​are summed and rounded as the iteration initial value, and the iteration initial value is substituted into the inverse synthetic Gauss-Newton algorithm for iterative optimization to obtain the left and right image matching points corresponding to the deformed speckle image.

[0022] More preferably, before continuously photographing the deformation state of the surface of the material to be measured by the binocular camera, the method further includes:

[0023] The internal and external parameters of the binocular camera are obtained through Zhang Zhengyou calibration method. The internal parameters include the principal point coordinates, focal length and distortion parameters of the left and right cameras, and the external parameters include the rotation matrix and translation matrix between the left and right cameras.

[0024] More preferably, the area where the speckle pattern is located constitutes a speckle area, the speckle pattern is evenly distributed in the speckle area, and there is a mark in the speckle area.

[0025] In a second aspect of the present application, a three-dimensional deformation information acquisition system for large deformation measurement of materials is provided, wherein the three-dimensional deformation information acquisition system comprises a speckle information acquisition module, a reference image deformation module and a three-dimensional deformation measurement module, wherein:

[0026] The speckle information acquisition module is used to continuously shoot the deformation state of the surface of the material to be tested through a binocular camera to obtain a speckle image sequence to be tested on the surface of the material to be tested, wherein the surface of the material to be tested has a pre-prepared speckle pattern;

[0027] The reference image deformation module is used to use a first speckle image to be measured in the sequence of speckle images to be measured as a first reference image, and a second speckle image to be measured in the sequence of speckle images to be measured as a first deformed image, and calculate the correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points, wherein the first speckle image to be measured represents the first speckle image to be measured in the sequence of speckle images to be measured, and the second speckle image to be measured represents any speckle image to be measured in the sequence of speckle images to be measured except the first speckle image to be measured;

[0028] The three-dimensional deformation measurement module is used for dividing the speckle image sequence to be measured into a plurality of speckle image groups to be measured based on a plurality of reference image update points, taking the first speckle image to be measured in each speckle image group to be measured as the second reference image, performing deformation calculation on each speckle image group to be measured, obtaining three-dimensional deformation information of each speckle image group to be measured, and integrating all the three-dimensional deformation information to obtain the three-dimensional deformation information at each moment.

[0029] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0030] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of a method for acquiring three-dimensional deformation information for large deformation measurement of materials.

[0031] The method and system for obtaining three-dimensional deformation information for large deformation measurement of materials provided by the present invention have the following beneficial effects compared with the prior art:

[0032] (1) The binocular camera provides accurate three-dimensional spatial information, and the prefabricated speckle pattern enhances the surface feature recognition capability, which can achieve deformation measurement with micron-level accuracy. At the same time, the speckle image sequence to be measured is divided into multiple image groups, and the updated reference image is used for analysis in each image group, which can effectively deal with the image decorrelation problem caused by excessive deformation, thereby expanding the measurement range of the digital image correlation method, which is suitable for deformation measurement in a wider range and also improves the accuracy of collecting three-dimensional deformation information. By dynamically updating the reference image during the deformation process, the problem of reduced correlation and decreased matching accuracy caused by excessive changes in the speckle pattern in the traditional digital image correlation method when the deformation is large is overcome, ensuring that high-precision deformation information can still be obtained under deformation conditions. By integrating the three-dimensional deformation information of each image group, continuous three-dimensional deformation information of the entire deformation process can be obtained, which fully reflects the deformation behavior of the material during the deformation process. The segmented processing and reference image updating method are adopted to effectively avoid the error accumulation problem that may occur in long-term deformation calculation, and ensure the reliability of the obtained three-dimensional deformation information.

[0033] (2) Through epipolar geometry and parallax constraints, the accuracy of left and right image matching is greatly improved, and the possibility of false matching is reduced. In addition, epipolar constraints and parallax constraints significantly reduce the matching search space and improve the computational efficiency of the algorithm. Through iterative optimization using the inverse synthetic Gauss-Newton method, sub-pixel matching accuracy is achieved, which greatly improves the accuracy of 3D reconstruction and depth estimation. The sub-region correlation coefficient threshold is used as the judgment criterion to ensure that only high-quality matching points are further optimized, thereby improving the reliability of the results.

[0034] (3) By using the AKAZE algorithm for feature extraction, the key feature points in the image can be identified quickly and accurately, which is suitable for the complex texture of the speckle image. The left and right camera matching values ​​and the deformation matching values ​​are obtained at the same time, realizing the comprehensive matching of the spatial and temporal dimensions and improving the overall matching accuracy. By comparing the correlation coefficients of the right camera images at different times, it is ensured that only high-quality matches are further processed, thereby enhancing the robustness of the algorithm. The sum of the left and right camera matching values ​​and the deformation matching values ​​is taken as the initial value of the iteration, which provides a good starting point for subsequent optimization and accelerates the convergence process. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1A schematic flow chart of a method for acquiring three-dimensional deformation information for large deformation measurement of materials provided by the present invention;

[0037] Figure 2 A schematic diagram of the structure of the full-field three-dimensional deformation measurement device provided by the present invention;

[0038] Figure 3 This is a schematic structural diagram of an electronic device provided by the present invention.

[0039] Explanation of reference numerals: 1. 3D deformation information acquisition system; 11. Speckle information acquisition module; 12. Reference image deformation module; 13. 3D deformation measurement module; 2. Electronic device; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. DETAILED DESCRIPTION

[0040] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention discloses a method for obtaining three-dimensional deformation information for large deformation measurement of materials, referring to Figure 1 The method includes steps S1 to S3.

[0042] Step S1, continuously photographing the deformation state of the surface of the material to be tested by a binocular camera to obtain a sequence of speckle images of the surface of the material to be tested, wherein the surface of the material to be tested has a pre-prepared speckle pattern.

[0043] In this step, a random and uniform speckle image is prepared on the surface of the test piece in advance. The specific speckle preparation method can be selected according to the specific experimental conditions, such as screen printing, water transfer stickers, stamps, etc. Then the component is clamped inside the stretching machine, and the component is continuously stretched to produce a large deformation. During the deformation of the component, multiple speckle images are continuously collected through the binocular system. The area where the speckle pattern is located constitutes the speckle area, the speckle pattern is evenly distributed in the speckle area, and there are marks in the speckle area.

[0044] In this embodiment, the internal parameters and external parameters of the binocular camera are obtained by Zhang Zhengyou calibration method, wherein the internal parameters include the principal point coordinates, focal length and distortion parameters of the left and right cameras, and the external parameters include the rotation matrix and translation matrix between the left and right cameras. A circular calibration plate is used as a calibration tool. The circular calibration plate is usually composed of a series of regularly arranged dots, and the positions and sizes of these dots are known. Multiple images of the circular calibration plate are taken from different angles and positions to ensure that the circular calibration plate can be clearly visible in the field of view of the left and right cameras. The captured images are preprocessed, such as denoising, contrast enhancement, etc., and the dot features on the circular calibration plate are detected and extracted. Using the extracted dot features and the known calibration plate parameters, the corresponding relationship between the image coordinate system and the world coordinate system is established, and the camera parameters are solved by the optimization algorithm.

[0045] Step S2: taking a first speckle image to be measured in the speckle image sequence to be measured as a first reference image, taking a second speckle image to be measured in the speckle image sequence to be measured as a first deformed image, and calculating a correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points, wherein the first speckle image to be measured represents the first speckle image to be measured in the speckle image sequence to be measured, and the second speckle image to be measured represents any speckle image to be measured in the speckle image sequence to be measured except the first speckle image to be measured.

[0046] In the step, the first image (first reference image) and the Mth image (first deformed image) in the speckle image sequence to be measured are read, and necessary preprocessing is performed on the two images, such as denoising, contrast enhancement, etc. A series of sub-areas are selected on the first reference image as correlation windows, and multiple correlation windows overlap and cover the entire first reference image area. For each correlation window, the best matching position is searched in the deformed image, and the normalized cross-correlation algorithm (NCC) is used, instead of recording the maximum correlation coefficient value and its corresponding position. The maximum correlation coefficient values ​​of all windows are collected, and the statistical characteristics of these correlation coefficients, such as the mean, median, standard deviation, etc., are calculated.

[0047] By establishing an initial reference datum and realizing dynamic reference updating, the accuracy of image registration and the flexibility of analysis are improved, and continuity analysis is realized, which reduces the cumulative error of measuring speckle image deformation. At the same time, the trend of speckle image changes over time or load can be captured. At the same time, by calculating the correlation coefficient distribution between the reference image and the deformed image, multiple reference image update points are obtained to prepare for the subsequent dynamic update of the reference image.

[0048] This step also includes steps S21 to S23.

[0049] Step S21, determining whether the weighted correlation coefficient between the first reference image and the first deformed image is within a preset threshold range.

[0050] In this step, the expression of the weighted correlation coefficient is:

[0051] C n =aC1+bC2+cC3+dC4

[0052] Among them, C n represents the total weighted correlation coefficient of the first reference image and the first deformed image, C1 represents the weighted correlation coefficient located in the first preset threshold interval, C2 represents the weighted correlation coefficient located in the second preset threshold interval, C3 represents the weighted correlation coefficient located in the third preset threshold interval, C4 represents the weighted correlation coefficient located in the fourth preset threshold interval, C1, C2, C3, C4 represent ZNCC correlation coefficients located in the intervals of 0.6-0.7, 0.7-0.8, 0.8-0.9, and above 0.9, respectively, a represents the weight coefficient corresponding to the weighted correlation coefficient located in the first preset threshold interval, b represents the weight coefficient corresponding to the weighted correlation coefficient located in the second preset threshold interval, c represents the weight coefficient corresponding to the weighted correlation coefficient located in the third preset threshold interval, and d represents the weight coefficient corresponding to the weighted correlation coefficient located in the fourth preset threshold interval.

[0053] Step S22: If the weighted correlation coefficient between the first reference image and the first deformed image is not within the preset threshold range, an intermediate image between the first reference image and the first deformed image is used as a transition deformed image.

[0054] In this step, if the weighted correlation coefficient between the first reference image and the first deformed image is within a preset threshold range, the current frame image is used as a new reference image and the subsequent images are used as a new deformed image sequence for subsequent deformation pre-calculation.

[0055] Step S23, repeatedly calculating the weighted correlation coefficient between the first reference image and the transition deformed image until the weighted correlation coefficient is within a preset threshold range, taking the transition deformed image as the transition reference image, and performing deformation calculation on the speckle image to be tested in the speckle image sequence to be tested, to obtain a plurality of reference image update points and deformed speckle images.

[0056] In one example, a first reference image (usually the first image in a sequence) is set, and a preset threshold interval is defined for determining a weighted correlation coefficient. A speckle image to be measured is selected from the sequence as the initial transition deformation image, and the weighted correlation coefficient between the first reference image and the transition deformation image is calculated. If the calculated weighted correlation coefficient is not within the preset threshold interval, a new transition deformation image (usually an intermediate image between the current transition deformation image and the first reference image) is selected, and the weighted correlation coefficient is recalculated. The above process is repeated until a transition deformation image that meets the conditions is found. Once a transition deformation image that meets the conditions is found, it is set as a new transition reference image, and the sequence number of the transition reference image is recorded. This is a reference image update point. The new transition reference image is used to perform deformation calculation on subsequent speckle images to be measured. The deformation calculation process includes image registration, displacement field calculation, etc. Based on the results of the deformation calculation, corresponding deformed speckle images are generated, and these deformed speckle images reflect the deformation relative to the reference image. Repeat the above steps for the remaining speckle images to be tested. Each time a new transition reference image is found, its serial number is recorded as the reference image update point. Finally, all recorded reference image update point serial numbers and the generated deformed speckle image sequence are output.

[0057] Step S3: dividing the speckle image sequence to be measured into a plurality of speckle image groups to be measured based on a plurality of reference image update points, taking the first speckle image to be measured in each speckle image group to be measured as the second reference image, performing deformation calculation on each speckle image group to be measured, obtaining three-dimensional deformation information of each speckle image group to be measured, and integrating all the three-dimensional deformation information to obtain the three-dimensional deformation information at each moment.

[0058] In this step, since the images in the same speckle image sequence to be measured use the first image in the group as the reference image when performing deformation calculation, and the key parameters such as step size and search range are the same, the current group of deformed images can be matched on the CPU in a multi-threaded computing manner to obtain the three-dimensional deformation information at different times. The multi-threaded computing method allows the deformation analysis of each image in the group to be performed in parallel, greatly improving the overall algorithm efficiency. After the calculation is completed, the update point image, that is, the last image in the group, is used as the new reference image for subsequent deformation calculation. When calculating the deformation information at each moment, the measurement result of each image is obtained by adding two parts, one is the sum of the deformation data of each reference update frame at the previous moment, and the other is the deformation calculation data in the group itself.

[0059] This step also includes steps S31 to S34.

[0060] Step S31, correcting and matching the left camera reference image and the right camera reference image of the binocular camera at the same time by epipolar geometry and parallax constraints, and obtaining left and right image matching points corresponding to the left camera reference image and the right camera reference image.

[0061] In this step, the binocular camera system is calibrated using a calibration plate to obtain the intrinsic parameter matrix and distortion parameters, and the relative position and direction (extrinsic parameter matrix) between the two cameras are calculated. The calibration parameters are used to correct the distortion of the left and right camera images and perform stereo correction (StereoRectification) so that the epipolar lines of the left and right images are parallel and horizontal. Using the epipolar geometry principle, the search range of the points in the left image in the right image is determined, and the search range is limited to the same horizontal line to reduce the matching search space. The minimum and maximum disparity values ​​are set to further limit the search range. Significant feature points are extracted from the left and right images, such as SIFT, SURF or ORB features. Use feature descriptors to perform preliminary matching between the left and right images, and apply epipolar constraints and disparity constraints to filter and optimize the matching results. The preliminary matching results are refined at the sub-pixel level to improve the matching accuracy. Apply the LeftRightConsistencyCheck to further improve the matching reliability and eliminate inconsistent matching points.

[0062] Step S32: If the sub-region correlation coefficients of the left camera reference image and the right camera reference image are greater than the preset sub-region correlation coefficients, the left and right image matching points are substituted into the inverse synthetic Gauss-Newton method for iterative optimization to obtain sub-pixel matching parameters.

[0063] In this step, the accuracy of left and right image matching is greatly improved and the possibility of false matching is reduced through epipolar geometry and parallax constraints. In addition, the epipolar constraints and parallax constraints significantly reduce the matching search space and improve the computational efficiency of the algorithm. Through iterative optimization using the inverse synthetic Gauss-Newton method, sub-pixel matching accuracy is achieved, which greatly improves the accuracy of 3D reconstruction and depth estimation. The sub-region correlation coefficient threshold is used as the judgment criterion to ensure that only high-quality matching points are further optimized, thereby improving the reliability of the results.

[0064] Step S33: extract and match image feature points of the speckle image to be measured and the deformed speckle image based on the AKAZE image feature algorithm to obtain left and right camera matching values, deformation matching values ​​and left and right deformation matching points corresponding to the image feature points.

[0065] In this step, the image feature points are extracted and matched based on the AKAZE image feature method, which mainly includes the following steps: image preprocessing, preprocessing the image to be tested, mainly including filtering and noise reduction, etc., to further improve the accuracy of the algorithm; constructing the scale space by the nonlinear diffusion filtering method, and then using the Hessian matrix to detect the local extreme points of the image, and comparing each target point with the eight adjacent pixels in the same layer and the pixels in the 3×3 window of the upper and lower layers, a total of 26 pixels, to obtain the extreme value, that is, the key point; using the M-LDB descriptor to describe the surrounding area of ​​the feature point, and obtain the feature vector of each key point; finally, the similarity of the feature vectors of different feature points can be calculated by the Hamming distance, so as to obtain the similarity of each feature point, and then obtain the pixel-level matching points of the two images.

[0066] Step S34: if the camera correlation coefficients of the right camera reference image and the right camera target image at different times are greater than the preset camera correlation coefficient, the left and right camera matching values ​​and the deformation matching values ​​are summed and rounded as the initial value of the iteration, and the initial value of the iteration is substituted into the inverse synthetic Gauss-Newton algorithm for iterative optimization to obtain the left and right image matching points corresponding to the deformed speckle image.

[0067] In this step, by using the AKAZE algorithm for feature extraction, the key feature points in the image can be identified quickly and accurately, which is suitable for the complex texture of the speckle image. The left and right camera matching values ​​and the deformation matching values ​​are obtained at the same time, which realizes the comprehensive matching of the spatial and temporal dimensions and improves the accuracy of the overall matching. By comparing the correlation coefficients of the right camera images at different times, it is ensured that only high-quality matches are further processed, which enhances the robustness of the algorithm. The sum of the left and right camera matching values ​​and the deformation matching values ​​is taken as the initial value of the iteration, which provides a good starting point for subsequent optimization and accelerates the convergence process.

[0068] In one example, for the matching between the left and right images at the same time, epipolar geometry and parallax constraints are used to simplify the matching process, and a hierarchical search method is used within the search range to quickly obtain pixel-level left and right matching points, which are then substituted into the reverse synthetic Gauss-Newton algorithm for iterative optimization to obtain sub-pixel matching parameters; the sub-area correlation between the left and right images at the same time is used as a sign of whether the spatial domain matching is successful or not. If the correlation coefficient is greater than a preset threshold (generally set to 0.9, which can be modified according to the specific situation), it means that the matching is successful; for the matching between images at different times before and after deformation, an AKA-based The image feature method of ZE is used to extract and match the image feature points, thereby obtaining the pixel-level matching points, which are then substituted into the reverse synthetic Gauss-Newton method for iterative optimization to obtain the sub-pixel matching parameters; the results of left and right camera matching and deformation matching are rounded and added as the initial value, and substituted into the reverse synthetic Gauss-Newton algorithm for iterative optimization to obtain the deformed right image matching points; the correlation coefficient distribution of the reference right image and the target right image at different times is used as a sign of whether the time series matching is successful or not. If the correlation is greater than the preset threshold, which can generally be 0.8 and can be adjusted according to the specific situation, it means that the matching is successful. The three-dimensional coordinates after deformation are subtracted from the three-dimensional coordinates before deformation to obtain the three-dimensional displacement data of the component surface at that moment.

[0069] The binocular camera provides accurate three-dimensional spatial information, and the prefabricated speckle pattern enhances the surface feature recognition capability, which can achieve deformation measurement with micron-level precision. At the same time, the speckle image sequence to be measured is divided into multiple image groups, and the updated reference image is used for analysis in each image group, which can effectively deal with the image loss correlation problem caused by excessive deformation, thereby expanding the measurement range of the digital image correlation method, which is suitable for deformation measurement in a wider range and improves the accuracy of collecting three-dimensional deformation information. By dynamically updating the reference image during the deformation process, the problem of reduced correlation and reduced matching accuracy caused by excessive changes in the speckle pattern in the traditional digital image correlation method during large deformation is overcome, ensuring that high-precision deformation information can still be obtained under deformation conditions. By integrating the three-dimensional deformation information of each image group, continuous three-dimensional deformation information of the entire deformation process can be obtained, which fully reflects the deformation behavior of the material during the deformation process. The segmented processing and updating of the reference image are adopted to effectively avoid the error accumulation problem that may occur in long-term deformation calculations, and ensure the reliability of the obtained three-dimensional deformation information.

[0070] Based on the above method, the present application embodiment discloses a three-dimensional deformation information acquisition system for large deformation measurement of materials, referring to Figure 2 The three-dimensional deformation information acquisition system 1 includes a speckle information acquisition module 11, a reference image deformation module 12 and a three-dimensional deformation measurement module 13, wherein:

[0071] The speckle information acquisition module (11) is used to continuously shoot the deformation state of the surface of the material to be tested through a binocular camera to obtain a speckle image sequence to be tested on the surface of the material to be tested, wherein the surface of the material to be tested has a pre-prepared speckle pattern;

[0072] The reference image deformation module (12) is used to use a first speckle image to be measured in the sequence of speckle images to be measured as a first reference image, and a second speckle image to be measured in the sequence of speckle images to be measured as a first deformed image, and calculate the correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points, wherein the first speckle image to be measured represents the first speckle image to be measured in the sequence of speckle images to be measured, and the second speckle image to be measured represents any speckle image to be measured in the sequence of speckle images to be measured except the first speckle image to be measured;

[0073] The three-dimensional deformation measurement module (13) is used to divide the speckle image sequence to be measured into a plurality of speckle image groups to be measured based on a plurality of reference image update points, use the first speckle image to be measured in each speckle image group to be measured as the second reference image, perform deformation calculation on each speckle image group to be measured, obtain three-dimensional deformation information of each speckle image group to be measured, and integrate all the three-dimensional deformation information to obtain the three-dimensional deformation information at each moment.

[0074] In one example, the reference image deformation module (12) is used to determine whether the weighted correlation coefficient between the first reference image and the first deformed image is within a preset threshold interval; if the weighted correlation coefficient between the first reference image and the first deformed image is not within the preset threshold interval, an intermediate image between the first reference image and the first deformed image is used as a transition deformed image; the weighted correlation coefficient between the first reference image and the transition deformed image is repeatedly calculated until the weighted correlation coefficient is within the preset threshold interval, the transition deformed image is used as a transition reference image, and deformation calculation is performed on the speckle image to be tested in the speckle image sequence to obtain a plurality of reference image update points and a deformed speckle image.

[0075] In one example, the expression for the weighted correlation coefficient is:

[0076] C n =aC1+bC2+cC3+dC4

[0077] Among them, C nrepresents the total weighted correlation coefficient of the first reference image and the first deformed image, C1 represents the weighted correlation coefficient located in the first preset threshold interval, C2 represents the weighted correlation coefficient located in the second preset threshold interval, C3 represents the weighted correlation coefficient located in the third preset threshold interval, C4 represents the weighted correlation coefficient located in the fourth preset threshold interval, a represents the weight coefficient corresponding to the weighted correlation coefficient located in the first preset threshold interval, b represents the weight coefficient corresponding to the weighted correlation coefficient located in the second preset threshold interval, c represents the weight coefficient corresponding to the weighted correlation coefficient located in the third preset threshold interval, and d represents the weight coefficient corresponding to the weighted correlation coefficient located in the fourth preset threshold interval.

[0078] In one example, the three-dimensional deformation measurement module 13 is used to correct and match the left camera reference image and the right camera reference image of the binocular camera at the same time through epipolar geometry and parallax constraints, and obtain left and right image matching points corresponding to the left camera reference image and the right camera reference image; if the sub-area correlation coefficient of the left camera reference image and the right camera reference image is greater than the preset sub-area correlation coefficient, the left and right image matching points are substituted into the inverse synthetic Gauss-Newton method for iterative optimization to obtain sub-pixel matching parameters.

[0079] In one example, the three-dimensional deformation measurement module 13 is used to extract and match image feature points of the speckle image to be measured and the deformed speckle image based on the AKAZE image feature algorithm, and obtain left and right camera matching values, deformation matching values, and left and right deformation matching points corresponding to the image feature points; if the camera correlation coefficients of the right camera reference image and the right camera target image at different times are greater than the preset camera correlation coefficient, the left and right camera matching values ​​and the deformation matching values ​​are summed and rounded as the initial values ​​of the iteration, and the initial values ​​of the iteration are substituted into the inverse synthetic Gauss-Newton algorithm for iterative optimization to obtain the left and right image matching points corresponding to the deformed speckle image.

[0080] In one example, the internal parameters and external parameters of the binocular camera are obtained by Zhang Zhengyou calibration method, where the internal parameters include the principal point coordinates, focal length and distortion parameters of the left and right cameras, and the external parameters include the rotation matrix and translation matrix between the left and right cameras.

[0081] In one example, the area where the speckle pattern is located constitutes a speckle area, the speckle pattern is evenly distributed in the speckle area, and there is a mark in the speckle area.

[0082] See also Figure 3 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device 2 may include: at least one processor 21 , at least one network interface 24 , a user interface 23 , a memory 25 , and at least one communication bus 22 .

[0083] The communication bus 22 is used to realize the connection and communication between these components.

[0084] The user interface 23 may include a display screen (Display) and a camera (Camera), and the optional user interface 23 may also include a standard wired interface and a wireless interface.

[0085] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0086] Among them, the processor 21 may include one or more processing cores. The processor 21 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 25, and calling data stored in the memory 25. Optionally, the processor 21 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 21 can integrate one or more combinations of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 21, and it can be implemented separately through a chip.

[0087] Among them, the memory 25 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 25 includes a non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 25 may also be optionally at least one storage device located away from the aforementioned processor 21. As Figure 3 As shown, the memory 25 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a method for acquiring three-dimensional deformation information for large deformation measurement of materials.

[0088] exist Figure 3 In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 21 can be used to call an application stored in the memory 25 for a method for obtaining three-dimensional deformation information for large deformation measurement of materials. When executed by one or more processors, the electronic device executes one or more methods in the above-mentioned embodiments.

[0089] A computer-readable storage medium stores instructions, which, when executed by one or more processors, enable the computer to execute one or more methods in the above-mentioned embodiments.

[0090] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0096] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the technical field that are not recorded in the present disclosure.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for obtaining three-dimensional deformation information for large deformation measurement of materials, characterized in that: The method comprises: Continuously photographing the deformation state of the surface of the material to be tested by a binocular camera to obtain a speckle image sequence of the surface of the material to be tested, wherein the surface of the material to be tested has a pre-prepared speckle pattern; taking a first speckle image to be measured in the sequence of speckle images to be measured as a first reference image, taking a second speckle image to be measured in the sequence of speckle images to be measured as a first deformed image, and calculating a correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points, wherein the first speckle image to be measured represents the first speckle image to be measured in the sequence of speckle images to be measured, and the second speckle image to be measured represents any speckle image to be measured in the sequence of speckle images to be measured except the first speckle image to be measured; The speckle image sequence to be measured is divided into a plurality of speckle image groups to be measured based on a plurality of reference image update points, the first speckle image to be measured in each speckle image group to be measured is used as the second reference image, deformation calculation is performed on each speckle image group to be measured, three-dimensional deformation information of each speckle image group to be measured is obtained, and all the three-dimensional deformation information is integrated to obtain the three-dimensional deformation information at each moment.

2. The method according to claim 1, characterized in that The calculating the weighted correlation coefficient between the reference image and the deformed image to obtain a plurality of reference image update points specifically includes: determining whether a weighted correlation coefficient between the first reference image and the first deformed image is within a preset threshold range; If the weighted correlation coefficient between the first reference image and the first deformed image is not within a preset threshold range, using an intermediate image between the first reference image and the first deformed image as a transition deformed image; The weighted correlation coefficient between the first reference image and the transition deformed image is repeatedly calculated until the weighted correlation coefficient is within a preset threshold range, the transition deformed image is used as a transition reference image, and deformation calculation is performed on the speckle image to be tested in the speckle image sequence to be tested, so as to obtain a plurality of reference image update points and a deformed speckle image.

3. The method according to claim 2, characterized in that The expression of the weighted correlation coefficient is: C n =aC1+bC2+cC3+dC4 Among them, C n represents the total weighted correlation coefficient of the first reference image and the first deformed image, C1 represents the weighted correlation coefficient located in the first preset threshold interval, C2 represents the weighted correlation coefficient located in the second preset threshold interval, C3 represents the weighted correlation coefficient located in the third preset threshold interval, C4 represents the weighted correlation coefficient located in the fourth preset threshold interval, a represents the weight coefficient corresponding to the weighted correlation coefficient located in the first preset threshold interval, b represents the weight coefficient corresponding to the weighted correlation coefficient located in the second preset threshold interval, c represents the weight coefficient corresponding to the weighted correlation coefficient located in the third preset threshold interval, and d represents the weight coefficient corresponding to the weighted correlation coefficient located in the fourth preset threshold interval.

4. The method according to claim 1, characterized in that The method further comprises: Correct and match the left camera reference image and the right camera reference image of the binocular camera at the same time by epipolar geometry and parallax constraints, and obtain left and right image matching points corresponding to the left camera reference image and the right camera reference image; If the sub-region correlation coefficients of the left camera reference image and the right camera reference image are greater than the preset sub-region correlation coefficients, the left and right image matching points are substituted into the inverse synthetic Gauss-Newton method for iterative optimization to obtain sub-pixel matching parameters.

5. The method according to claim 2, characterized in that The method further comprises: performing image feature point extraction and matching operations on the speckle image to be measured and the deformed speckle image based on the AKAZE image feature algorithm to obtain left and right camera matching values, deformation matching values, and left and right deformation matching points corresponding to the image feature points; If the camera correlation coefficients of the right camera reference image and the right camera target image at different times are greater than the preset camera correlation coefficient, the left and right camera matching values ​​and the deformation matching values ​​are summed and rounded as the iteration initial value, and the iteration initial value is substituted into the inverse synthetic Gauss-Newton algorithm for iterative optimization to obtain the left and right image matching points corresponding to the deformed speckle image.

6. The method according to claim 5, characterized in that Before the continuous photographing of the deformation state of the surface of the material to be measured by the binocular camera, the method further includes: The internal and external parameters of the binocular camera are obtained through Zhang Zhengyou calibration method. The internal parameters include the principal point coordinates, focal length and distortion parameters of the left and right cameras, and the external parameters include the rotation matrix and translation matrix between the left and right cameras.

7. The method according to claim 1, characterized in that The area where the speckle pattern is located constitutes a speckle area, the speckle pattern is evenly distributed in the speckle area, and there is a mark in the speckle area.

8. A three-dimensional deformation information acquisition system for large deformation measurement of materials, characterized in that: The three-dimensional deformation information acquisition system (1) comprises a speckle information acquisition module (11), a reference image deformation module (12) and a three-dimensional deformation measurement module (13), wherein: The speckle information acquisition module (11) is used to continuously photograph the deformation state of the surface of the material to be tested by a binocular camera to obtain a speckle image sequence to be tested on the surface of the material to be tested, wherein the surface of the material to be tested has a pre-prepared speckle pattern; The reference image deformation module (12) is used to use a first speckle image to be measured in the sequence of speckle images to be measured as a first reference image, and a second speckle image to be measured in the sequence of speckle images to be measured as a first deformed image, and calculate the correlation coefficient distribution between the reference image and the deformed image to obtain a plurality of reference image update points, wherein the first speckle image to be measured represents the first speckle image to be measured in the sequence of speckle images to be measured, and the second speckle image to be measured represents any one speckle image to be measured in the sequence of speckle images to be measured except the first speckle image to be measured; The three-dimensional deformation measurement module (13) is used to divide the speckle image sequence to be measured into a plurality of speckle image groups to be measured based on a plurality of reference image update points, use the first speckle image to be measured in each speckle image group to be measured as a second reference image, perform deformation calculation on each speckle image group to be measured, obtain three-dimensional deformation information of each speckle image group to be measured, and integrate all the three-dimensional deformation information to obtain the three-dimensional deformation information at each moment.

9. An electronic device, characterized in that: The electronic device (2) comprises a processor (21), a memory (25), a user interface (23) and a network interface (24), wherein the memory (25) is used to store instructions, the user interface (23) and the network interface (24) are used to communicate with other devices, and the processor (21) is used to execute the instructions stored in the memory (25) so that the electronic device (2) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images