3D-DIC camera self-motion compensation method capable of automatically selecting reference features

Abstract

The invention discloses a 3D-DIC camera self-motion compensation method capable of automatically selecting reference features. The 3D-DIC camera self-motion compensation method comprises the following steps: S1, calculating a difference graph of a reference image and a current image and meshing the difference graph; s2, calculating the overall difference graph and the standard deviation sum mean value of each grid, and if a certain grid meets a judgment condition, determining that the grid is a stable region; s3, motion feature points in the stable area are removed; s4, performing feature point matching on the static feature points on the two images shot by the two cameras at the same time, calculating a three-dimensional coordinate, and if the difference between the depth values of the feature points on the two images is smaller than a set value, reserving the feature points; and S5, calculating an affine transformation matrix between the image planes before and after the same camera moves, and transforming the current image to a reference image view angle through reverse transformation. According to the method, the accuracy and robustness of camera self-motion compensation are improved by automatically eliminating the motion feature points in the camera self-motion compensation reference feature points and feature points with too large depth difference from the measured target.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision displacement measurement, and specifically relates to a self-motion compensation method for a 3D-DIC camera that automatically selects reference features. Background Technique

[0002] Two-dimensional digital image correlation technology (2D Digital Image Correlation, 2D-DIC) is a non-contact measurement method in machine vision. 2D-DIC captures and analyzes a sequence of speckle images before and after the deformation of the object to be measured, calculates displacement and deformation, and then extracts key mechanical parameters. Based on the two-dimensional digital image correlation technology, three-dimensional digital image correlation technology (Three-Dimensional Digital Image Correction, 3D-DIC) introduces a binocular stereo vision system, which can obtain the full-field three-dimensional coordinate information of the surface of the object to be measured, so as to analyze its three-dimensional deformation / displacement field. 3D-DIC technology has been widely applied in many fields such as civil engineering, mechanical engineering, and aerospace.

[0003] The three-dimensional digital image correlation method has achieved sub-pixel measurement accuracy in a controlled experimental environment. However, when this method extends from the laboratory environment to complex engineering application scenarios, the self-motion of the camera becomes one of the key factors affecting the image acquisition quality and measurement accuracy. In recent years, domestic and foreign scholars have carried out effective research work on the measurement errors introduced by the self-motion of the camera. However, in practical applications, if moving feature points or feature points with a large depth difference from the target to be measured are included in the reference feature points, the compensation accuracy will be significantly reduced. Camera self-motion compensation is still a technology that needs to be further developed and improved. Summary of the Invention

[0004] In view of the above problems existing in the prior art, the present invention adopts a self-motion compensation method for a 3D-DIC camera that automatically selects reference features. Based on the separation of static and dynamic regions by grid segmentation of the temporal difference image, static feature points are selected for compensation; a feature point screening mechanism with depth constraints is introduced to eliminate feature points with a large depth difference from the background and the target to be measured, improving the accuracy of camera motion compensation during DIC measurement.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A self-motion compensation method for a 3D-DIC camera that automatically selects reference features, comprising the following steps:

[0007] Step S1: Calculate the difference image between the reference image and the current image, and perform grid division on the difference image for regional processing;

[0008] Step S2: Calculate the overall standard deviation and mean of the difference image, as well as the standard deviation and mean of the difference image within each grid. If the standard deviation and mean of the difference image within a certain grid meet the set judgment conditions, then determine that grid as a stable area;

[0009] Step S3: Use the Speeded Up Robust Features (SURF) algorithm to detect the feature points in the stable area and eliminate the moving feature points, leaving the stationary feature points;

[0010] Step S4: Perform feature point matching on the stationary feature points on the two images taken by the two cameras at the same moment, and calculate the three-dimensional coordinates of each matching feature point: If the difference between the average depth of the feature point and the feature point at the previous moment of the target to be measured is less than the set value, then retain this feature point; otherwise, eliminate this feature point to obtain a set of stationary feature points;

[0011] Step S5: Calculate the affine transformation matrix between the camera coordinate systems before and after the movement of the same camera according to the parameters of the stationary feature points obtained in Step S4, and transform the current image to the perspective of the reference image through affine transformation to achieve motion compensation for the camera.

[0012] As a further improvement of the above technical solution:

[0013] In Step S2, the judgment condition for the grid to be a stable area is:

[0014] (μ c - 3σ c < μ w < μ c + 3σ c ) && (σ w < 3σ c )

[0015] In the formula, μ c is the mean gray value of the entire difference image, and σ c is the standard deviation of the gray value of the entire difference image; μ w is the mean gray value of the difference image within the grid currently being judged, and σ w is the standard deviation of the gray value of the difference image within the grid currently being judged; && represents logical "AND". When μ w and σ w meet this condition, then determine the current grid as a stable area.

[0016] In Step S3, the following threshold judgment is performed on the feature points:

[0017]

[0018] Among them, d k is the total displacement of the feature point relative to its position in the reference image, is the average displacement of all the feature points that match within the current grid and the corresponding reference image grid. If D k = 1, then this feature point is a stationary feature point. If D k = 0, then this feature point is a moving feature point.

[0019] In step S3, assume that the initial position of a feature point in the reference image is (x1, y1), and its position in the current image is (x i , y i ). Then:

[0020]

[0021] Step S4 includes the following steps:

[0022] Step S41: Calculate the three-dimensional coordinates (x i , y i , z i ) of the matching stationary feature points on the two images;

[0023] Step S42: Calculate the average coordinate Z s in the Z direction of all the feature points detected on the target to be measured at the previous moment, and calculate its mean value u Z and standard deviation σ Z .

[0024] Step S43: Determine the parameter δ, δ = kσ Z , where k is a constant, and the preferred value range is [1, 3];

[0025] Step S44: Use D = [Z s - δ, Z s + δ] as the condition for screening the Z-direction coordinate z i , and only retain the feature points where z i ∈ D, and eliminate other points.

[0026] Step S4 is called the screening of feature points in a specific depth interval.

[0027] In step S5, according to the coordinates of more than 6 stationary feature points obtained in step S4 in the two camera coordinate systems, the affine transformation matrix is solved using the least squares method.

[0028] The beneficial effects of the present invention are:

[0029] (1) A method for automatically screening reference feature points that are stationary within the camera's field of view and have a depth close to the target to be measured is proposed, which can greatly improve the effect of camera self-motion compensation and the accuracy of three-dimensional displacement measurement in the 3D-DIC method.

[0030] (2) When screening for stationary feature points within the field of view, first screen for stable regions through image difference operations with very low computational complexity and grid processing, and then search for stationary feature points within the stable regions. This can significantly reduce the computational amount and has stronger anti-interference ability.

[0031] (3) The method can significantly reduce the requirements for the usage environment of 3D-DIC deformation / displacement measurement, which helps to further promote machine vision measurement technology in the industry and improve the level of automation and intelligence. Description of the Drawings

[0032] Figure 1 is a schematic diagram of the principle of three-dimensional digital image correlation.

[0033] Figure 2 is a schematic diagram of the process of the present invention.

[0034] Figure 3 is a flowchart of extracting stationary feature points by the frame difference method and the accelerated robust feature algorithm of the present invention.

[0035] Figure 4 is a flowchart of screening feature points in a specific depth interval

[0036] Figure 5 is a schematic diagram of the coordinate transformation relationship of the binocular vision system.

[0037] Figure 6(a) is a comparison of the true displacement value of the target speckle in the X direction and the 3D-DIC displacement measurement value.

[0038] Figure 6(b) is a comparison of the true displacement value of the target speckle in the Y direction and the 3D-DIC displacement measurement value.

[0039] Figure 6(c) is a comparison of the true displacement value of the target speckle in the Z direction and the 3D-DIC displacement measurement value.

[0040] Figure 6(d) is a comparison of the true total displacement value of the target speckle in three directions and the 3D-DIC displacement measurement value.

[0041] Figure 7(a) is a comparison of the true displacement value of the target speckle in the X direction, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after using camera self-motion compensation.

[0042] Figure 7(b) is a comparison of the true displacement value of the target speckle in the Y direction, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after using camera self-motion compensation.

[0043] Figure 7(c) shows the comparison of the true displacement value of the target speckle in the Z direction, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after using camera self-motion compensation.

[0044] Figure 7(d) shows the comparison of the true three-dimensional displacement value of the target speckle, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after using camera self-motion compensation.

[0045] Figure 8(a) shows the comparison of the true displacement value of the target speckle in the X direction, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after screening reference feature points using the method of the present invention and performing camera self-motion compensation.

[0046] Figure 8(b) shows the comparison of the true displacement value of the target speckle in the Y direction, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after screening reference feature points using the method of the present invention and performing camera self-motion compensation.

[0047] Figure 8(c) shows the comparison of the true displacement value of the target speckle in the Z direction, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after screening reference feature points using the method of the present invention and performing camera self-motion compensation.

[0048] Figure 8(d) shows the comparison of the true three-dimensional displacement value of the target speckle, the 3D-DIC displacement measurement value without using camera self-motion compensation, and the 3D-DIC displacement measurement value after screening reference feature points using the method of the present invention and performing camera self-motion compensation. Detailed Embodiments

[0049] The following describes in detail the specific embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.

[0050] For ease of description, spatial relative terms such as "above", "over", "on the upper surface", "upper", etc. can be used here to describe the spatial positional relationship of a device or feature shown in the figure with other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation in addition to the orientation described in the figure for the device. For example, if the device in the attached figure is inverted, the device described as "above other devices or structures" or "over other devices or structures" will then be positioned as "below other devices or structures" or "under other devices or structures". Thus, the exemplary term "above" can include both the orientations of "above" and "below". The device can also be positioned in other different ways (rotated 9 degrees or in other orientations), and corresponding interpretations are made for the spatial relative descriptions used here.

[0051] The 3D-DIC camera self-motion compensation method for automatically selecting reference features first introduces the two-dimensional digital image correlation method. The two-dimensional digital image correlation method is a non-contact measurement technique based on the gray distribution of images, used to analyze the displacement and deformation of the structure to be measured on a two-dimensional plane. Its principle is to perform gray correlation analysis on two images of the structure to be measured before and after deformation to determine the displacement and deformation of the corresponding region. Image matching is a key step in 2D-DIC. Displacement and strain data are obtained by matching the image features of the structure to be measured before and after deformation at different times. The image of the structure to be measured before deformation is called the reference image, and the image after deformation is called the current image. In the reference image, a (2N + 1)×(2N + 1) region with the point (x, y) as the seed point is selected as the reference subset, where N is a natural number. Then, the current subset with the highest correlation with the reference subset is searched for in the current image. The correlation between subsets is measured by the zero-mean normalized sum of squared differences (Zero-Normalized Sum of Squared Differences Criterion, C ZNSSD ), and its value range is (0, 1). When the value of C ZNSSD is closer to 0, the correlation between the reference subset and the current subset is stronger. The calculation formula of C ZNSSD is:

[0052]

[0053] In the formula, I is the number of pixels in each subset, f l represents the pixel points of the reference subset, g i represents the pixel points of the current subset. Pixel points refer to the pixel coordinates of feature points, represents the average gray value of the reference subset, represents the average gray value of the current subset, and The calculation formula is as follows:

[0054]

[0055] f(x, y) and g(x′, y′) are the pixel values of the corresponding points (x, y) and (x′, y′) in the reference subset and the current subset respectively.

[0056] According to the above principle, the current subset with the maximum correlation with the reference subset can be determined through the matching process. Further, by calculating the coordinate difference of the image center points of the reference subset and the current subset, the two-dimensional displacement data of the object surface can be obtained.

[0057] The above is the principle of the two-dimensional digital image method.

[0058] The principle of the three-dimensional digital image correlation method is introduced below. The three-dimensional digital image correlation method is a three-dimensional measurement method that combines the two-dimensional digital image correlation method and the binocular vision system, and is used for three-dimensional shape and deformation analysis of the object to be measured. Its matching principle is as Figure 1 shown. Taking the speckle image obtained by the left camera in the initial state as the reference image, the right camera image first performs a stereo match according to the reference image, and then the two camera images perform a temporal match.

[0059] After obtaining the current subset that is most relevant to the reference subset through the above matching, by using the calibration parameters of the binocular camera system, the projection relationship between the three-dimensional coordinates P(X W , Y W , Z W ) and the image coordinates P1(x, y) and P2(x′, y′) of the two cameras is established:

[0060]

[0061] In the formula, M1 and M2 are the camera projection matrices, and S1 and S2 are the scale factors. Then, the three-dimensional coordinates can be solved by the least squares method.

[0062] Aiming at the problem that the camera motion interferes with the measurement accuracy of the three-dimensional digital image correlation displacement, this scheme adopts a robust compensation algorithm that combines static features and depth information screening. Specifically, this scheme uses the image feature points to estimate the camera self-motion model. First, the image is divided into grid regions for processing to construct a feature subset with uniform spatial distribution. Then, the motion feature points generated by environmental changes are removed through threshold judgment, that is, the motion feature points of its own motion existing in the image. Finally, the reference points with depths close to the measurement target are selected to solve the camera self-motion parameters. The specific process is as Figure 2 and Figure 3 shown.

[0063] The inter-frame difference method is a technique commonly used to detect moving regions in video and image sequences. It determines whether there is motion in an image by comparing the pixel gray-scale changes in adjacent images. In the acquired image sequence, the reference image is denoted as I1(x,y), and the current image is denoted as I t (x,y), where x and y are the abscissa and ordinate values of the gray-scale image respectively. Subsequently, the difference map ΔT of the reference image and the current image is calculated, and ΔT is divided into 20×20 grids. Calculate the overall standard deviation and mean of the difference map and the standard deviation and mean of the difference map within each grid. If the standard deviation and mean of the difference map within a certain grid meet the set judgment conditions, then that grid is determined to be a stable region:

[0064] (μ c -3σ c <μ w <μ c +3σ c )&&(σ w <3σ c )

[0065] In the formula, μ c is the mean gray-scale value of the entire difference map, and σ c is the standard deviation of the gray-scale value of the entire difference map; μ w is the mean gray-scale value of the difference map within the grid currently being judged, and σ w is the standard deviation of the gray-scale value of the difference map within the grid currently being judged; && represents logical "AND". For the grids not determined to be stable regions, their gray-scale values are all set to 0, so that this region does not participate in subsequent calculations.

[0066] The Speeded-Up Robust Features (SURF) algorithm is an efficient feature point detection method that can extract feature points in consecutive images and generate feature point descriptors. Using the first frame as the reference image, after detecting the feature point pairs in the reference image and the current image using the SURF algorithm, the Random Sample Consensus (RANSAC) algorithm is used to screen the matching point pairs, so as to filter out the wrong matching points and retain the inliers that conform to the motion model. Subsequently, the feature points in the above-mentioned determined stable regions are judged to screen out the moving feature points in the stable regions, and this type of feature points does not participate in subsequent calculations.

[0067] Based on the above principle, first use the inter-frame difference method to perform difference calculations on the meshed regions to determine whether they are stable regions, and then use the accelerated robust feature algorithm to detect the feature points in the stable regions.

[0068] All the feature points on the image are divided into two categories: one is the moving feature points, and the other is the stationary feature points. The feature points in the stable area are not necessarily all stationary feature points, and there are a small number of moving feature points. In order to further screen and remove the moving feature points in the stable area, the initial position of the reference image is (x1, y1), and the position of the current image is (x i , y i ), and its displacement is:

[0069]

[0070] Among them, d k represents the displacement of this feature point relative to the reference. In order to reflect the overall motion state of the feature points in the grid, further calculate the average displacement of all the matching feature points in the current grid G t and the reference grid G as:

[0071]

[0072] Among them, n represents the number of matching feature points in the image grid, reflecting the overall motion state of the background feature points. Combining formulas (9) and (10), the following threshold determination is performed on the feature points:

[0073]

[0074] For the feature points with the determination result of 0, it means that the displacement of this point significantly deviates from the average motion state in the grid, and these feature points are identified as moving feature points and do not participate in the calculation of the subsequent camera motion model.

[0075] Through the above series of steps, the accurate determination of the moving area and the effective screening of the moving feature points are realized, providing a more accurate and reliable data basis for the subsequent analysis.

[0076] After the screening of the stationary feature points is realized, the remaining stationary feature points of the left image and the right image, that is, the remaining stationary feature points after removing the moving feature points on the image, are subjected to feature point matching for the subsequent camera motion model estimation. The left image and the right image are two images taken by two cameras at the same moment. According to the pinhole camera model, the point P(X, Y, Z) in the world coordinate system and its corresponding pixel coordinates (x, y) have the following relationship:

[0077]

[0078] In the formula, f is the focal length of the camera, (c x , c y) is the coordinate of the principal point of the image. Assume there are two points P1(X1, Y1, Z1) and P2(X2, Y2, Z2) in the world coordinate system, where P1 is the point close to the measurement target, and its corresponding pixel coordinate is p1(x1, y1). The depth difference between P2 and P1 is ΔZ. Depth refers to the Z coordinate of the feature point, which is the depth value corresponding to the origin of the world coordinate system of the left camera. Using P1 to estimate the camera motion model, we can obtain

[0079] p′ = A·p1 + t (13)

[0080] where p′ is the compensated point, and A is the rotation matrix t is the translation vector When using P2 to estimate the camera motion matrix, there is a depth difference ΔZ. Combining (12) and (13), we can obtain

[0081]

[0082] It can be seen from the above formula that when the distance of the measurement target is not far, ΔZ is a factor affecting its compensation effect. Then the deviation E of the estimated affine transformation parameters from the actual values is:

[0083]

[0084] The affine transformation parameters refer to the parameters in the above rotation matrix A and translation vector t. It can be seen from the above formula that when ΔZ approaches 0, the error also approaches 0. Therefore, in order to calculate the affine transformation matrix more accurately and achieve camera motion compensation, feature points close to the depth of the measured object are selected. This can reduce the influence of depth difference on affine transformation, thereby improving the accuracy and reliability of the calculated affine transformation matrix.

[0085] Calculate the three-dimensional coordinates (x i , y i , z i ) of the left and right matching feature points through the internal and external parameters obtained by camera calibration. Subsequently, calculate the average coordinate Z s in the Z direction of all the feature points detected on the speckle of the measured target at the previous moment, and calculate its mean value u Z and σ Z . By analyzing the depth distribution of the measured speckle, determine an appropriate range δ = kσ Z , where k is a constant, and the preferred value range is [1, 3]. Use D = [Z s - δ, Z s + δ] as the condition for screening the Z-direction coordinate z i . Only retain the feature points where z i ∈ D for calculating the camera motion model. The stationary feature points obtained after screening are the reference features.

[0086] Among them, the average Z coordinate of the speckles of the target to be measured used in the above process is that of the previous moment, because the camera may have moved at the current moment, and the specific parameters of the movement are still unknown when the method reaches this point, so that of the previous moment is used. Which specific moment can be flexibly selected. In this embodiment, the previous moment is the moment of the previous image acquisition.

[0087] After implementing the above process, the final feature point set is obtained, and the motion model of the camera is calculated using this point set to achieve motion compensation for the camera. For a binocular vision system, the relationship between the world coordinate system and the two camera coordinate systems is as Figure 4 shown, where H1 and H2 respectively represent the motions of the two camera coordinate systems relative to the initial world coordinate system; R2 and T2 describe the relative translation and rotation relationships between the two cameras.

[0088] Select the first frame as the reference image. Assuming that the pixel points before and after the camera movement are (x, y) and (x′, y′) respectively, then the image point before the camera movement can be expressed as a function of the corresponding image point after the camera movement:

[0089]

[0090] where a1 to a8 are coefficients.

[0091] The above is the general form of the projection model. In this study, the working distance of the experiment is 0.75 m, and the speckle movement range of the target to be measured is 0–5 mm. At this time, the movement between camera images is mainly composed of translation, rotation, and scaling. Therefore, compared with the projection transformation model with more degrees of freedom, the affine matrix can more reasonably and efficiently describe the movement relationship between images, and the above formula can be transformed into:

[0092]

[0093] At this time, the affine matrix contains 6 unknown coefficients. When the number of matching feature point pairs between the current image and the reference image is not less than 3 pairs, the affine transformation matrix between the image planes before and after the camera movement can be calculated, and then the current image is transformed to the perspective of the reference image through inverse transformation, so as to achieve motion compensation for the camera.

[0094] After completing the camera motion compensation, the displacement of the target speckles based on the image sequence is recalculated. For the images before and after compensation, the 3D-DIC algorithm is used to calculate the actual displacement of the target speckles. In order to verify the compensation effect, the root mean square error (RMSE) is further used to evaluate the accuracy of the displacement measurement results, and the calculation formula is as follows:

[0095]

[0096] where d i represents the displacement measurement value, represents the true displacement value, and N represents the acquired image sequence. Subsequently, substituting into (18), by calculating W, the improvement degree of the camera motion compensation on the displacement measurement accuracy can be quantitatively evaluated. When W decreases, the compensated displacement result is closer to the true value, that is, the compensation strategy is effective.

[0097] To verify the effectiveness of the proposed method, a rigid body displacement experiment is used for verification and analysis. The experimental system consists of two Point_gray cameras, camera brackets, a computer, a three-dimensional micro-displacement platform, and a measured speckle specimen. The minimum readings of the three-dimensional micro-displacement platform in the x, y, and z directions are all 0.01 mm. Pre-made landmark speckles with a certain texture distribution are pasted on the measured speckle specimen as the measured object.

[0098] The experiment uses Point_gray cameras with a resolution of 2048 pixel×2048 pixel, and the focal lengths of the lenses are all f0 = 12 mm. The internal and external parameters of the cameras are obtained using the classic Zhang Zhengyou calibration method. The array size of the checkerboard used for calibration is 11×11, and the side length of the grid is 20 mm. The calibration results are shown in Table 1, where (u0, v0) and (u1, v1) respectively represent the center point coordinates of the left and right image coordinate systems, and α x and α y are the radial distortion coefficient matrices of the cameras.

[0099] Table 1 Camera calibration parameters

[0100]

[0101] After calibration, to verify the accuracy of the 3D-DIC algorithm used in this paper for measuring three-dimensional displacements, the micro-displacement platform is adjusted to make the target speckle move alternately in the x, y, and z directions. It moves 10 times in each direction, and the displacement step size each time is 0.1 mm. The evaluation of the displacement measurement accuracy is carried out by calculating the root mean square error between the measured displacement and the actual displacement.

[0102] The results show that in the three directions, the measured values of DIC and the values of the micro-displacement platform are in good agreement. Specifically, the RMSE of the displacement in the x direction is 0.012 mm; the RMSE in the y direction is 0.014 mm; the RMSE in the z direction is 0.007 mm, and the RMSE of the three-dimensional displacement in space is 0.0092 mm. This shows that in different directions, the measurement error is small and has good measurement accuracy, verifying the effectiveness of the adopted displacement measurement method.

[0103] Next, through experimental verification, the effect of camera self-motion compensation on ensuring the accuracy of the 3D-DIC method is verified. The above-mentioned three-dimensional micro-displacement platform is still used, and the target speckle marker is fixed on the platform surface. The target speckle moves alternately along the x, y, and z directions. Each time the target speckle moves, an image is collected, for a total of 31 images. During the image acquisition process, external disturbances are artificially introduced to simulate camera jitter in the real environment. Different marker points are set at the same background depth. The marker points are divided into two types. One is the stationary feature points fixed on the anti-vibration platform, and the other is the moving feature points with vibration applied by the shaker. Subsequently, under the condition of camera jitter, the measurement error situations of implementing camera self-motion compensation (without using the reference feature point screening method proposed in the present invention) and not implementing camera self-motion compensation are compared, as shown in Figures 7(a) to 7(d) shown. In the displacement compensation effect diagram without feature point screening, when the camera jitters, the measured values show obvious fluctuations and deviate significantly from the true values, showing irregular up and down fluctuations. This indicates that camera jitter will significantly reduce the accuracy and stability of DIC displacement measurement, verifying the necessity of the jitter compensation algorithm. When using the method without feature point screening, especially when there are moving feature points in the field of view, its compensation algorithm fails to effectively correct the measurement deviation.

[0104] Next, the reference feature points are screened using the method of the present invention, and its compensation effect is as shown in Figures 8(a) to 8(b) shown. The comparison of the effects of camera self-motion compensation without screening feature points and camera self-motion compensation using the method of the present invention on improving the measurement accuracy of the 3D-DIC method is shown in Table 2. The results show that in this experiment, without camera self-motion compensation, the RMSE of the three-dimensional displacement reaches 0.8207 mm, and the measurement has basically failed; after using camera self-motion compensation, the RMSE is reduced to 0.1609 mm; and after screening the reference feature points using the method of the present invention and then performing camera self-motion compensation, the RMSE can be further reduced to 0.0384 mm. This experiment not only illustrates the necessity of camera self-motion compensation under field conditions, but also shows that the method of the present invention has significant advantages compared with the existing camera self-motion compensation methods.

[0105] Table 2 Comparison of RMSE of displacement measurement

[0106]

[0107]

[0108] Finally, it is necessary to clarify here that: the above embodiments are only used to further illustrate the technical solutions of the present invention in detail, and cannot be understood as a limitation on the protection scope of the present invention. Some non-essential improvements and adjustments made by those skilled in the art based on the above content of the present invention all belong to the protection scope of the present invention.

Claims

1. A 3D-DIC camera self-motion compensation method for automatically selecting reference features, characterized in that It includes the following steps: Step S1: Calculate the difference map between the reference image and the current image, and perform grid-based regional processing on the difference map; Step S2: Calculate the overall standard deviation and mean of the difference map, as well as the standard deviation and mean of the difference map within each grid. If the standard deviation and mean of the difference map within a certain grid meet the set judgment conditions, then determine that grid as a stable region; Step S3: Use the Speeded Up Robust Features (SURF) algorithm to detect feature points in the stable region and eliminate moving feature points, leaving stationary feature points; Step S4: Perform feature point matching on the stationary feature points on the two images captured by the two cameras at the same moment, and calculate the three-dimensional coordinates of each matching feature point: If the difference between the average depth of the feature point and the feature point at the previous moment of the target to be measured is less than the set value, then retain the feature point, otherwise eliminate the feature point, to obtain a set of stationary feature points; Step S5: Calculate the affine transformation matrix between the camera coordinate systems before and after the movement of the same camera according to the parameters of the stationary feature points obtained in Step S4, and transform the current image to the perspective of the reference image through affine transformation to achieve motion compensation for the camera.

2. The method according to claim 1, wherein: In Step S2, the judgment condition for the grid to be a stable region is: (μ c - 3σ c < μ w < μ c + 3σ c ) && (σ w < 3σ c ) Where, μ c is the average gray value of the entire difference image, and σ c is the standard deviation of the gray value of the entire difference image; μ w is the average gray value of the difference image within the grid currently being judged, and σ w is the standard deviation of the gray value of the difference image within the grid currently being judged; && represents the logical "AND". When μ w and σ w meet this condition, it is determined that the current grid is a stable area.

3. The method according to claim 1, wherein: In Step S3, the following threshold judgment is performed on the feature points: where d k is the total displacement of the feature point relative to its position in the reference image, is the average displacement of all feature points matched within the current grid and the corresponding reference image grid. If D k = 1, then the feature point is a stationary feature point. If D k = 0, then the feature point is a moving feature point.

4. The method according to claim 3, characterized in that: In step S3, assume that the initial position of a feature point in the reference image is (x1, y1), and the position in the current image is (x i , y i ). Then:

5. The method according to claim 1, wherein: Step S4 includes the following steps: Step S41: Calculate the three-dimensional coordinates (x i , y i , z i ) of the matching stationary feature points on the two images; Step S42: Calculate the average Z-direction coordinate Z of all the feature points detected on the target to be measured at the previous moment s , and calculate its mean value μ Z and standard deviation σ Z , Step S43: Determine the parameter δ, where δ = kσ Z , where k is a constant with a value range of [1, 3]; Step S44: Take D = [Z s -δ, Z s +δ] as the condition for screening the Z-direction coordinate z i , and only retain the feature points where z i ∈D, and eliminate other points.

6. The method according to claim 1, characterized in that: In Step S5, according to the coordinates of more than six stationary feature points obtained in Step S4 in the two camera coordinate systems, the least squares method is used to solve the affine transformation matrix.

Images