A monocular real-time measurement method of prism speckle for three-dimensional deformation
Through the real-time measurement method of prism speckle monocular, the image matching and three-dimensional reconstruction are used to use the ORB+BEBLID algorithm to solve the problems of large equipment, high cost and inability to measure real-time in the existing technology, and the three-dimensional adaptability to small spaces and low-cost Deformation measurement.
Patent Information
- Application Number
- CN202111620753.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-12-27
AI Technical Summary
The existing three-dimensional deformation measurement methods have problems such as large equipment size, high cost, and the inability to measure and adapt to small spaces in real time, especially the application of binocular cameras in actual engineering sites is limited.
The real-time measurement method of prism speckle monocular is adopted, and the step-by-step process is: determining the distance between the prism and the camera's optical center, collecting the calibration plate images, solving the camera parameters, ejecting the speckle, collecting the images before and after deformation, and using the ORB+BEBLID algorithm to perform images Matching and three-dimensional reconstruction to realize real-time measurement of three-dimensional deformation.
Real-time measurement of three-dimensional deformation of equipment with small size, low cost and adaptable to small spaces. It has the characteristics of non-contactness and high efficiency, and is suitable for high-speed field applications.
Smart Images

Figure CN114529598B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical fields of engineering materials, optical mechanics and deformation measurement, and relates to a prism speckle monocular real-time measurement method suitable for three-dimensional deformation. Background Art
[0002] In recent years, with the continuous development of science and technology and the continuous expansion of people's living needs, new materials in various fields have emerged in an endless stream. The mechanical properties of materials have a great influence on the engineering use of materials. The most critical is deformation measurement. The three-dimensional micro-deformation measurement of materials can be used for strain deformation measurement of large oil pipelines, concrete mechanical properties test, etc. to analyze the micro-deformation degree of materials. It can predict the safety performance and service life of materials in various engineering scenarios, and provide a theoretical basis for structural optimization, reliability evaluation and other issues in actual engineering design.
[0003] Existing methods for measuring material deformation include traditional methods such as resistance strain gauges, extensometers and other technologies, which are pasted on the surface of the specimen to observe the deformation of the area of interest of the material, but they can only measure the material point by point and cannot obtain full-field data, so they cannot measure excessive deformation; in recent years, optical-based deformation measurement methods include photoelasticity and laser holographic interferometry, which require the use of lasers as light sources and have complex optical paths that are difficult to arrange. The measurement results are easily affected by the outside world, which is convenient for research in the laboratory, but has limited application in actual engineering sites.
[0004] The three-dimensional deformation (out-of-plane displacement) of the material needs to be captured by a binocular camera. However, for some small spaces, the binocular camera is large in size. In addition, the binocular camera has the problem of asynchronous shooting and the equipment cost is high. Summary of the invention
[0005] The purpose of the present invention is to provide a prism speckle monocular real-time measurement method suitable for three-dimensional deformation, which can realize real-time and accurate measurement of three-dimensional deformation while reducing the measurement cost.
[0006] The technical solution adopted by the present invention is a monocular real-time measurement method of prism speckle applicable to three-dimensional deformation, which specifically includes the following steps:
[0007] Step 1, determine the distance between the prism and the optical center of the camera so that the camera plane contains all the objects to be observed and the object image fully occupies the two camera planes, that is, to obtain an effective FOV;
[0008] Step 2, collecting the calibration plate image;
[0009] Step 3, solve the internal and external parameters of the camera: use the collected prism imaging image to segment the left and right views according to the image pixel size, and use the dual-target positioning method to obtain the camera's external parameters R, T, and the internal parameters u0, v0, αx , α y , β;
[0010] Step 4, spraying speckles onto the material to be tested;
[0011] Step 5, collecting speckle images before and after deformation;
[0012] Step 6: Based on the camera internal and external parameters obtained by the camera calibration in step 3 and the speckle pattern of the material to be tested collected in step 5, the ORB+BEBLID algorithm is used to match the pixels of the left and right views. The ORB algorithm detects the feature points, and the BEBLID algorithm describes the detected feature points. Then, according to the triangulation principle, the depth of the surface of the material to be tested is calculated, and then the entire three-dimensional contour of the material to be tested is reconstructed to obtain the three-dimensional coordinate information.
[0013] Step 7, before the material to be tested is deformed, execute step 6; after the material to be tested is deformed, execute step 6 again; subtract the corresponding dimensions of the three-dimensional coordinates obtained before and after the deformation to obtain the deformation information in each dimension.
[0014] The present invention is also characterized in that:
[0015] Step 2: Collecting the calibration image must meet the following conditions:
[0016] (a) The number of checkerboard squares on one side of the calibration board must be equal to or even and mutually exclusive with the number on the other side;
[0017] (b) The inclination angle between the calibration plate and the camera optical axis does not exceed 45 degrees;
[0018] (c) The calibration plate image occupies one third of the interface.
[0019] In step 3, the rotation matrix R of the camera extrinsic parameters is a unit matrix:
[0020]
[0021] Let b represent the baseline length of binocular vision, then the translation vector T should ideally be:
[0022]
[0023] In step 6, the ORB algorithm is used to detect feature points, and the BEBLID descriptor is used to describe the feature points to complete the matching. The FAST algorithm used in ORB feature detection is used. For the feature points detected in the image, the BEBLID descriptor D(w) is:
[0024]
[0025] Where A is a symmetric matrix, and h(x) is the w response vector of K weak learners to the image patch.
[0026] In step 7, the following formula is used to calculate a certain point P(X w ,Y w ,Z w ) to rebuild:
[0027]
[0028] Among them, P(X w ,Y w ,Z w ) is the world coordinate of the point to be measured, (u1,v1) is the pixel coordinate of the point to be measured in the left view, (u2,v2) is the pixel coordinate of the point to be measured in the right view, are the projection matrices of the virtual left and right cameras respectively, where i is 1 to 3 and j is 1 to 4. The following formula represents the conversion relationship between pixel coordinates and world coordinates, which is composed of camera intrinsic parameters and extrinsic parameters:
[0029]
[0030] Where M is K[R|T], Z c is the scale factor, the camera external parameters include a rotation matrix R and a translation matrix T, the camera internal parameters are K, and the formula parameters after calibration are as follows:
[0031]
[0032] Among them, u0, v0 is the optical center of the camera, that is, the coordinates of the intersection of the camera optical axis and the image plane; α x , α y is the number of pixels per millimeter in the (x, y) direction of the image coordinate system; β is the tilt parameter, which is generated because the image coordinate system is orthogonal to the two coordinate axes.
[0033] The triangulation principle formula used in step 6 is as follows:
[0034]
[0035] Among them, f is the focal length of the camera, b is the baseline length of the left and right cameras, and d is the disparity obtained by stereo matching.
[0036] The beneficial effects of the present invention are as follows:
[0037] 1. Compared with traditional binocular stereo vision, the present invention only requires one camera and one prism, and the device is small in size and can be used in a small space.
[0038] 2. The present invention is a non-contact measurement method that is applicable to a wide range of scenarios.
[0039] 3. For scenarios with limited budget, the present invention only requires one camera, and the hardware equipment cost is low.
[0040] 4. Since the present invention uses a single camera and a prism to simulate binocular imaging, camera synchronous acquisition can be achieved.
[0041] 5. Since the speckle image matching algorithm used in the present invention is highly efficient, it is suitable for scenes where high-speed cameras are used. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a flow chart of the monocular real-time measurement method of prism speckle applicable to three-dimensional deformation of the present invention;
[0043] Figure 2 It is a schematic diagram of prism imaging in the prism speckle monocular real-time measurement method applicable to three-dimensional deformation of the present invention;
[0044] Figure 3 It is a shooting angle azimuth diagram of a calibration plate in the prism speckle monocular real-time measurement method applicable to three-dimensional deformation of the present invention;
[0045] Figure 4 It is a main structure diagram of the measuring device in the prism speckle monocular real-time measurement method applicable to three-dimensional deformation of the present invention;
[0046] Figure 5 The ORB+BEBLID algorithm of the prism speckle monocular real-time measurement method suitable for three-dimensional deformation of the present invention simulates the speckle image to extract the feature matching result. DETAILED DESCRIPTION
[0047] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0048] The present invention is applicable to a monocular real-time measurement method of prism speckle of three-dimensional deformation, and the required measurement system includes a CCD camera, a triangular prism, a bottle of speckle spray paint, a chessboard calibration plate, a computer, and a material to be measured. The computer includes an image acquisition control unit, an image segmentation unit, and a camera parameter calibration unit, wherein the image acquisition control unit is used to control the CCD camera to acquire images; the image segmentation unit is used to perform the following steps 3 and 5; the camera parameter calibration unit is used to perform the following step 4 to complete the calibration of the virtual binocular measurement system.
[0049] The prism speckle monocular measurement method for three-dimensional deformation proposed by the present invention has the following specific operation steps: Figure 1 shown.
[0050] Step 1: Determine the prism position: First, before purchasing a prism, you need to consider the actual application
[0051] For the requirements of the width and length of the entire system, select the angle, volume, size and other parameters of the prism; after determining the above parameters, there is a distance range x between the prism and the camera lens. If the width of the prism is d, the focal length of the camera is f, and the length of the camera image sensor is t, then:
[0052]
[0053] According to the fixed camera position and the position of the material to be measured, move the prism forward and backward; adjust the camera focal length and determine the distance between the prism and the camera optical center so that the camera plane contains all the objects to be observed and the object image fully occupies the two camera planes, that is, obtain the effective FOV (the area where the camera can capture the image), such as Figure 2 As shown, the effective FOV of the system is between R2 and R3.
[0054] Step 2, collect the calibration plate image: First, ensure that the calibration plate has high accuracy. The present invention uses a 12×9 checkerboard, each grid is 5mm, and there are 88 corner points in total. Place the checkerboard calibration plate within the imaging range of the virtual binocular measurement system, and use the measurement system to collect (rotate or translate) in different orientations, but the inclination angle between the calibration plate and the camera optical axis should not be greater than 45 degrees, such as Figure 3 As shown; collect as many images as possible, because in many cases more than half of the images will eventually be screened out, so this example collects 57 images.
[0055] Step 3, solve the internal and external parameters of the camera: use the collected prism imaging image to segment the left and right views according to the image pixel size, and use the dual-target positioning method to obtain the camera's external parameters R, T, and the internal parameters u0, v0, α x , α y , β; This example adopts Zhang Zhengyou's calibration method, and uses the calibration toolbox that comes with MATLAB. For the calibration results, since the optical axes of the left and right cameras in the virtual binocular vision measurement system obtained by this measurement system are almost parallel to the XOZ plane of the original camera coordinate system. The relative pose transformation obtained after the calibration of the left and right cameras in the binocular system is represented by a 3*3 dimensional rotation matrix R and a 3*1 translation vector T. Therefore, in the present invention, the rotation matrix R of the camera extrinsic parameter should theoretically be a unit matrix (but in practice it can only be close to the unit matrix):
[0056]
[0057] Let b represent the baseline length of binocular vision, then the translation vector T should ideally be:
[0058]
[0059] Step 4, spraying speckle on the material to be tested: Surface treatment of the material to be tested, including but not limited to removing surface dust, spraying the material to be tested with spray paint, gently applying force until the speckle color evenly covers the entire surface, waiting for 3-5 minutes for the spray paint to take shape, and obtaining the speckle template pattern distributed on the surface of the material to be tested.
[0060] Step 5, collect the speckle images before and after deformation: Place the material to be tested at the same approximate position as the calibration plate, no more than 10 cm before and after, and the camera's shooting position is as follows: Figure 4 The speckle images of the material to be tested before and after deformation are collected, and the images are segmented according to the pixel size to obtain the left and right views.
[0061] Step 6, speckle image matching, 3D reconstruction: Based on the camera internal and external parameters obtained by the camera calibration in step 3, and the speckle image of the material to be tested collected in step 5, the ORB+BEBLID algorithm is first used to match the pixels of the left and right views, such as Figure 5 As shown, the computer simulates the speckle to extract and match the image feature points; then according to the triangulation principle, the depth information of the surface of the workpiece to be measured is calculated, and then the entire three-dimensional contour of the workpiece to be measured is reconstructed to obtain the three-dimensional coordinate information.
[0062] The triangulation principle formula used is as follows:
[0063]
[0064] Among them, f is the focal length of the camera, b is the baseline length of the left and right cameras, d is the disparity obtained by stereo matching, z is the depth information, xl is the coordinate of a point in space in the left camera, and xr is the coordinate of a point in space in the right camera.
[0065] In the image matching step, step 1): ORB uses the FAST algorithm to detect feature points. First, a pixel point P is selected from the image, and its grayscale value is set to Ip; consider the 16 pixels in the circle around the pixel point; FAST only compares P with the 4 equally spaced pixels in the circle. If the brightness of at least three consecutive pixels is higher or lower than lp, P is selected as a key point. The second step BEBLID describes the feature point, and the descriptor is:
[0066] Among them, A is a symmetric matrix, and its eigendecomposition can be used to select the eigenvector with the largest eigenvalue; h(x) is the w response vector of K weak learners to the image patch.
[0067] After the matching is completed, the three-dimensional reconstruction of the material to be tested is performed from the left and right views, with a certain point to be tested P(X w ,Y w ,Z w ) Take 3D reconstruction calculation as an example:
[0068]
[0069] Among them, P(X w ,Y w ,Z w ) is the world coordinate of the point to be measured, (u1,v1) is the pixel coordinate of the point to be measured in the left view, (u2,v2) is the pixel coordinate of the point to be measured in the right view, are the projection matrices of the virtual left and right cameras (i=1,2,3; j=1,2,3,4); the following formula represents the conversion relationship between pixel coordinates and world coordinates, which is composed of camera intrinsic parameters and extrinsic parameters:
[0070]
[0071] Where M is K[R|T], Z c is the scale factor, the camera external parameters include a rotation matrix R and a translation matrix T, the camera internal parameters are K, and the formula parameters after calibration are as follows:
[0072]
[0073] Among them, u0, v0 is the optical center of the camera, that is, the coordinates of the intersection of the camera optical axis and the image plane; α x , α y is the number of pixels per millimeter in the (x, y) direction of the image coordinate system; β is the tilt parameter, which is generated because the image coordinate system is orthogonal to the two coordinate axes.
[0074] Step 7, calculate the three-dimensional deformation: before the workpiece to be measured is deformed, execute step 6; after the workpiece to be measured is deformed, execute step 6 again; subtract the corresponding dimensions of the three-dimensional coordinates obtained before and after the deformation, the formula is as follows:
[0075] ΔX=X1-X0, ΔY=Y1-Y0, ΔZ=Z1-Z0,
[0076] Among them, ΔX, ΔY, ΔZ are the deformation displacements, X0, Y0, Z0 are the coordinates of the workpiece before deformation, and X1, Y1, Z1 are the coordinates of the workpiece after deformation. Subtracting the two can obtain the deformation information of the region of interest (ROI) of the material to be tested in each dimension.
Claims
1. A monocular real-time measurement method of prism speckle for three-dimensional deformation, characterized by: The specific steps include: Step 1, determine the distance between the prism and the optical center of the camera so that the camera plane contains all the objects to be observed and the object image fully occupies the two camera planes, that is, to obtain an effective FOV; Step 2, collecting the calibration plate image; Step 3, solve the internal and external parameters of the camera: use the collected prism imaging image to segment the left and right views according to the image pixel size, and use the dual-target positioning method to obtain the camera's external parameters R, T, and the internal parameters u0, v0, α x , α y , β; Step 4, spraying speckles onto the material to be tested; Step 5, collecting speckle images before and after deformation; Step 6: Based on the camera internal and external parameters obtained by the camera calibration in step 3 and the speckle pattern of the material to be tested collected in step 5, the ORB+BEBLID algorithm is used to match the pixels of the left and right views. The ORB algorithm detects the feature points, and the BEBLID algorithm describes the detected feature points. Then, according to the triangulation principle, the depth of the surface of the material to be tested is calculated, and then the entire three-dimensional contour of the material to be tested is reconstructed to obtain the three-dimensional coordinate information. Step 7, before the material to be tested is deformed, execute step 6; after the material to be tested is deformed, execute step 6 again; subtract the corresponding dimensions of the three-dimensional coordinates obtained before and after the deformation to obtain the deformation information in each dimension; in step 7, the following formula is used to calculate the deformation information of a certain point P(X) on the surface of the object to be tested: w ,Y w ,Z w ) to rebuild: Among them, P(X w ,Y w ,Z w ) is the world coordinate of the point to be measured, (u1,v1) is the pixel coordinate of the point to be measured in the left view, (u2,v2) is the pixel coordinate of the point to be measured in the right view, are the projection matrices of the virtual left and right cameras respectively, where i is 1 to 3 and j is 1 to 4. The following formula represents the conversion relationship between pixel coordinates and world coordinates, which is composed of camera intrinsic parameters and extrinsic parameters: Where, m is K[R|T], Z c is the scale factor, the camera external parameters include a rotation matrix R and a translation matrix T, the camera internal parameters are K, and the formula parameters after calibration are as follows: Among them, u0, v0 is the optical center of the camera, that is, the coordinates of the intersection of the camera optical axis and the image plane; α x , α y is the number of pixels per millimeter in the (x, y) direction of the image coordinate system; β is the tilt parameter, which is generated because the image coordinate system is orthogonal to the two coordinate axes.
2. The prism speckle monocular real-time measurement method applicable to three-dimensional deformation according to claim 1, characterized in that: The acquisition of calibration images in step 2 must meet the following conditions: (a) The number of checkerboard squares on one side of the calibration board must be equal to or even and mutually exclusive with the number on the other side; (b) The inclination angle between the calibration plate and the camera optical axis does not exceed 45 degrees; (c) The calibration plate image occupies one third of the interface.
3. The prism speckle monocular real-time measurement method applicable to three-dimensional deformation according to claim 2, characterized in that: In step 3, the rotation matrix R of the camera extrinsic parameter is a unit matrix: Let b represent the baseline length of binocular vision, then the translation vector T should ideally be:
4. The monocular real-time measurement method of prism speckle applicable to three-dimensional deformation according to claim 3, characterized in that: In step 6, the ORB algorithm is used to detect feature points, and the BEBLID descriptor is used to describe the feature points to complete the matching. The FAST algorithm used in the ORB feature detection is used. For the feature points detected in the image, the BEBLID descriptor D(w) is: Where A is a symmetric matrix; h(x) is the w response vector of K weak learners to the image patch.
5. The prism speckle monocular real-time measurement method applicable to three-dimensional deformation according to claim 4, characterized in that: The triangulation principle formula used in step 6 is as follows: Among them, f is the focal length of the camera, b is the baseline length of the left and right cameras, and d is the disparity obtained by stereo matching.
