A method for measuring the surface shape of highly reflective objects based on multiple cameras

Abstract

The present invention discloses a multi-camera-based method for measuring the surface shape of highly reflective objects. The method is applied to a system composed of a reference plane, a camera array, a standard plane mirror, an optical platform, and a computer. The method comprises the following steps: S1, each camera completes posture calibration with an LCD display screen by collecting a screen target image from the plane mirror image; S2, utilizing the uniqueness of the display screen to complete global calibration between the cameras; S3, measuring the standard plane mirror, and optimizing global calibration parameters by adding plane constraints using the characteristics of the standard plane mirror; S4, projecting a coded image onto the surface to be measured through the screen, collecting the modulated images reflected by the screen using the camera array, and reconstructing them separately; S5, performing splicing according to the global calibration parameters to complete the measurement. The present invention is a method for high-speed detection of the surface shape of large-sized, large-curvature, highly reflective objects, requiring only one calibration before measurement.

Description

Technical Field

[0001] The invention relates to a method for measuring the shape of a high-reflective surface based on multiple cameras. Background Art

[0002] Nowadays, three-dimensional measurement of highly reflective surfaces is an important part of industrial measurement. With the continuous development of industries such as automobile assembly, precision polishing, and free-form surface processing, the demand for accurate and efficient measurement of highly reflective surfaces, especially large-sized and large-curvature surfaces, has become increasingly strong. Phase deflectometry is a non-contact three-dimensional measurement method for highly reflective surfaces developed in recent years. It is favored for its high sensitivity and easy correction of system errors. However, this method is mainly based on gradient information reconstruction and can only obtain relative point cloud information. Therefore, it can only construct a single-viewpoint system to achieve local measurement of the surface to be measured. The size that can be reconstructed at one time is limited, and blind spots are easily generated in the measurement of large-curvature surfaces. It can be seen that there is currently a lack of a convenient, fast, and highly flexible measurement method to achieve rapid overall measurement of large-sized and large-curvature surfaces. Summary of the Invention

[0003] In order to address the shortcomings of the above-mentioned prior art, the present invention proposes a method for measuring the surface shape of highly reflective objects based on multiple cameras, in order to solve the local problem of measuring the surface shape of highly reflective surfaces with a single camera, thereby enabling the overall measurement of the surface shape of highly reflective surfaces with large size and large curvature, and improving the accuracy and efficiency of measuring large-size and large-curvature surfaces.

[0004] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0005] The multi-camera-based method for measuring the surface shape of a highly reflective object of the present invention is characterized in that five cameras are distributed at the center position above and around the optical platform, and the fields of view of each camera are non-overlapping. Each camera is connected to a computer. The method for measuring the surface shape of a highly reflective object is performed according to the following steps:

[0006] Step 1: Placing a standard plane mirror on the optical platform and adjusting the angle of the standard plane mirror so that each camera can indirectly observe a unique reference plane through the standard plane mirror; projecting a fringe image of the reference plane onto the standard plane mirror, and using five cameras to respectively capture mirror feature point images of the reference plane after reflection from the standard plane mirror;

[0007] Step 2: First, obtain the intrinsic parameters of each camera and the extrinsic parameters of the mirror camera through the Zhang Zhengyou calibration method, and use formula (1) to construct the posture conversion relationship model between the reference plane and the i-th camera:

[0008]

[0009] In formula (1), A i is the intrinsic parameter of the i-th camera, · is the normalization operation, v i is the normalized image plane coordinate of the i-th camera, I is the third-order unit matrix, P w is the target coordinate of the reference plane, n is the surface normal vector of the standard plane mirror, d is the distance from the standard plane mirror to the single camera, R i and T i They represent the rotation matrix and translation vector from the reference plane coordinate system to the i-th camera coordinate system; P w Represents the coordinate set of feature corner points on the reference plane, i∈[1,5];

[0010] Step 3: Solve equation (1) and use equation (2) to obtain the position transformation relationship between the i-th camera coordinate system and the reference plane coordinate system:

[0011] X i =R i ·P w +T i (2)

[0012] In formula (2), X i Represents the feature corner coordinate set P w The coordinate set in the i-th camera coordinate system;

[0013] Step 4: Based on the identity of the reference plane position, use equations (3) and (4) to obtain the rotation matrix R from the j-th camera coordinate system to the i-th camera coordinate system: ij and the translation vector T ij :

[0014]

[0015]

[0016] In formula (3) and formula (4), R j and T j They represent the rotation matrix and translation vector from the reference plane coordinate system to the j-th camera coordinate system, Represents R j The inverse matrix of

[0017] Step 5: Use the standard plane mirror as the object to be measured, and transmit sinusoidal fringe images with 4-step phase shifts in the horizontal and vertical directions to the standard plane mirror through the reference plane, and the phase difference between two adjacent fringe patterns is π / 2:

[0018] Step 6: using five cameras to respectively capture fringe images modulated by the surface of the object being measured and sending the images to the computer;

[0019] Step 7: The computer demodulates the fringe image captured by the i-th camera, and then performs phase unwrapping on the modulated image using a multi-frequency heterodyne method, thereby obtaining a modulated continuous phase from a relative phase principal value;

[0020] Step 8: Obtain a dense correspondence between the two-dimensional feature points of the image plane of the i-th camera and the three-dimensional feature points of the reference plane based on the relationship between the reference plane pixels and the phase in the sinusoidal fringe image and the relationship between the image pixels and the phase in the collected fringe image;

[0021] Step 9: Using the attitude conversion relationship model and the dense reflection correspondence relationship, construct the partial differential equation of the depth of the feature points on the surface of the object to be measured and solve it to obtain a quadratic polynomial about the depth of the feature points on the surface of the object to be measured. After solving the quadratic polynomial, the depth information of the surface of the object to be measured relative to the i-th camera is obtained, thereby obtaining the absolute position coordinates p of the surface of the object to be measured relative to the i-th camera. i =s i ·v i , which is the point cloud coordinate of the surface of the object being measured, and the surface shape of the object being measured is preliminarily restored;

[0022] Step 10: Use the back-projection model to establish a reprojection cost function, and transform the point cloud coordinates p i =s i ·v i The reprojection cost function is input as the initial value, and the Levenberg-Marquardt algorithm is used to constrain the reprojection cost function to obtain the precise point cloud coordinates of the surface of the object relative to the i-th camera;

[0023] Step 11: According to the process of steps 7 to 10, the precise point cloud coordinates of the surface of the object under test relative to each camera are obtained, thereby restoring the local three-dimensional information of the surface of the object under test;

[0024] Step 12: Let the j-th camera coordinate system be the main reference coordinate system, and use formula (5) to obtain the overall three-dimensional information p of the surface of the object under test relative to the j-th camera coordinate system: j :

[0025] p j =R ij ·p i +T ij (5)

[0026] Step 13: Pre-calibrate the five cameras:

[0027] Use formula (6) to establish the plane cost function F m :

[0028] Fm =minλ t E tf +λ c Cerr (6)

[0029] In formula (6), Cerr is the plane fitting function for the overall three-dimensional information p j The fitting error obtained after processing, E tf is the global reprojection cost function, and is obtained from Equation (7), t ,λ c are two scaling factors;

[0030] E tf =||X i -(R ij X j +T ij )|| 2 (7)

[0031] The Levenberg-Marquardt algorithm is used to calculate the planar cost function F m Constraints are applied to obtain optimal global parameters, including: optimal rotation matrix and the optimal translation vector

[0032] Step 14: Replace the standard plane mirror with the object to be measured, and based on the optimal global parameters, perform overall measurement of the surface of the object to be measured according to the process of steps 5 to 12.

[0033] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the method for measuring the surface shape of a highly reflective object, and the processor is configured to execute the program stored in the memory.

[0034] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium. The computer-readable storage medium is characterized in that the computer program executes the steps of the method for measuring the surface shape of a highly reflective object when the computer program is run by a processor.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The present invention builds a multi-camera measurement system, utilizes phase-shift deflectometry to project fringes, and photographs the highly reflective surface to be measured from different angles to perform three-dimensional topography measurement of various parts of large-scale, high-curvature, highly reflective surfaces. Current deflectometry methods are primarily based on the reconstruction of gradient information, which can only obtain relative point cloud information. Consequently, they can only construct single-viewpoint systems, enabling local measurement of the surface to be measured. The size that can be reconstructed at one time is limited, and blind spots are easily generated when measuring surfaces with high curvature. The present invention, based on absolute coordinate reconstruction, allows for global calibration to unify multi-viewpoint measurement results into a single coordinate system, thereby achieving simultaneous multi-angle measurement. This overcomes the localization issue of single-camera measurement of aspheric surfaces and completes the overall measurement of large-scale, high-curvature, highly reflective surfaces.

[0037] 2. Based on the ideal flatness of a standard plane mirror, the present invention proposes a coplanar constraint to optimize global parameters. The calibration of global parameters is achieved by measuring the standard plane mirror, thereby reducing system errors and improving the measurement accuracy of the multi-viewpoint measurement system.

[0038] 3. For different objects to be measured, especially those with large size and large curvature, such as automobile windshields, fuselage metal skins, semiconductor wafers and other highly reflective surface measurement objects, the number and arrangement of camera arrays can be flexibly set, and the method has strong compatibility.

[0039] 4. The present invention uses fringe projection to perform non-contact measurement of the measured surface. Compared with traditional contact measurement, the speed and efficiency are greatly improved. This method has the characteristics of high sensitivity and easy correction of system errors, which ensures the reliability and accuracy of surface detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a system structure diagram of the present invention;

[0041] Figure 2 is a flow chart of the method of the present invention;

[0042] Reference numerals in the figure: 1 first camera; 2 second camera; 3 third camera; 4 fourth camera; 5 fifth camera; 6 computer; 7 object to be measured; 8 optical platform; 9 reference plane. DETAILED DESCRIPTION

[0043] In this embodiment, a method for measuring the surface shape of a highly reflective object based on multiple cameras is provided. Figure 1 As shown, five cameras are distributed at the center and around the optical platform 8, and the fields of view of each camera are not overlapped. Each camera is connected to the computer 6, as shown in FIG. Figure 2 As shown, the method for measuring the surface shape of a highly reflective object is performed in the following steps:

[0044] Step 1: Place a standard plane mirror on the optical platform 8 and adjust the angle of the standard plane mirror so that each camera can indirectly observe a unique reference plane 9 through the standard plane mirror; project a fringe image of the reference plane 9 onto the standard plane mirror, and use five cameras to capture the mirror feature point images of the reference plane after reflection from the standard plane mirror;

[0045] In a specific implementation, when the five cameras are capturing images, the reference plane is kept stationary, and the angle of the standard plane mirror is changed in different directions to ensure that each group of cameras captures at least three images containing different angles of the standard plane mirror.

[0046] Step 2: First, obtain the intrinsic parameters of each camera and the extrinsic parameters of the mirror camera through the Zhang Zhengyou calibration method, and use formula (1) to construct the attitude conversion relationship model between the reference plane 9 and the i-th camera:

[0047]

[0048] In formula (1), A i is the intrinsic parameter matrix of the i-th camera, · is the normalization operation, v i is the normalized image plane coordinate of the i-th camera, I is the third-order unit matrix, P w is the target coordinate of reference plane 9, n is the surface normal vector of the standard plane mirror, d is the distance from the standard plane mirror to the single camera, R i and T i They represent the rotation matrix and translation vector from the reference plane coordinate system to the i-th camera coordinate system; P w Represents the coordinate set of feature corner points on the reference plane 9, i∈[1,5];

[0049] Step 3: Solve equation (1) and use equation (2) to obtain the position transformation relationship between the i-th camera coordinate system and the reference plane coordinate system:

[0050] X i =R i ·P w +T i (2)

[0051] In formula (2), X i Represents the feature corner coordinate set P w The coordinate set in the i-th camera coordinate system;

[0052] Step 4: Based on the identity of the position of the reference plane 9, use equations (3) and (4) to obtain the rotation matrix R from the j-th camera coordinate system to the i-th camera coordinate system: ij and the translation vector T ij :

[0053]

[0054]

[0055] In formula (3) and formula (4), R j and T j They represent the rotation matrix and translation vector from the reference plane coordinate system to the j-th camera coordinate system, Represents R j The inverse matrix of

[0056] Step 5: Use the standard plane mirror as the object to be measured, and transmit the sinusoidal fringe image I with 4-step phase shift in the horizontal and vertical directions to the standard plane mirror through the reference plane 9. n As shown in formula (5), the phase difference between two adjacent fringe patterns is π / 2:

[0057]

[0058] In formula (5), n∈[1,4] is the number of stripes, is the main value of the relative phase.

[0059] Step 6: Use five cameras to collect fringe images modulated by the surface of the object being measured and send them to the computer 6;

[0060] Step 7: The computer 6 demodulates the fringe image captured by the i-th camera, and then uses a multi-frequency heterodyne method to perform phase unwrapping on the modulated image, thereby obtaining the modulated continuous phase from the relative phase principal value;

[0061] Step 8: Based on the relationship between the reference plane pixels and phase in the sinusoidal fringe image and the image pixels and phase in the acquired fringe image, obtain the dense correspondence relationship m between the two-dimensional feature points of the image plane of the i-th camera and the three-dimensional feature points of the reference plane. i (x, y), where (x, y) is the coordinate of the two-dimensional feature point in the image plane of the i-th camera;

[0062] Step 9: Using the attitude conversion relationship model and the dense reflection correspondence relationship, use formula (6) to construct the partial differential equation of the depth of the feature point on the surface of the measured object and solve it:

[0063]

[0064] In formula (6), s i represents the depth of the feature point of the measured surface in the i-th camera coordinate system, (x, y) is the coordinate of the two-dimensional feature point in the image plane of the i-th camera, and the partial differential equation (6) is solved at each two-dimensional feature point (x0, y0) in the image plane of the i-th camera. The final result is the depth s of the feature point of the measured surface in the i-th camera coordinate system. iThe quadratic polynomial of (x,y) is shown in formula (7):

[0065] Ds i 2 +Es i +F=0 (7)

[0066] In formula (7), D, E and F are the coefficients of the polynomial, which are obtained by solving the partial differential equation (6). After solving the quadratic polynomial, we get s i (x, y), thereby obtaining the absolute position coordinates p of the surface of the object relative to the i-th camera i =s i ·v i , which is the point cloud coordinate of the surface of the object being measured, and the surface shape of the object being measured is preliminarily restored;

[0067] Step 10: Use formula (8) to construct a back-projection model, and further estimate the coordinates of the two-dimensional points on the reference plane obtained by back-projection. With the actual reference plane coordinate P w The difference between the point cloud coordinates p and the point cloud coordinates p is used to construct the reprojection cost function. i =s i ·v i The reprojection cost function is input as the initial value, and the Levenberg-Marquardt algorithm is used to constrain the reprojection cost function to obtain the precise point cloud coordinates of the surface of the object relative to the i-th camera.

[0068]

[0069]

[0070] In equations (8) and (9), r3 represents the geometric rotation extrinsic parameter R of the i-th camera i The third rotation component, R i =(r1r2r3), r3 T represents the transposed matrix of r3, Unit vectors representing the mirror normal vector, l, l * are the unit vectors of the reflected light direction and the incident light direction at the coordinates (x, y) of the two-dimensional feature point in the image plane of the i-th camera, respectively, and the geometric parameters R i 、T i and the mirror parameter n are solved in step 3.

[0071] Step 11: According to the process of steps 7 to 10, the precise point cloud coordinates of the surface of the object under test relative to each camera are obtained, thereby restoring the local three-dimensional information of the surface of the object under test;

[0072] Step 12: Let the j-th camera coordinate system be the main reference coordinate system, and use formula (10) to obtain the overall three-dimensional information p of the surface of the object under test relative to the j-th camera coordinate system: j :

[0073] p j =R ij ·p i +T ij (10)

[0074] Step 13: Pre-calibrate the five cameras:

[0075] Use formula (11) to establish the plane cost function F m :

[0076] F m =minλ t E tf +λ c Cerr (11)

[0077] In formula (11), Cerr is the plane fitting function for the overall three-dimensional information p j The fitting error obtained after processing, E tf is the global reprojection cost function, and is obtained from Equation (12), t ,λ c are two scaling factors;

[0078] E tf =||X i -(R ij X j +T ij )|| 2 (12)

[0079] The Levenberg-Marquardt algorithm is used to calculate the planar cost function F m Constraints are applied to obtain optimal global parameters, including: optimal rotation matrix and the optimal translation vector

[0080] Step 14: Replace the standard plane mirror with the object to be measured 7, and based on the optimal global parameters, perform overall measurement on the surface of the object to be measured 7 according to the process of steps 5 to 12.

[0081] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0082] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

Claims

1. A method for measuring the surface shape of a highly reflective object based on multiple cameras, characterized in that: Five cameras are distributed at the center and around the optical platform (8), and the fields of view of each camera are non-overlapping. Each camera is connected to a computer (6). The method for measuring the surface shape of a highly reflective object is performed according to the following steps: Step 1: placing a standard plane mirror on the optical platform (8), and adjusting the angle of the standard plane mirror so that each camera can indirectly observe a unique reference plane (9) through the standard plane mirror; projecting a fringe image of the reference plane (9) onto the standard plane mirror, and using five cameras to respectively capture the mirror feature point images of the reference plane after being reflected by the standard plane mirror; Step 2: First, obtain the intrinsic parameters of each camera and the extrinsic parameters of the mirror camera through the Zhang Zhengyou calibration method, and use formula (1) to construct the posture conversion relationship model between the reference plane (9) and the i-th camera: (1) In formula (1), is the intrinsic parameter of the i-th camera, is the normalization operation, is the normalized image plane coordinate of the ith camera, is the third-order identity matrix, is the surface normal vector of the standard plane mirror, is the distance from the standard plane mirror to the single camera, and Respectively represent the rotation matrix and translation vector from the reference plane coordinate system to the i-th camera coordinate system; represents the coordinate set of the feature corner points on the reference plane (9), ; Step 3: Solve equation (1) and use equation (2) to obtain the position transformation relationship between the i-th camera coordinate system and the reference plane coordinate system: (2) In formula (2), Represents the feature corner coordinate set; Step 4: Based on the identity of the position of the reference plane (9), the rotation matrix from the j-th camera coordinate system to the i-th camera coordinate system is obtained using equations (3) and (4). and translation vectors : (3) (4) In formula (3) and formula (4), and They represent the rotation matrix and translation vector from the reference plane coordinate system to the j-th camera coordinate system, express The inverse matrix of Step 5: Use the standard plane mirror as the object to be measured, and transmit sinusoidal fringe images with 4-step phase shifts in the horizontal and vertical directions to the standard plane mirror through the reference plane (9), and the phase difference between two adjacent fringe images is : Step 6: using five cameras to respectively capture fringe images modulated by the surface of the object being measured and sending the images to the computer (6); Step 7, the computer (6) demodulates the fringe image collected by the i-th camera, and then uses a multi-frequency heterodyne method to perform phase unwrapping on the modulated image, thereby obtaining the modulated continuous phase from the relative phase main value; Step 8: Obtain a dense correspondence between the two-dimensional feature points of the image plane of the i-th camera and the three-dimensional feature points of the reference plane based on the relationship between the reference plane pixels and the phase in the sinusoidal fringe image and the relationship between the image pixels and the phase in the collected fringe image; Step 9: Using the attitude conversion relationship model and the dense reflection correspondence relationship, construct the partial differential equation of the depth of the feature points on the surface of the object to be measured and solve it to obtain a quadratic polynomial about the depth of the feature points on the surface of the object to be measured. After solving the quadratic polynomial, the depth information of the surface of the object to be measured relative to the i-th camera is obtained, thereby obtaining the absolute position coordinates of the surface of the object to be measured relative to the i-th camera. , which is the point cloud coordinate of the surface of the object being measured, and the surface shape of the object being measured is preliminarily restored; Step 10: Use the back-projection model to establish a reprojection cost function, and convert the absolute position coordinates The reprojection cost function is input as the initial value, and the Levenberg-Marquardt algorithm is used to constrain the reprojection cost function to obtain the precise point cloud coordinates of the surface of the object relative to the i-th camera; Step 11: According to the process of steps 7 to 10, the precise point cloud coordinates of the surface of the object under test relative to each camera are obtained, thereby restoring the local three-dimensional information of the surface of the object under test; Step 12: Let the j-th camera coordinate system be the main reference coordinate system, and use formula (5) to obtain the overall three-dimensional information of the surface of the object under test relative to the j-th camera coordinate system: : (5) Step 13: Pre-calibrate the five cameras: Use formula (6) to establish the plane cost function : (6) In formula (6), To use the plane fitting function to fit the overall three-dimensional information The fitting error obtained after processing is is the global reprojection cost function, and is obtained by formula (7), 、 are two scaling factors; (7) The Levenberg-Marquardt algorithm is used to calculate the planar cost function Constraints are applied to obtain optimal global parameters, including: optimal rotation matrix and the optimal translation vector ; Step 14: Replace the standard plane mirror with the object to be measured (7), and based on the optimal global parameters, perform overall measurement on the surface of the object to be measured (7) according to the process of steps 5 to 12.

2. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the method for measuring the surface shape of a highly reflective object according to claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for measuring the surface shape of a highly reflective object according to claim 1 are executed.

Images