Calibration method and system for position relationship in three-dimensional measurement system, storage medium

Abstract

A kind of method and system for calibrating position relationship in three-dimensional measurement system, the method first takes object plane coordinate system as world coordinate system, calibrates the internal parameter and external parameter of camera according to calibration plate image, obtains the position relationship of object plane and camera;Introduce virtual space coordinate system which is mirror symmetry about object plane mirror image with projection device coordinate system, the position relationship of projection device and camera is calibrated using virtual space coordinate system, so as to adapt to the three-dimensional measurement of mirror object;When calibrating the position relationship of projection device and camera, the transformation relationship of virtual space coordinate system and camera coordinate system under multiple postures of object plane is used to calculate, to obtain more accurate estimated value;Finally, the estimated value obtained is used as initial value, global optimization is carried out, so as to improve the precision of calibration, and provide basis for subsequent three-dimensional measurement of object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to a method and system for calibrating position relationship in a three-dimensional measurement system and a storage medium. BACKGROUND

[0002] Glass substrates, automotive glass, silicon wafers and the like as representative mirror surface reflecting objects are widely used in the production of various related products and have become an indispensable part of people's daily life. Glass substrates for flat panel liquid crystal displays, automotive glass, silicon wafers for processing chips and the like have strict requirements for the surface flatness and processing precision in production and manufacturing, resulting in high production cost. In production and manufacturing, the light beam projected onto the surface of the glass substrate, silicon wafer and the like will be mirror reflected like a common mirror, which brings difficulty to the non-destructive measurement of the surface shape in the processing and manufacturing process. Undoubtedly, the research on the surface shape measurement method of mirror objects such as glass substrates, silicon wafers and free-form glass can guide the finishing and measurement process in the production process, and has very important significance for improving the production and processing efficiency and quality of glass substrates, silicon wafers and free-form glass and reducing the production cost.

[0003] The artificial detection method is a method of detecting the surface of the mirror object using the naked eye under strong light. This method is relatively inefficient and harmful to the health of workers, and the detection result is greatly affected by individuals. However, considering the overall speed and accuracy of the quality detection process, the quality detection of mirror objects in the industrial environment is still dominated by the artificial detection method, but this situation needs to be changed.

[0004] Three-dimensional measurement technology can be divided into contact measurement and non-contact measurement according to the contact method. The three-coordinate measuring machine (CMM, Coordinate Measuring Machine) is a traditional contact three-dimensional topography measurement technology. It can measure complex topography objects and has high measurement accuracy, but since it needs to contact the surface of the object, when measuring the height of each point, there are problems such as slow measurement speed, long time consumption and easy wear of the object surface. Optical three-dimensional measurement technology such as machine vision detection method has attracted widespread attention and become a hot topic in academic research field due to its non-contact, high precision, fast measurement speed and full-field measurement. With the development of digital signal processing technology and related devices, three-dimensional measurement technology will develop towards high speed and high precision, and the measured object will develop towards large size and microstructure. Machine vision detection method is a detection method with broad prospects, which can solve the problem of low efficiency of artificial detection method, and the detection field is often large. However, there are still some problems to be solved when applying machine vision detection method to three-dimensional measurement of mirror objects. SUMMARY

[0005] The application provides a position relationship calibration method and system in a three-dimensional measurement system, a storage medium, and aims to improve the calibration accuracy of the position relationship among a projection device, an object plane and a camera in the three-dimensional measurement system.

[0006] According to a first aspect, a position relationship calibration method in a three-dimensional measurement system is provided in an embodiment, the three-dimensional measurement system comprising a projection device, an object plane and a camera, the position relationship comprising a transformation relationship between a projection device coordinate system and a camera coordinate system and a transformation relationship between an object plane coordinate system and the camera coordinate system, and the calibration method comprising:

[0007] An object plane calibration board image is acquired, the object plane calibration board image being an image captured by the camera when the projection device projects a calibration board pattern on the object plane;

[0008] The object plane coordinate system is taken as a world coordinate system, and the intrinsic and extrinsic parameters of the camera are calibrated according to the calibration board image, the extrinsic parameters representing the transformation relationship between the world coordinate system and the camera coordinate system;

[0009] The object plane is transformed into several poses, an object plane calibration board image or a grating projection image under each pose is acquired, the coordinates of a pixel point in the calibration board image or the grating projection image in the camera coordinate system and the coordinates of a corresponding point in a virtual space coordinate system are acquired, the grating projection image of the object plane being an image captured by the camera when the projection device performs grating projection on the object plane, and the virtual space coordinate system being a space coordinate system obtained by mirror symmetry of the projection device coordinate system with respect to the object plane;

[0010] For each pose of the object plane, the transformation relationship between the virtual space coordinate system and the camera coordinate system under the pose is calculated according to the coordinates of the pixel point in the calibration board image or the grating projection image in the camera coordinate system and the coordinates of the corresponding point in the virtual space coordinate system under the pose;

[0011] According to the geometric relationship between the virtual space coordinate system and the projection device coordinate system, the transformation relationship between the projection device coordinate system and the camera coordinate system is calculated by using the transformation relationship between the virtual space coordinate system and the camera coordinate system under all poses;

[0012] The transformation relationship between the projection device coordinate system and the camera coordinate system is globally optimized according to a preset global optimization objective function.

[0013] According to a second aspect, in an embodiment, a system for calibrating position relationship in a three-dimensional measurement system is provided, the three-dimensional measurement system comprising a projection device, an object plane and a camera, the position relationship comprising a transformation relationship between a coordinate system of the projection device and a coordinate system of the camera and a transformation relationship between a coordinate system of the object plane and the coordinate system of the camera, the system comprising:

[0014] a calibration board image acquisition module configured to acquire a calibration board image of the object plane, the calibration board image of the object plane being an image captured by the camera when the projection device projects a calibration board pattern on the object plane;

[0015] a first calibration module configured to calibrate intrinsic parameters and extrinsic parameters of the camera according to the calibration board image, the extrinsic parameters representing the transformation relationship between a world coordinate system and the coordinate system of the camera, the world coordinate system being the coordinate system of the object plane;

[0016] a multi-pose coordinate acquisition module configured to transform the object plane into a plurality of poses, acquire a calibration board image or a grating projection image of the object plane in each pose, acquire coordinates of a pixel point in the calibration board image or the grating projection image in the coordinate system of the camera and coordinates of a corresponding point in a virtual space coordinate system, the grating projection image of the object plane being an image captured by the camera when the projection device performs grating projection on the object plane, the virtual space coordinate system being a space coordinate system obtained by mirror symmetry of the coordinate system of the projection device with respect to the object plane;

[0017] a second calibration module configured to calculate, for each pose of the object plane, a transformation relationship between the virtual space coordinate system and the coordinate system of the camera according to the coordinates of the pixel point in the calibration board image or the grating projection image in the pose in the coordinate system of the camera and the coordinates of the corresponding point in the virtual space coordinate system;

[0018] a third calibration module configured to calculate the transformation relationship between the coordinate system of the projection device and the coordinate system of the camera according to a geometric relationship between the virtual space coordinate system and the coordinate system of the projection device and the transformation relationships between the virtual space coordinate systems and the coordinate system of the camera in all poses;

[0019] a global optimization module configured to globally optimize the transformation relationship between the coordinate system of the projection device and the coordinate system of the camera according to a preset global optimization objective function.

[0020] According to a third aspect, in an embodiment, a computer readable storage medium is provided, the medium storing a program, the program being executable by a processor to implement the calibration method according to the first aspect.

[0021] The method and system for calibrating the positional relationship in the three-dimensional measurement system according to the above embodiment are used for calibrating the positional relationship among the projection device, the object plane and the camera in the three-dimensional measurement system. Since the mirror surface of the mirror object reflects the light, the virtual image of the light source about the mirror surface of the object plane is observed in the camera. Therefore, the virtual space coordinate system is introduced, which is the space coordinate system obtained by performing the mirror symmetry on the projection device coordinate system about the object plane. The transformation relationship between the virtual space coordinate system and the camera coordinate system is calibrated first, and then the positional relationship between the projection device and the camera is calibrated by using the transformation relationship between the virtual space coordinate system and the camera coordinate system, so that the three-dimensional measurement of the mirror object can be adapted. When the positional relationship between the projection device and the camera is calibrated, the transformation relationship between the virtual space coordinate system and the camera coordinate system of the object plane in multiple postures is used for calculation, which is beneficial to obtaining more accurate estimation value. Finally, the obtained estimation value is used as the initial value for global optimization, so that the calibration precision is improved, and a good foundation is provided for the subsequent three-dimensional measurement of the object. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The structural schematic diagram of the three-dimensional measurement system of an embodiment is shown in FIG. 1.

[0023] Figure 2 The transformation schematic diagram of the coordinate systems in the pinhole camera model is shown in FIG. 2.

[0024] Figure 3 The schematic diagram of the projected grating image in an embodiment is shown in FIG. 3.

[0025] Figure 4 The flowchart of the method for calibrating the positional relationship in the three-dimensional measurement system of an embodiment is shown in FIG. 4.

[0026] Figure 5 The flowchart of the method for calibrating the intrinsic and extrinsic parameters of the camera according to the calibration board image in an embodiment is shown in FIG. 5.

[0027] Figure 6 The structural schematic diagram of the calibration system of the positional relationship in the three-dimensional measurement system of an embodiment is shown in FIG. 6. DETAILED DESCRIPTION

[0028] The application will be described in further detail below with specific reference being made to the drawings. Like elements are referred to with like reference numerals throughout the specification. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the application. However, it will be apparent to one skilled in the art that the application can be practiced without the specific details given. In other instances, well-known methods have not been described in detail in order not to unnecessarily obscure the application. Embodiments of the application will support these and other realms of applicability.

[0029] In addition, the features, operations, or characteristics described in the specification can be combined in any suitable manner in various embodiments. Also, each of the steps or actions in the method descriptions can be performed in any suitable order, as would be apparent to one skilled in the art, unless otherwise indicated or required by the specification. Therefore, the order in which the operations are described is not necessarily the order in which the operations are performed, unless otherwise indicated or required by the specification.

[0030] The serial numbers of components in this paper, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any order or technical meaning. The "connection" and "coupling" in this paper include direct and indirect connection (coupling) unless otherwise specified.

[0031] Highly reflective or transparent objects are widely used in industrial production practice, such as smooth car bodies and their rearview mirrors, smooth glass surfaces, and glass substrates used as flat panel displays, etc. The light beams projected onto the surfaces of such objects will be specularly reflected, so they are often referred to as specular objects. The specular reflection produced by specular objects brings difficulties to non-destructive measurement of their surface shape in the manufacturing process. For non-destructive detection of specular surface shape, the following problems mainly exist:

[0032] (1) Difficulty in presenting defects, which cannot be fully presented or photographed from a certain direction;

[0033] (2) Difficulty in presenting two-dimensional images of some defects, especially for objects with black background and low contrast surface;

[0034] (3) Some specular or quasi-specular objects have unique high light transmission and invisibility, which are the focus and difficulty in the field of optical three-dimensional detection.

[0035] Current three-dimensional measurement of surface shape of specular objects using machine vision mainly adopts interference method. The interference method is generally used for measuring objects with regular surface shape and has very high measurement accuracy, but it usually requires complex and expensive compensation optical system and strict stable environment, and has limited measurement range and poor universality.

[0036] Three-dimensional measurement of object surface based on machine vision is usually realized by a three-dimensional measurement system. Please refer to Figure 1 In an embodiment, the three-dimensional measurement system includes a projection device 1, an object plane 2 and a camera 3. The projection device 1 can be any device that can emit light to form a pattern on the object plane 2, such as an LCD (Liquid Crystal Display), an LED (Light-Emitting Diode) display screen, etc. The object plane 2 is a platform for carrying the object to be measured, such as a stage, etc. The camera 3 can be a CCD (Charge-coupled Device) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, etc. Assuming that the object plane 2 produces specular reflection, the light emitted by the projection device 1 is specularly reflected by the object plane 2 and forms an image on the imaging plane of the camera 3. Please refer to Figure 1 At this time, only the virtual image of the projection device 1 can be observed on the imaging plane of the camera 3, which is equivalent to a virtual projection device 1', and the projection device 1 and the projection device 1' are mirror-symmetric about the object plane 2.

[0037] Three-dimensional measurement using the three-dimensional measurement system can be performed using grating projection and other methods. The three-dimensional measurement method based on grating projection projects a certain regular grating fringe onto the object surface, analyzes the acquired fringe image as a carrier of three-dimensional information, and obtains the surface information of the object, such as height information, etc. based on visual principles.

[0038] Before using the above three-dimensional measurement system to measure the object, the positional relationship between the projection device 1, the object plane 2 and the camera 3 needs to be calibrated, so that the points on the projection device 1, the object plane 2 and the camera 3 can be converted to the same coordinate system for calculation. The present application provides a method for calibrating the positional relationship in the above three-dimensional measurement system, wherein the positional relationship is represented by the transformation relationship of the coordinate systems, including the transformation relationship of the projection device coordinate system and the camera coordinate system, and the transformation relationship of the object plane coordinate system and the camera coordinate system. Please refer to Figure 1The projection device coordinate system is a spatial coordinate system established on the projection device 1, the object plane coordinate system is a spatial coordinate system established on the object plane 2, and the camera coordinate system is a spatial coordinate system established on the camera 3. To adapt to specular reflection, this invention also introduces a virtual spatial coordinate system for calibration. The virtual spatial coordinate system is a spatial coordinate system established on the virtual projection device 1′, that is, the virtual spatial coordinate system is a spatial coordinate system obtained by mirroring the projection device coordinate system about the object plane 2. The camera coordinate system is denoted as c, the object plane coordinate system as w, the projection device coordinate system as s, and the virtual spatial coordinate system as v. Figure 1 The origin and axis directions of the coordinate system are shown for illustrative purposes only. In practice, the origin and axis directions can be set according to specific needs, and are not limited here.

[0039] To better understand the technical solution of this invention, camera calibration and grating projection will be introduced below.

[0040] The purpose of camera calibration is to obtain the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients. Current camera calibration methods are mostly designed and calculated based on Zhang Zhengyou's calibration method, and mainly include the following calculation steps:

[0041] (1) Obtain the homography matrix based on the correspondence between the world coordinates and image coordinates of the feature points in the calibration plate;

[0042] (2) Decompose the homography matrix and calculate the initial parameters of the intrinsic or extrinsic parameters;

[0043] (3) The initial parameters are nonlinearly optimized using the LM (Levenberg-Marquardt) algorithm, and the intrinsic parameters, extrinsic parameters and distortion coefficients are iteratively calculated to obtain the final calibration results.

[0044] The projection transformation relationships between different coordinate systems during the camera's imaging process can be represented using a pinhole camera model, such as... Figure 2 As shown. Point P in the World Coordinate System (WCS). w To obtain point P, the projection of the lens center onto the imaging plane... w Image coordinates q projected onto the imaging plane i First, it needs to be transformed into the Camera Coordinate System (CCS). The x-axis and y-axis of the camera coordinate system are parallel to the c-axis and r-axis of the image, respectively, and the z-axis is perpendicular to the imaging plane of the image. The direction of the z-axis will be set so that the z-coordinate of all points in front of the camera is positive. The c-axis of the image is horizontal, and the r-axis is vertical.Figure 2 Chinese x c axis, y c axis and z c The axes represent the x-axis, y-axis, and z-axis of the camera coordinate system, respectively. The transformation from the world coordinate system to the camera coordinate system can be achieved using the formula p. c = c H w p w Let p represent this, where p c =(x c ,y c ,z c ) T p represents the coordinates in the camera coordinate system. w =(x w ,y w ,z w ) T These are the coordinates in the world coordinate system. c H w It can be represented by a rotation matrix R and a translation vector t.

[0045] After transforming the world coordinate system to the camera coordinate system, it needs to be transformed to the image plane coordinate system, which is a process of transforming 3D coordinates to 2D coordinates. This transformation can be represented as:

[0046]

[0047] Where f represents the focal length of the camera lens, (u,v) T Represents coordinates in a planar coordinate system.

[0048] After being projected onto the imaging plane, the lens distortion will cause the coordinate q to... c =(u,v) T The changes result in distorted coordinates on the imaging plane. This change can be modeled independently on the image plane, meaning three-dimensional information is not required. For most lenses, their distortion can be adequately approximated as radial distortion. There are typically two models used to describe this distortion: a division model and a polynomial model. The division model is as follows:

[0049]

[0050] The parameter κ represents the radial distortion level. If κ is negative, it becomes barrel distortion; if κ is positive, it becomes pincushion distortion. The distortion can be corrected using the following formula:

[0051]

[0052] The polynomial model is as follows:

[0053]

[0054] where k1, k2, k3, p1, p2 are model coefficients. According to the above model, the non-distorted coordinates q can be solved by Newton method given the distortion coefficients and the distorted coordinates c = (u, v) T .

[0055] Finally, the image plane coordinates are converted to image coordinates (ICS, Image Coordinate System) by the following equation:

[0056]

[0057] where s x and s y are the pixel size in horizontal and vertical direction respectively, (c x , c y ) is the principal point, usually the center of the image.

[0058] Therefore, the whole transformation can be represented as:

[0059]

[0060] This is the mathematical model for camera calibration. Among them is the intrinsic part of the camera, and the rotation matrix R and the translation vector t are the extrinsic part. Further simplification can be represented as:

[0061] sm= A[R|t]M, (1)

[0062] where A is the intrinsic matrix of the camera,

[0063] For grating projection, any existing grating projection method can be used. In one embodiment, Gray code images and phase shift images can be projected. Please refer to Figure 3 , which are Gray code images and phase shift images with a width of 32 pixels projected by an embodiment. The image sequence with serial numbers 1-4 is a Gray code image, and the image sequence with serial numbers 5-8 is a phase shift image.

[0064] Since the phase shift image has periodicity, the acquired phase is in the range of [0, 2π], which needs to be converted to the absolute phase of 2kπ (k is an integer). After obtaining the absolute phase image, combined with the calibrated position relationship, three-dimensional data can be generated.

[0065] Phase shifting method is widely used in optical measurement, because of its high precision and speed of measurement, so for high precision of three-dimensional measurement of objects, generally using phase shifting method. In the phase shifting method, the process of obtaining the phase is as follows: (1) first by phase shifting method formula to obtain the sawtooth phase value, the value range is [-π, π], called the truncated phase; (2) the sawtooth phase value is restored to continuous phase value, called absolute phase. This process is called phase unwrapping (or phase unwrapping, phase unwrapping).

[0066] There are many methods to obtain phase value by phase shifting method, for example, can use N step phase shifting method to solve. If the projection light intensity is standard cosine distribution, then the phase shifting image is moved 2π / N each time, a new light intensity function I n (x, y) is obtained, where (x, y) is the coordinates of the pixel points in the phase shifting image. Among them, four step phase shifting method is more commonly used, because this method can eliminate the nonlinear effect of the detector. Four step phase shifting method is to move the projected phase shifting image each time π / 2, three times. Figure 3 The phase shifting image shown is a four step phase shifting image. The light intensity function of four step phase shifting can be expressed as:

[0067]

[0068] Where I i (i = 1, 2, 3, 4) is the gray value of the i-th phase shifting image, I'(x, y) is the background value of the fringe light intensity, that is, the original light intensity emitted by the projection device 1, I''(x, y) is the modulation light intensity value, Is the phase value to be solved. It can be obtained that:

[0069]

[0070] The phase calculated by the phase shifting method is only the principal value of the phase, which contains an inverse tangent function, and the value range is [-π, π], and the phase is discontinuous. In view of this problem, since there is a difference of 2kπ between the phase obtained by the phase shifting method formula and the true value, therefore, k must be solved, so as to restore the principal value of the phase to the true absolute phase. Therefore, the complete phase value, that is, the absolute phase formula should be:

[0071]

[0072] Where k(x, y) is an integer, representing the integer multiple of 2π corresponding to the pixel point (x, y). It can be seen that the key to phase unwrapping is to determine the decoding period k(x, y). In practical application, k(x, y) represents the period number of the grating fringe pattern where the pixel point (x, y) is located, that is, which fringe does the pixel point (x, y) belong to, and which period does the fringe belong to. For example, please refer to Figure 3It can be seen that the phase-shifted images are arranged periodically, the 0th-3rd pixels are in the first period, the 4th-7th pixels are in the second period, and so on. The period number of the grating stripe in which the pixel point is located can be obtained according to the Gray code encoding information of the grating stripe.

[0073] The calibration method of the positional relationship in the three-dimensional measurement system of the application will be introduced below. Please refer to Figure 4 In an embodiment, the method comprises steps 100-600, which will be described below.

[0074] Step 100: Obtain the calibration board image of the object plane.

[0075] The calibration board image of the object plane is an image captured by the camera 3 when the projection device 1 projects the calibration board pattern on the object plane 2. The calibration board image can be a checkerboard image, a circular array image, etc. In order to prevent the influence of mirror reflection on the calibration of the camera internal and external parameters, a piece of white paper can be placed on the object plane 2, so that the calibration board pattern is projected on the white paper, avoiding mirror reflection.

[0076] Step 200: Take the object plane coordinate system as the world coordinate system, and calibrate the internal and external parameters of the camera according to the calibration board image. Since the external parameter represents the transformation relationship between the world coordinate system and the camera coordinate system, the obtained external parameter also represents the transformation relationship between the object plane coordinate system and the camera coordinate system.

[0077] Please refer to Figure 5 In an embodiment, step 200 comprises steps 210-240.

[0078] Step 210: Obtain the feature points in the calibration board image, as well as the image coordinates and corresponding world coordinates of the feature points.

[0079] For a checkerboard, the feature points are the corners of the checkerboard, and for a circular array, the feature points are the centers of the circular feature points in the circular array, i.e. the circular patterns in the circular array. The world coordinates of the feature points can be obtained by constructing a world coordinate system according to the parameter information of the calibration board pattern, which can include the size of the calibration board, the size of the checkerboard, the radius of the circular feature points, the distance between the feature points, etc. The acquisition of the feature points in the calibration board image and the image coordinates of the feature points can be realized by the existing technology, which will not be described here.

[0080] Step 220: Calculate the homography matrix according to the image coordinates and corresponding world coordinates of the feature points. It can be understood that the image coordinates p i and the corresponding world coordinates p w of multiple feature points, and the transformation relationship p i = H pw , the objective function is established: min∑[p i -Hp w ] 2 The homography matrix H can be calculated by using the least square method. The elements in the homography matrix H are represented by h0, h1, h2, h3, h4, h5, h6, h7, and h8. Then

[0081] Step 230: According to the constraint relationship between the homography matrix H and the camera intrinsic parameters, the intrinsic parameters of the camera are calculated by using the homography matrix H.

[0082] Let the equivalent focal length f x =f / s x , f y =f / s y , then the intrinsic part can be represented as When establishing the world coordinate system, it is generally considered that the points on the calibration board are located on the plane z=0, so the rotation and translation in the z direction can be ignored, and thus the extrinsic part can be represented as where r1, r2, r3, r4, r5, and r6 are elements of the rotation matrix, t x and t y are the x component and y component of the translation vector respectively. Therefore, we have

[0083]

[0084] If the origin of the image coordinate system is set as the image center, then we can get

[0085]

[0086] From the orthogonality constraint of the vectors in the rotation matrix, we can get:

[0087]

[0088] According to the orthogonality and unitary constraints, the constraint relationship between the homography matrix H and the camera intrinsic parameters can be obtained:

[0089]

[0090] where Then according to the above constraint relationship, the equivalent focal length f x and f y can be calculated from the homography matrix H, and the camera principal axis point coordinates (c x , c y ) can be obtained from the camera manual.

[0091] Step 240: According to the homography matrix H, the rotation matrix R and the translation vector t are calculated.

[0092] From formula (1), H=A[R|t], according to orthogonality, the following can be obtained:

[0093] H=[h1 h2 h3]=A[r1 r2 t],

[0094] Where [r1 r2 t]=[R|t], h1 is the first column vector of the homography matrix H, h2 is the second column vector of the homography matrix H, h3 is the third column vector of the homography matrix H, r1 is the first column vector of the rotation matrix R, and r2 is the second column vector of the rotation matrix R. The parameter matrix A can be calculated according to the following constraint condition:

[0095]

[0096] The vectors r1 and r2 can be calculated according to r1=A -1 h1, r2=A -1 h2, then the rotation matrix R=[r1 r2], and the translation vector t can be calculated according to t=A -1 h3, so that the extrinsic part is obtained.

[0097] Then the transformation relationship between the object plane coordinate system and the camera coordinate system can be represented by the rotation matrix R and the translation vector t, which can be denoted as And

[0098] Step 300: Transform the object plane into several poses, obtain the calibration board image or the grating projection image of the object plane under each pose, obtain the coordinates of the pixel points in the camera coordinate system in the calibration board image or the grating projection image, and the coordinates of the corresponding points in the virtual space coordinate system.

[0099] In order to make the calibration result more accurate, the transformation relationship between the projection device coordinate system and the camera coordinate system is calibrated by randomly transforming the object plane 2 into multiple poses, and the calibration is performed with the help of the virtual space coordinate system to adapt to the mirror reflection. In each pose, the projection device 1 projects the calibration board pattern or the grating stripe pattern on the object plane 2, and the camera 3 takes a picture to obtain the calibration board image or the grating projection image of the object plane 2.

[0100] As can be seen from the above, the image coordinates can be transformed into the camera coordinate system by the intrinsic parameters of the camera, so that the coordinates of the pixel points in the camera coordinate system can be obtained according to the image coordinates of the pixel points in the calibration board image or the grating projection image and the intrinsic parameters of the camera. The pixel points of the calibration board image can be feature points in the calibration board image.

[0101] The point in the virtual space coordinate system corresponding to the pixel point, i.e. the point in the virtual space coordinate system observed from the pixel point. For the calibration board image, the world coordinates of the pixel point in the calibration board image can be obtained as the coordinates of the point in the virtual space coordinate system corresponding to the pixel point, which can be obtained by transforming the image coordinates of the pixel point through the intrinsic and extrinsic parameters of the camera. For the grating projection image, the phase information of the pixel point in the grating projection image is obtained by performing phase unwrapping on the grating projection image, and then the screen point coordinates (x s ,y s ) corresponding to the pixel point are obtained according to the phase information of the pixel point; the coordinates p s =(x s ,y s ,0) are taken as the coordinates of the point in the virtual space coordinate system corresponding to the pixel point in the grating projection image. The screen point refers to the point on the screen of the projection device 1, and the screen point coordinates refer to the two-dimensional coordinates of the screen point on the screen. The relationship between the pixel point and the corresponding screen point is that if the light emitted by the screen point A is reflected to obtain the pixel point C in the camera imaging, the screen point A is called the screen point corresponding to the pixel point C. The phase unwrapping can refer to the above introduction or the prior art.

[0102] In an embodiment, the grating projection image can include an X-direction grating projection image and a Y-direction grating projection image. The projection device 1 displays the X-direction (i.e. horizontal direction) grating fringes on its screen, and the grating projection image obtained by the camera 3 shooting the object plane is called the X-direction grating projection image; the projection device 1 displays the Y-direction (i.e. vertical direction) grating fringes on its screen, and the grating projection image obtained by the camera 3 shooting the object plane is called the Y-direction grating projection image.

[0103] For each pixel point, the phase unwrapping of the X-direction grating projection image can obtain a phase, which is called the X-phase, and the phase unwrapping of the Y-direction grating projection image can also obtain a phase, which is called the Y-phase, and the complete screen point coordinates (x s ,y s ) can be obtained by using the X-phase and the Y-phase. Specifically, first, the X-direction grating projection image is phase unwrapped to obtain the X-phase of each pixel point, and then the Y-direction grating projection image is phase unwrapped to obtain the Y-phase of each pixel point. Then for each pixel point, the corresponding screen point coordinates (x s ,y s ) are calculated according to the X-phase and the Y-phase , and the calculation formula is as follows:

[0104]

[0105] where T x represents the period number of the grating stripe that the pixel point in the X direction grating projection image locates in, T y represents the period number of the grating stripe that the pixel point in the Y direction grating projection image locates in.

[0106] Step 400: for each pose of the object plane, according to the coordinates of the pixel points in the camera coordinate system and the coordinates of the corresponding points in the virtual space coordinate system in the calibration board image or the grating projection image under the pose, the transformation relationship between the virtual space coordinate system and the camera coordinate system under the pose is calculated.

[0107] The transformation relationship between the virtual space coordinate system and the camera coordinate system can be calculated by using the coordinates of the multiple groups of pixel points in the camera coordinate system and the coordinates of the corresponding points in the virtual space coordinate system. In an embodiment, the transformation relationship between the virtual space coordinate system and the camera coordinate system can be represented by a rotation matrix R and a translation vector t , the point in the virtual space coordinate system can be represented as X , the point in the camera coordinate system can be represented as X , and the transformation relationship can be established as:

[0108]

[0109] where k represents the serial number of the pixel point, i represents the i-th pose, and i = 1, 2, …, N c , N c represents the number of poses, represents the coordinates of the k-th pixel point in the camera coordinate system under the i-th pose, represents the coordinates of the point in the virtual space coordinate system corresponding to the k-th pixel point under the i-th pose, represents the rotation matrix of the virtual space coordinate system and the camera coordinate system under the i-th pose, represents the translation vector of the virtual space coordinate system and the camera coordinate system under the i-th pose.

[0110] According to the above formula, the following objective function can be established, and the least square method is used to calculate the transformation relationship between the virtual space coordinate system and the camera coordinate system under each pose:

[0111]

[0112] where N represents the number of pixel points in the calibration board image or the grating projection image.

[0113] Step 500: according to the geometric relationship between the virtual space coordinate system and the projection device coordinate system, the transformation relationship between the projection device coordinate system and the camera coordinate system is calculated by using the transformation relationship between the virtual space coordinate system and the camera coordinate system under all poses.

[0114] In an embodiment, the transformation relationship between the projection device coordinate system and the camera coordinate system can be represented by a rotation matrix and a translation vector . Please refer to Figure 1 For a point p on the projection device, after reflection on a point on the object plane, the imaging in the camera is a mirror image point p' on the virtual projection device. According to the geometric relationship, the transformation from the virtual space coordinate system to the camera coordinate system can be represented as:

[0115]

[0116] where I is the unit matrix, n is the normal vector at the reflection point on the object plane, and d is the distance from the camera to the object plane, which can be the distance from the optical center of the camera lens to the object plane.

[0117] When the transformation from the virtual space coordinate system to the camera coordinate system is represented by a rotation matrix and a translation vector , we can get:

[0118]

[0119] In an embodiment of the present application, according to the formula, the rotation matrix and the translation vector are used to calibrate the rotation matrix and the translation vector .

[0120] In order to balance the calculation accuracy and efficiency, the object plane can be transformed into three poses in step 300, and the transformation relationship between the projection device coordinate system and the camera coordinate system is calculated using the transformation relationship between the virtual space coordinate system and the camera coordinate system in the three poses.

[0121] For any p, q ∈ {1, 2, 3}, define m p,q = n p × n q , where n i represents the normal vector of the object plane in the i-th pose. Since holds, m p,q is the eigenvector corresponding to the smallest eigenvalue obtained by singular value decomposition of the matrix . According to m p,q = n p × n q , the normal vector n p,q can be obtained from m i .

[0122] Therefore, in step 500 of one embodiment of the present application, singular value decomposition is performed on the matrix of any p, q ∈ {1, 2, 3} to obtain the eigenvector m corresponding to the minimum eigenvalue p,q ; and then the normal vector n is calculated according to the following formula i :

[0123]

[0124] The rotation matrix R between the projection device coordinate system and the camera coordinate system can be calculated according to the formula Here n is the normal vector in any posture, i and R is the rotation matrix between the virtual space coordinate system and the camera coordinate system in the posture.

[0125] The translation vector t between the projection device coordinate system and the camera coordinate system can be calculated according to the following formula

[0126]

[0127] where d1, d2 and d3 are the distances from the camera to the object plane in the three postures. The above formula can be expressed in the form of Ax = b, where Therefore, the solution is x = (A T A) - 1 A T b.

[0128] Thus, the transformation relationship between the projection device coordinate system and the camera coordinate system is obtained.

[0129] Step 600: globally optimizing the transformation relationship between the projection device coordinate system and the camera coordinate system according to a preset global optimization objective function.

[0130] Since the transformation relationship between the projection device coordinate system and the camera coordinate system obtained in step 500 can only be a local optimal solution, this step globally optimizes it to search for a global optimal solution and improve the accuracy of calibration. Global optimization mainly performs iterative optimization of parameters based on a preset global optimization objective function, and the initial value of iteration is the transformation relationship between the projection device coordinate system and the camera coordinate system obtained in step 500. The global optimization objective function can be designed based on the idea of minimizing the error between the actual value and the estimated value. In one embodiment, if the transformation relationship between the projection device coordinate system and the camera coordinate system is represented by the rotation matrix R and the translation vector t, the global optimization objective function can be:

[0131] ​​​​

[0132] where x ik represents the coordinates of the kth pixel point of the image of the calibration board or the image of the grating projection of the object plane under the ith pose in the camera coordinate system, N c represents the number of poses, z ik = f(p i ′ k ), v i = d i n i , n i represents the normal vector of the object plane under the ith pose, d i represents the distance between the object plane and the camera under the ith pose, p ik represents the coordinates of the point in the projection device coordinate system corresponding to x ik , p i ′ k is the coordinates of the virtual image point of p ik observed in the imaging plane of the camera in the camera coordinate system, z ik represents the coordinates of the point p i ′ k after the non-linear mapping f of the camera, is a vector composed of the parameters to be optimized.

[0133] where x ik can be obtained according to the image coordinates of the pixel points in the image of the calibration board or the image of the grating projection and the intrinsic parameters of the camera. p ik can be obtained according to the transformation relationship between the projection device coordinate system and the camera coordinate system that has been obtained, i.e., according to the rotation matrix and the translation vector . The point p ik Since the specular reflection of the object plane forms a virtual image on the imaging plane of the camera, the true point p ik cannot be seen from the imaging plane of the camera, but its virtual image point p i ′ k , the virtual image point p i ′ k is mirror-symmetric to the point p ik about the object plane, and p i ′ k calculated here is the coordinates converted to the camera coordinate system. The non-linear mapping f refers to the distortion of the camera lens.

[0134] According to the above global optimization objective function, the parameters to be optimized are iteratively optimized, and the final transformation relationship between the projection device coordinate system and the camera coordinate system is obtained, thereby completing the calibration of the positional relationship in the three-dimensional measurement system.

[0135] On the basis of the method for calibrating the positional relationship in the three-dimensional measurement system, the application further provides a system for calibrating the positional relationship in a three-dimensional measurement system Figure 6 In an embodiment, the system comprises a calibration board image acquisition module 1, a first calibration module 2, a multi-pose coordinate acquisition module 3, a second calibration module 4, a third calibration module 5 and a global optimization module 6, which are described below respectively.

[0136] The calibration board image acquisition module 1 is used to acquire the calibration board image of the object plane, which is the image captured by the camera when the projection device projects the calibration board pattern on the object plane.

[0137] The first calibration module 2 is used to calibrate the intrinsic and extrinsic parameters of the camera according to the calibration board image, with the object plane coordinate system as the world coordinate system. Since the extrinsic parameter represents the transformation relationship between the world coordinate system and the camera coordinate system, the obtained extrinsic parameter also represents the transformation relationship between the object plane coordinate system and the camera coordinate system.

[0138] In an embodiment, the intrinsic parameters of the camera include the equivalent focal lengths f x and f y , and the extrinsic parameters include a rotation matrix R and a translation vector t. The first calibration module 2 is specifically used to acquire the feature points in the calibration board image, as well as the image coordinates and corresponding world coordinates of the feature points; calculate the homography matrix H from the image coordinates and corresponding world coordinates of the feature points; calculate the equivalent focal lengths f of the camera from the homography matrix H according to the following constraint relationship: x y

[0139]

[0140] wherein The rotation matrix R and the translation vector t are calculated from the homography matrix H.

[0141] In an embodiment, the first calibration module 2 is specifically used to calculate the parameter matrix A according to the following constraint condition:

[0142]

[0143] wherein h1 is the first column vector of the homography matrix H, and h2 is the second column vector of the homography matrix H; the vectors r1 and r2 are calculated according to r1=A - 1 h1 and r2=A -1 h2, and then the rotation matrix R=[r1 r2]; the translation vector t is calculated according to t=A -1 h3, wherein h3 is the third column vector of the homography matrix H.

[0144] ​​The multi-pose coordinate acquisition module 3 is configured to transform the object plane into a plurality of poses, acquire an image of a calibration board or a grating projection image of the object plane in each pose, acquire coordinates of a pixel point in the image of the calibration board or the grating projection image in a camera coordinate system, and coordinates of a point in a corresponding virtual space coordinate system. The grating projection image of the object plane is an image of the object plane captured by the camera when the projection device performs grating projection on the object plane, and the virtual space coordinate system is a space coordinate system obtained by mirror symmetry of the projection device coordinate system with respect to the object plane.

[0145] In an embodiment, the multi-pose coordinate acquisition module 3 is specifically configured to: obtain coordinates of a pixel point in a camera coordinate system according to an image coordinate of the pixel point in the image of the calibration board or the grating projection image and an intrinsic parameter of the camera; for the image of the calibration board, acquire a world coordinate corresponding to the pixel point in the image of the calibration board as the coordinates of the point in the corresponding virtual space coordinate system; and for the grating projection image, perform phase unwrapping processing on the grating projection image to obtain phase information of the pixel point in the grating projection image, and acquire a screen point coordinate (x s ,y s ) corresponding to the pixel point according to the phase information of the pixel point, wherein the screen point coordinate is taken as the coordinates of the point in the corresponding virtual space coordinate system. s s s

[0146] In an embodiment, the grating projection image can include an X-direction grating projection image and a Y-direction grating projection image. The X-direction grating projection image is an image of the grating projection captured by the camera on the object plane when the projection device displays grating fringes along the X direction (i.e., the horizontal direction) on the screen of the projection device; and the Y-direction grating projection image is an image of the grating projection captured by the camera on the object plane when the projection device displays grating fringes along the Y direction (i.e., the vertical direction) on the screen of the projection device. The multi-pose coordinate acquisition module 3 is further configured to: perform phase unwrapping processing on the X-direction grating projection image to obtain an X phase perform phase unwrapping processing on the Y-direction grating projection image to obtain a Y phase for each pixel point, calculate a corresponding screen point coordinate (x s ,y s ) according to the X phase and the Y phase of the pixel point, and the calculation formula is as follows:

[0147]

[0148] wherein T x ​​​T represents the period number of the grating stripe in which the pixel point in the X direction grating projection image is located y T represents the period number of the grating stripe in which the pixel point in the Y direction grating projection image is located.

[0149] The second calibration module 4 is configured to calculate, for each pose of the object plane, a transformation relationship between the virtual space coordinate system and the camera coordinate system according to coordinates of the pixel points in the calibration board image or the grating projection image in the camera coordinate system and coordinates of the corresponding points in the virtual space coordinate system in the pose.

[0150] In an embodiment, the transformation relationship between the virtual space coordinate system and the camera coordinate system includes a rotation matrix and a translation vector The second calibration module 4 is specifically configured to calculate, for each pose, the transformation relationship between the virtual space coordinate system and the camera coordinate system by using a least square method according to the following objective function:

[0151]

[0152] wherein k represents the serial number of the pixel point, N represents the number of the pixel points in the calibration board image or the grating projection image, i represents the i-th pose, and i = 1, 2, …, N c , N c represents the number of the poses, represents the coordinates of the k-th pixel point in the camera coordinate system in the i-th pose, represents the coordinates of the point in the virtual space coordinate system corresponding to the k-th pixel point in the i-th pose, represents the rotation matrix of the virtual space coordinate system and the camera coordinate system in the i-th pose, represents the translation vector of the virtual space coordinate system and the camera coordinate system in the i-th pose.

[0153] The third calibration module 5 is configured to calculate, according to the geometric relationship between the virtual space coordinate system and the projection device coordinate system, the transformation relationship between the projection device coordinate system and the camera coordinate system by using the transformation relationship between the virtual space coordinate system and the camera coordinate system in all poses.

[0154] In an embodiment, the transformation relationship between the projection device coordinate system and the camera coordinate system includes a rotation matrix and a translation vector The number of the poses of the object plane is three, and the third calibration module 5 is specifically configured to, for any p, q ∈ {1, 2, 3}, perform singular value decomposition on the matrix to obtain a feature vector m p,q corresponding to the minimum eigenvalue; and calculate a normal vector n i according to the following formula:

[0155]

[0156] wherein n i represents the normal vector of the object plane under the i-th pose;

[0157] According to the formula the rotation matrix of the projection device coordinate system and the camera coordinate system is calculated wherein I is the unit matrix, n i is the normal vector under any pose, is the rotation matrix of the virtual space coordinate system and the camera coordinate system under the pose; and the translation vector of the projection device coordinate system and the camera coordinate system is calculated according to the following formula

[0158]

[0159] wherein d1, d2 and d3 are the distances between the object plane and the camera under the three poses respectively.

[0160] The global optimization module 6 is configured to globally optimize the transformation relationship between the projection device coordinate system and the camera coordinate system according to a preset global optimization objective function.

[0161] In an embodiment, the transformation relationship between the projection device coordinate system and the camera coordinate system includes a rotation matrix and a translation vector The global optimization objective function can be:

[0162]

[0163] wherein x ik represents the coordinates of the k-th pixel point of the calibration board image or the raster projection image of the object plane under the i-th pose in the camera coordinate system, Nc represents the number of poses, z ik = f(p i ′ k ), v i = d i n i , n i represents the normal vector of the object plane under the i-th pose, d i represents the distance between the object plane and the camera under the i-th pose, p ik represents the coordinates of the point in the projection device coordinate system corresponding to x ik , p i ′ k is the coordinates of the virtual image point of p ik observed in the imaging plane of the camera in the camera coordinate system, z ik represents the point p i ′ kcoordinates obtained through a non-linear mapping f of the camera, is a vector composed of the parameters to be optimized.

[0164] where x ik The image coordinates of the pixel points in the calibration board image or the raster projection image and the intrinsic parameters of the camera can be used to obtain p. ik The transformation relationship between the projection device coordinate system and the camera coordinate system can be obtained according to the obtained p, i.e., according to the rotation matrix R and the translation vector t. The point p ik Since the mirror reflection of the object plane forms a virtual image on the imaging plane of the camera, the true point p ik cannot be seen from the imaging plane of the camera, but a virtual image point p i ′ k The virtual image point p i ′ k is symmetrical to the point p ik with respect to the object plane. The p i ′ k calculated here is the coordinate converted to the camera coordinate system. The non-linear mapping f refers to the distortion of the camera lens.

[0165] The global optimization module 6 iteratively optimizes the parameters to be optimized according to the above global optimization objective function, and obtains the final transformation relationship between the projection device coordinate system and the camera coordinate system, thereby completing the calibration of the positional relationship in the three-dimensional measurement system.

[0166] The calibration method and system of the positional relationship in the three-dimensional measurement system according to the above embodiment are used to calibrate the positional relationship between the projection device, the object plane and the camera in the three-dimensional measurement system, wherein the positional relationship is represented by the transformation relationship between the projection device coordinate system and the camera coordinate system and the transformation relationship between the object plane coordinate system and the camera coordinate system. In the calibration process, a virtual space coordinate system is introduced, which is a space coordinate system obtained by mirror symmetry of the projection device coordinate system with respect to the object plane. The transformation relationship between the virtual space coordinate system and the camera coordinate system is first calibrated, and then the transformation relationship between the virtual space coordinate system and the camera coordinate system is used to calibrate the positional relationship between the projection device and the camera, so as to adapt to subsequent three-dimensional measurement of the mirror object. When calibrating the positional relationship between the projection device and the camera, the transformation relationship between the virtual space coordinate system and the camera coordinate system of the object plane in multiple poses is used for calculation, which is conducive to obtaining more accurate estimation values and avoiding falling into a local optimal solution with a large deviation from the true value. Finally, the obtained estimation values are used as initial values for global optimization, thereby improving the accuracy of the calibration and providing a good foundation for subsequent three-dimensional measurement of the object.

[0167] ​Those skilled in the art can understand that all or part of the functions of various methods in the above embodiments can be realized by hardware or by a computer program. When all or part of the functions in the above embodiments are realized by a computer program, the program can be stored in a computer readable storage medium, which can include a read-only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, and the like. The above functions are realized by executing the program by a computer. For example, the program is stored in a memory of a device, and the above functions are realized by executing the program in the memory by a processor. In addition, when all or part of the functions in the above embodiments are realized by a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk, or a mobile hard disk, and is saved in a memory of a local device by downloading or copying, or the system of the local device is updated, and the above functions are realized by executing the program in the memory by a processor.

[0168] The above application of specific examples to the present application is described, which is only used to help understand the present application and does not limit the present application. For those skilled in the art, according to the idea of the present application, a number of simple deductions, deformations or substitutions can be made.

Claims

1. A method of calibrating positional relationships in a three-dimensional measurement system comprising a projection device, an object plane and a camera, characterized in that The position relationship comprises a transformation relationship between the projection device coordinate system and the camera coordinate system and a transformation relationship between the object plane coordinate system and the camera coordinate system, and the calibration method comprises the following steps: An image of a calibration board of the object plane is acquired, the image of the calibration board of the object plane being an image captured by the camera on the object plane when the projection device projects a calibration board pattern on the object plane; The object plane coordinate system is taken as a world coordinate system, and intrinsic and extrinsic parameters of the camera are calibrated according to the image of the calibration board, the extrinsic parameters representing a transformation relationship between the world coordinate system and the camera coordinate system; The object plane is transformed into several poses, an image of the calibration board of the object plane or a grating projection image of the object plane is acquired under each pose, coordinates of a pixel point in the camera coordinate system and coordinates of a corresponding point in a virtual space coordinate system in the image of the calibration board or the grating projection image are acquired, the grating projection image of the object plane being an image captured by the camera when the projection device performs grating projection on the object plane, and the virtual space coordinate system being a space coordinate system obtained by mirror symmetry of the projection device coordinate system with respect to the object plane; For each pose of the object plane, a transformation relationship between the virtual space coordinate system and the camera coordinate system under the pose is calculated according to the coordinates of the pixel point in the camera coordinate system and the coordinates of the corresponding point in the virtual space coordinate system in the image of the calibration board or the grating projection image under the pose; A transformation relationship between the projection device coordinate system and the camera coordinate system is calculated according to a geometric relationship between the virtual space coordinate system and the projection device coordinate system and the transformation relationships between the virtual space coordinate systems and the camera coordinate systems under all the poses; The transformation relationship between the projection device coordinate system and the camera coordinate system is globally optimized according to a preset global optimization objective function.

2. The calibration method of claim 1, wherein, Intrinsic parameters of the camera include an equivalent focal length f x and f y Extrinsic parameters of the camera include a rotation matrix R and a translation vector t; The object plane coordinate system is taken as a world coordinate system, and intrinsic and extrinsic parameters of the camera are calibrated according to the image of the calibration board, the extrinsic parameters representing a transformation relationship between the world coordinate system and the camera coordinate system; Feature points in the image of the calibration board, image coordinates of the feature points and corresponding world coordinates of the feature points are acquired; A homography matrix is calculated according to the image coordinates of the feature points and the corresponding world coordinates of the feature points The equivalent focal length f of the camera is calculated from the homography matrix H according to the following constraint relation x and f y : wherein A rotation matrix R and a translation vector t are calculated according to the homography matrix H.

3. The calibration method of claim 2, wherein, The rotation matrix R and the translation vector t are calculated according to the homography matrix H, and the calculation comprises the following steps: A parameter matrix A is calculated according to the following constraint condition: wherein h1 is a first column vector of the homography matrix H, and h2 is a second column vector of the homography matrix H; According to r1 = A -1 h1, r2 = A -1 h2The vectors r1 and r2 are calculated, and the rotation matrix R = [r1 r2]; According to t = A -1 h3a translation vector t is calculated, where h3is the third column vector of the homography matrix H.

4. The calibration method of claim 1, wherein, The coordinates of the pixel point in the camera coordinate system and the coordinates of the corresponding point in the virtual space coordinate system are acquired, and the acquisition comprises the following steps: The coordinates of the pixel point in the camera coordinate system are obtained according to image coordinates of the pixel point in the image of the calibration board or the grating projection image and the intrinsic parameters of the camera; For the image of the calibration board, the world coordinates of the pixel point in the image of the calibration board are taken as the coordinates of the corresponding point in the virtual space coordinate system. For the grating projection image, the grating projection image is subjected to phase unwrapping to obtain phase information of a pixel point in the grating projection image, and a screen point coordinate (x s ,y s ) corresponding to the pixel point is obtained according to the phase information of the pixel point, wherein the screen point refers to a point on a screen of the projection device, and the screen point coordinate refers to a two-dimensional coordinate of the screen point on the screen; and the coordinate p s =(x s ,y s ,0) is taken as a coordinate of a point in a virtual space coordinate system corresponding to the pixel point in the grating projection image.

5. The calibration method of claim 4, wherein, The grating projection image comprises an X-direction grating projection image and a Y-direction grating projection image, the X-direction grating projection image being a grating projection image obtained by the camera when the projection device displays grating fringes along the X direction on its screen and the camera photographs the object plane, and the Y-direction grating projection image being a grating projection image obtained by the camera when the projection device displays grating fringes along the Y direction on its screen and the camera photographs the object plane. The phase-unwrapping processing is performed on the fringe projection image to obtain phase information of a pixel point in the fringe projection image, and screen point coordinates (x s ,y s ) corresponding to the pixel point are obtained according to the phase information of the pixel point. s s ) corresponding to the pixel point are obtained according to the phase information of the pixel point. performing an unwrapping process on the X-directional fringe projection image to obtain an X phase of each pixel point performing an unwrapping process on the Y-directional fringe projection image to obtain a Y phase of each pixel point For each pixel point, according to its X phase and Y phase , the corresponding screen point coordinates (x s , y s ) are calculated, and the calculation formula is as follows: wherein T x represents the period number of the grating stripe in which the pixel point is located in the grating projection image in the X direction, T y represents the period number of the grating stripe in which the pixel point is located in the grating projection image in the Y direction.

6. The calibration method of claim 1, wherein, The transformation relationship between the virtual space coordinate system and the camera coordinate system includes a rotation matrix and a translation vector For each pose of the object plane, according to the coordinates of the pixel points in the camera coordinate system in the calibration plate image or the raster projection image under the pose, and the coordinates of the corresponding points in the virtual space coordinate system, the transformation relationship between the virtual space coordinate system and the camera coordinate system under the pose is calculated, including: The transformation relationship between the virtual space coordinate system and the camera coordinate system in each pose is calculated by using a least square method according to a following objective function: wherein k represents the serial number of the pixel point, N represents the number of pixel points in the calibration board image or the raster projection image, i represents the i-th pose, and i = 1, 2, …, N c , N c represents the number of poses, represents the coordinates of the k-th pixel point in the camera coordinate system under the i-th pose, represents the coordinates of the point corresponding to the k-th pixel point in the virtual space coordinate system under the i-th pose, represents the rotation matrix of the virtual space coordinate system and the camera coordinate system under the i-th pose, represents the translation vector of the virtual space coordinate system and the camera coordinate system under the i-th pose.

7. The calibration method of claim 6, wherein, The transformation relationship between the projection device coordinate system and the camera coordinate system comprises a rotation matrix and a translation vector The number of the postures is three; and the transformation relationship between the projection device coordinate system and the camera coordinate system is calculated according to the geometric relationship between the virtual space coordinate system and the projection device coordinate system, and by using the transformation relationship between the virtual space coordinate system and the camera coordinate system in all postures, comprising: For any p, q e {1, 2, 3}, singular value decomposition is performed on the matrix to obtain the eigenvector m p,q corresponding to the minimum eigenvalue. The normal vector n is calculated according to the following formula i where n i denotes the normal vector of the object plane at the i-th pose: According to the formula The rotation matrix of the projection device coordinate system and the camera coordinate system is calculated Where I is a unit matrix, n i is the normal vector under any one pose, is the rotation matrix of the virtual space coordinate system and the camera coordinate system under the pose; The translation vector between the projection device coordinate system and the camera coordinate system is calculated according to the following formula Wherein d1, d2 and d3 are distances between the object plane and the camera in the three poses respectively.

8. The calibration method of any one of claims 1 to 7, wherein, The global optimization objective function is: where x ik represents the coordinates of the kth pixel point in the i th pose in the camera coordinate system, N c represents the number of poses, z ik = f(p i ′ k ), represents the rotation matrix of the projection device coordinate system and the camera coordinate system, represents the translation vector of the projection device coordinate system and the camera coordinate system, v i = d i n i , n i represents the normal vector of the object plane in the i th pose, d i represents the distance between the object plane and the camera in the i th pose, p ik represents the coordinates of the point in the projection device coordinate system corresponding to x ik , p i ′ k is the coordinates of the virtual image point of p ik observed in the imaging plane of the camera in the camera coordinate system, z ik represents the coordinates of the point p i ′ k after the nonlinear mapping f of the camera, is a vector composed of the parameters to be optimized.

9. A system for calibrating positional relationships in a three-dimensional measurement system, the three-dimensional measurement system comprising a projection device, an object plane and a camera, characterized in that The position relationship comprises a transformation relationship between the projection device coordinate system and the camera coordinate system and a transformation relationship between the object plane coordinate system and the camera coordinate system, and the calibration system comprises: A calibration board image acquisition module configured to acquire a calibration board image of the object plane, the calibration board image of the object plane being an image photographed by the camera when the projection device projects a calibration board pattern on the object plane; A first calibration module configured to calibrate intrinsic parameters and extrinsic parameters of the camera according to the calibration board image, the extrinsic parameters representing a transformation relationship between a world coordinate system and the camera coordinate system, and the object plane coordinate system being taken as the world coordinate system; A multi-pose coordinate acquisition module configured to transform the object plane into a plurality of poses, acquire a calibration board image or a grating projection image of the object plane in each pose, acquire coordinates of a pixel point in the calibration board image or the grating projection image in the camera coordinate system and coordinates of a corresponding point in a virtual space coordinate system, the grating projection image of the object plane being an image photographed by the camera when the projection device performs grating projection on the object plane, and the virtual space coordinate system being a space coordinate system obtained by mirror symmetry of the projection device coordinate system with respect to the object plane; A second calibration module configured to calculate, for each pose of the object plane, a transformation relationship between the virtual space coordinate system and the camera coordinate system according to the coordinates of the pixel point in the calibration board image or the grating projection image in the camera coordinate system and the coordinates of the corresponding point in the virtual space coordinate system in the pose; A third calibration module configured to calculate a transformation relationship between the projection device coordinate system and the camera coordinate system according to a geometric relationship between the virtual space coordinate system and the projection device coordinate system and by using the transformation relationship between the virtual space coordinate system and the camera coordinate system in all the poses; A global optimization module configured to globally optimize the transformation relationship between the projection device coordinate system and the camera coordinate system according to a preset global optimization objective function.

10. A computer-readable storage medium, characterized in that, The medium has a program stored thereon, and the program can be executed by the processor to implement the calibration method according to any one of claims 1-8.

Images