Fast Calibration Method for Structured Light System for Industrial Site

Abstract

The present invention discloses a fast calibration method for a structured light system for industrial sites, comprising the following steps: respectively calibrating the internal parameters of the projector and the camera, fixing the positions of the projector and the camera, and placing the calibration board within the common field of view of the camera and the projector; placing the calibration board relatively parallelly twice successively at two upper and lower reference poses to form two reference planes, and collecting images of the calibration board corresponding to the two reference poses through the camera; estimating the external parameters of the camera and the projector of the two reference planes by using the PNP algorithm according to the images of the calibration board, generating a plurality of virtual planes, and calculating the corresponding external parameters of the camera and the projector generated by each virtual plane, and generating corresponding world coordinate data and absolute phase on the reference plane and the virtual plane; calibrating the P3DM model by using the world coordinate data and the absolute phase corresponding to the reference plane and the virtual plane, and finding the corresponding parameters in the P3DM model by using the LUT method to realize three-dimensional reconstruction.

Description

Technical Field

[0001] The present invention relates to the field of calibration, and particularly to a rapid calibration method for a structured light system for industrial sites. Background Art

[0002] With the gradual development of three-dimensional measurement technology based on structured light, vision three-dimensional measurement technology has been widely studied not only in the academic community but also widely applied in the industrial community. However, in the complex industrial site environment, there are still a series of problems in the use of structured light three-dimensional measurement systems, causing difficulties in the practical application of structured light three-dimensional measurement systems. The calibration of structured light systems in industrial sites mainly has the following problems:

[0003] (1) Traditional calibration methods based on binocular stereo vision require time-consuming matching point search during three-dimensional reconstruction, resulting in low measurement efficiency and occupying more computer resources, and are not suitable for detection in industrial sites.

[0004] (2) Traditional calibration methods based on phase height mapping usually require precise platforms or other instruments, leading to an increase in calibration costs. The complex experimental operations and expensive and precise experimental instruments make calibration in industrial sites a big problem.

[0005] (3) In the actual industrial site environment, due to transportation and handling, the external parameters of the calibrated structured light system often change, and the measurement system needs to be recalibrated.

[0006] Currently, regarding how to improve the calibration accuracy and efficiency of structured light systems in industrial site environments and achieve rapid calibration in industrial sites has become a very concerned issue in the industry. Summary of the Invention

[0007] Aiming at the above-mentioned technical problems, the purpose of the embodiments of the present application is to propose a rapid calibration method for a structured light system for industrial sites to solve the technical problems mentioned in the above background art part. Based on the P3DM model, the PNP algorithm is used for pose estimation of the calibration board, and rapid and high-precision system calibration is achieved by only collecting calibration board images at two positions. While ensuring measurement accuracy, the calibration speed is accelerated, making high-precision on-site calibration of structured light systems possible.

[0008] The present invention provides a rapid calibration method for a structured light system for industrial sites, including the following steps:

[0009] Calibrate the internal parameters of the projector and the camera respectively, fix the positions of the projector and the camera, and place the calibration board within the common field of view of the camera and the projector;

[0010] Place the calibration board parallel to each other twice successively at the upper and lower reference poses to form two reference planes, and collect images of the calibration board corresponding to the two reference poses through the camera;

[0011] According to the images of the calibration board, use the PNP algorithm to estimate the external parameters of the camera and the projector for the two reference planes, generate multiple virtual planes, and calculate the corresponding external parameters of the camera and the projector generated by each virtual plane. Generate corresponding world coordinate data and absolute phase on the reference plane and the virtual plane;

[0012] Calibrate the P3DM model using the world coordinate data and absolute phase corresponding to the reference plane and the virtual plane, and use the LUT method to find the corresponding parameters in the P3DM model to achieve three-dimensional reconstruction.

[0013] Preferably, the internal parameters include the focal length, the coordinates of the principal point of the image, and the distortion parameters. The external parameters are the relative positions of the camera or the projector and the world coordinate system in a static scene, including the rotation matrix and the translation vector. The relationship between the internal and external parameters of the camera or the projector is:

[0014]

[0015]

[0016] Among them, the two-dimensional coordinates of the calibration board image collected by the camera are (u c , v c ), the world coordinates corresponding to this pixel are (X c , Y c , Z c ), the corresponding projector pixel coordinates are (u p , v p ), the world coordinates corresponding to this pixel are (X p , Y p , Z p ), f c and f p respectively represent the focal lengths of the camera and the projector, represents the focal length on the horizontal axis of the camera, represents the focal length on the vertical axis of the camera, represents the focal length on the horizontal axis of the projector, represents the focal length on the vertical axis of the projector, (u0, v0) represents the image center coordinates, A c and A p are the internal parameter matrices of the camera and the projector respectively, [R c , T c and [R p , T p respectively represent the external parameter matrices of the camera and the projector.

[0017] Preferably, the internal parameters of the projector and the camera are calibrated respectively, specifically including:

[0018] The Zhang-Zhengyou calibration method is used to calibrate the internal parameters of the camera;

[0019] The projector projects sine gratings in the horizontal and vertical directions onto the calibration board, and the camera is used to collect the images of the calibration board. Among them, the gray values of the sine gratings in the horizontal and vertical directions are respectively expressed as:

[0020]

[0021] where λ u and λ v respectively represent the fringe periods of the sine gratings in the horizontal and vertical directions, represents the set phase shift amount, and respectively represent the gray values of the sine gratings projected in the horizontal and vertical directions;

[0022] The light intensity distribution function on the calibration board image collected by the camera is:

[0023]

[0024] where and represent the light intensities corresponding to the horizontal and vertical sine stripe projections collected by the camera projected onto the calibration board at (u c , v c ). α, a(u c , v c ) and β, b(u c , v c ) respectively represent the encoded pattern and the collected background intensity and modulation intensity;

[0025] The wrapped phase is calculated by the least squares method as shown in the following formula:

[0026]

[0027] The multi-frequency heterodyne method is used to perform phase unwrapping on it to obtain the corresponding absolute phase Φ u (u c , v c ) and Φ v (u c , v c );

[0028] The corresponding projector pixel coordinates are calculated according to the absolute phase as shown in the following formula:

[0029]

[0030] Treat the projector as a reverse camera and calibrate the internal parameters of the projector.

[0031] Preferably, according to the image of the calibration board, use the PNP algorithm to estimate the external parameters of the camera and the projector for two reference planes, generate multiple virtual planes, and calculate the corresponding external parameters of the camera and the projector generated by each virtual plane, specifically including:

[0032] Use the PNP algorithm to estimate the rotation vector and translation vector of the two reference planes relative to the projector and the camera;

[0033] Calculate the difference between the rotation vectors and translation vectors of the two reference planes, as shown in the following formula:

[0034]

[0035] where the subscripts c and p represent the camera coordinate system and the projector coordinate system respectively, and the superscript * represents the difference, rvec top and rvec bottom represent the rotation vectors of the top and bottom of the reference plane respectively, and T top and T bottom represent the translation vectors of the top and bottom of the reference plane respectively;

[0036] Calculate equidistant virtual planes based on the difference between the rotation vectors and translation vectors, as shown in the following formula:

[0037]

[0038] where m is the number of generated virtual planes, rvec i and T i represent the rotation vector and translation vector of the i-th obtained virtual plane, and use the Rodriguez formula to convert the rotation vector rvec i to the rotation matrix R i .

[0039] Preferably, generate corresponding world coordinate data and absolute phase on the reference plane and the virtual plane, specifically including:

[0040] Calculate the world coordinate data on the target plane according to the external parameters of the camera. The target plane includes the reference plane and the virtual plane, and the coordinate mapping relationship is as follows:

[0041]

[0042] where A c represents the internal parameter matrix of the camera, R c represents the rotation matrix in the external parameters of the camera, and T cis the translation vector in the extrinsic parameters of the camera. The formula from the two-dimensional coordinates (u, v) to the three-dimensional coordinates (X w , Y w , Z w ) is as follows:

[0043]

[0044] According to the plane assumption of the calibration board, set Z w = 0, calculate the magnitude of the scale factor s c , and substitute the scale factor s c into the following formula to calculate [X w , Y w :

[0045]

[0046] Convert the target plane coordinates (X w , Y w , Z w ) to the coordinates (X wr , Y wr , Z wr ) in the reference coordinate system, as shown in the following formula:

[0047]

[0048] Among them, the coordinate system of the generated bottommost virtual plane is used as the reference coordinate system, and its corresponding extrinsic parameter is

[0049] The unique camera coordinates (u p , v p ) corresponding to any projector coordinates (u p , v p );

[0050] In the projector space, its absolute phase Φ satisfies the following formula:

[0051] Φ = 2π × u p / t;

[0052] Among them, t represents the period of the fringe phase, and the fringe pattern changes along the sine direction;

[0053] Define the corresponding absolute phase pixel by pixel according to the corresponding camera extrinsic parameters and projector extrinsic parameters generated for each virtual plane.

[0054] Preferably, calibrate the P3DM model using the world coordinate data and absolute phase corresponding to the reference plane and the virtual plane, specifically including:

[0055] According to the derivation of the stereo vision model, the mapping relationship between world coordinates and absolute phase is expressed in the mathematical form of a polynomial as shown in the following formula:

[0056]

[0057] where N represents the order of the polynomial, Φ c is the absolute phase at a certain pixel point on the image plane, (X c , Y c , Z c ) is the world coordinate corresponding to this pixel point, a0~a n , b0~b n and c0~c n are calibration parameters. Substitute the world coordinate data (X wi , Y wi , Z wi , Φ wi ) generated for each virtual plane into the following matrix:

[0058]

[0059] A = [a0 … a n T ;

[0060] X s = [1 / X w1 … 1 / X wm T ;

[0061] According to the above three formulas, the following formula can be obtained:

[0062] X s = PA;

[0063] According to the least squares method, multiply both sides of the formula by the transpose of matrix P to obtain the following formula:

[0064] A = (P T P) -1 P T X s ;

[0065] The factors a0, a1, …, a n of matrix A are calibration mapping coefficients. Similarly, calculate b0, b1, …, b n and c0, c1, …, c n .

[0066] Preferably, use the LUT method to find the corresponding parameters in the P3DM model to achieve three-dimensional reconstruction, specifically including:

[0067] ​Project a sine grating onto the object to be measured and acquire an image, and calculate the corresponding absolute phase Φ obj , use the LUT to obtain the mapping coefficient of the corresponding pixel, substitute the absolute phase into the corresponding mapping coefficient, and obtain the corresponding world coordinate data.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] (1) When the internal parameters of the camera and the projector are known, the present invention only needs to determine the reference plane through two calibration plates to establish the phase-three-dimensional coordinate mapping. The calibration process is simple. Using an artificially generated unambiguous phase map as the sample data for calibration effectively avoids the influence of human factors on the system calibration accuracy, and has a high system calibration accuracy.

[0070] (2) The present invention adopts a fast three-dimensional reconstruction method based on LUT. Compared with the time-consuming matching point search in the traditional binocular stereo vision method, it significantly improves the three-dimensional reconstruction efficiency, and at the same time has the feasibility of using GPU for parallel processing acceleration.

[0071] (3) The present invention uses a checkerboard instead of the high-precision displacement platform required in the phase-height mapping calibration, which is beneficial to reducing the calibration cost.

[0072] (4) In view of the fact that the vibration during the transportation process and the on-site environment in the industrial field application scenario may cause the external parameters of the system to change, the present invention realizes the rapid system calibration in the industrial field when the internal parameters of the camera and the projector are known. Description of the Drawings

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0074] Figure 1 It is a schematic flow chart of the fast calibration method of the structured light system for the industrial field in the embodiment of the present application;

[0075] Figure 2 It is a system calibration flow chart of the fast calibration method of the structured light system for the industrial field in the embodiment of the present application;

[0076] Figure 3 It is a schematic diagram of the projector calibration of the fast calibration method of the structured light system for the industrial field in the embodiment of the present application;

[0077] Figure 4Schematic diagram of the experimental results of the three-dimensional reconstruction accuracy of the structured light system rapid calibration method for industrial field in the embodiment of the present application. a, b, c, and d respectively represent the three-dimensional reconstruction results of the object to be measured at different spatial positions from low to high. Detailed implementation manners

[0078] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0079] Figure 1 A rapid calibration method for a structured light system for industrial field provided by an embodiment of the present application is shown, including the following steps:

[0080] S1. Calibrate the internal parameters of the projector and the camera respectively, fix the positions of the projector and the camera, and place the calibration board within the common field of view of the camera and the projector.

[0081] In a specific embodiment, the internal parameters include the focal length, the coordinates of the principal point of the image, and the distortion parameters, and the external parameters are the relative positions of the camera or the projector and the world coordinate system in a static scene, including the rotation matrix and the translation vector. The relationship between the internal parameters and the external parameters of the camera or the projector is as follows:

[0082]

[0083]

[0084] Among them, the two-dimensional coordinates of the calibration board image collected by the camera are (u c , v c ), the world coordinates corresponding to this pixel are (X c , Y c , Z c ), the corresponding projector pixel coordinates are (u p , v p ), the world coordinates corresponding to this pixel are (X p , Y p , Z p ), f c and f p respectively represent the focal lengths of the camera and the projector, represents the focal length on the horizontal axis of the camera, represents the focal length on the vertical axis of the camera, represents the focal length on the horizontal axis of the projector, represents the focal length on the vertical axis of the projector, (u0, v0) represents the image center coordinates, Ac and A p are the internal parameter matrices of the camera and the projector respectively, and [R c , T c and [R p , T p represent the external parameter matrices of the camera and the projector respectively.

[0085] In a specific embodiment, the internal parameters of the projector and the camera are calibrated respectively, which specifically includes:

[0086] The Zhang-Zhengyou calibration method is used to calibrate the internal parameters of the camera;

[0087] The projector projects sine gratings in the horizontal and vertical directions onto the calibration plate, and the camera is used to collect images of the calibration plate. Among them, the gray values of the sine gratings in the horizontal and vertical directions are respectively expressed as:

[0088]

[0089] Among them, λ u and λ v represent the fringe periods of the sine gratings in the horizontal and vertical directions respectively, represents the set phase shift amount, and represent the gray values of the sine gratings projected in the horizontal and vertical directions respectively;

[0090] The light intensity distribution function on the calibration plate image collected by the camera is:

[0091]

[0092] Among them, and represent the light intensities corresponding to the sine fringes in the horizontal and vertical directions collected by the camera projected onto the calibration plate at (u c , v c ), α, a(u c , v c ) and β, b(u c , v c ) represent the encoded pattern and the collected background intensity and modulation intensity respectively;

[0093] The wrapped phase is calculated by the least squares method as shown in the following formula:

[0094]

[0095] The arctangent function is used in the above formula, and the bar position is periodically truncated at [-π, π]. The multi-frequency heterodyne method is used to expand its phase to obtain the corresponding absolute phase Φ u (uc , v c ), and Φ v (u c , v c );

[0096] Obtain the corresponding projector pixel coordinates according to the absolute phase, as shown in the following formula:

[0097]

[0098] Regard the projector as a reverse camera and calibrate the internal parameters of the projector.

[0099] Specifically, referring to Figure 2 , this method uses a CCD camera and a DLP projector. In the whole calibration process, the PNP algorithm is first used to estimate the external parameters of two reference planes and generate corresponding virtual planes. Phase and corresponding world coordinate data are generated on the two reference planes and the generated virtual planes to realize the calibration of the fitting parameters of the P3DM model. The whole calibration process only needs to collect two sets of calibration plate pictures, and the process is simple, flexible and efficient. Referring to Figure 3 , since the projector cannot directly obtain the feature points in space, the projection of the projector can be regarded as an inverse imaging process. By calculating the orthogonal absolute phase, the corresponding relationship between the camera pixel coordinates and the projector pixel coordinates is established, and the projector is calibrated in the same way as the camera calibration.

[0100] Specifically, first complete the internal parameter calibration of the projector and the camera under laboratory conditions. Since the orthogonal absolute phase can provide the corresponding position relationship from the camera to the DMD chip of the projector, the projection of the projector can be regarded as an inverse imaging process. Both the camera and the projector are based on the pinhole model, and their calibration mainly includes the calibration of internal parameters and external parameters. Among them, the internal parameters include the focal length, the coordinates of the principal point of the image, and the distortion parameters, while the external parameters describe the relative position of the camera or the projector and the world coordinate system in a static scene, including the rotation matrix and the translation vector. The relationship between the two can be described according to the pinhole imaging model. According to the Zhang Zhengyou calibration method, the camera can complete the calibration by capturing the images of the calibration plate in different poses and extracting its characteristic corner points. Regard the projection of the projector as an inverse imaging process. By calculating the orthogonal absolute phase to establish the corresponding relationship between the camera pixel coordinates and the projector pixel coordinates, the internal parameter calibration of the projector can be realized. The internal parameter calibration under laboratory conditions only needs to be carried out once. The internal and external parameters of the camera and the projector are calibrated separately to solve the problem that the external parameters of the system are likely to change and need to be recalibrated in industrial field applications.

[0101] Furthermore, a low-cost chessboard calibration plate is used. Adjust the position of the chessboard calibration plate so that it is within the field of view and depth of field, and ensure that the corner points of the calibration plate are clear.

[0102] S2. Place the calibration board parallel to each other twice successively at the upper and lower reference poses to form two reference planes, and collect the images of the calibration board corresponding to the two reference poses through the camera.

[0103] Specifically, place the checkerboard parallel to each other twice successively at the upper and lower reference poses, and use the camera to collect the corresponding checkerboard corner points and stripe patterns. Use a low-cost checkerboard to replace the traditional high-precision platform, which significantly reduces the calibration cost and ease of use while ensuring the calibration accuracy. The entire calibration process only needs to collect the calibration board patterns of two poses, thus shortening the calibration time and reducing the operation difficulty of workers for calibration.

[0104] S3. According to the images of the calibration board, use the PNP algorithm to estimate the external parameters of the camera and the projector for the two reference planes, generate multiple virtual planes, and calculate the corresponding external parameters of the camera and the projector generated by each virtual plane, and generate the corresponding world coordinate data and absolute phase on the reference plane and the virtual plane.

[0105] In a specific embodiment, according to the images of the calibration board, use the PNP algorithm to estimate the external parameters of the camera and the projector for the two reference planes, generate multiple virtual planes, and calculate the corresponding external parameters of the camera and the projector generated by each virtual plane, specifically including:

[0106] Use the PNP algorithm to estimate the rotation vectors and translation vectors of the two reference planes relative to the projector and the camera;

[0107] Calculate the difference between the rotation vectors and translation vectors of the two reference planes, as shown in the following formula:

[0108]

[0109] Among them, the subscripts c and p represent the camera coordinate system and the projector coordinate system respectively, and the superscript * represents the difference, rvec top and rvec bottom represent the rotation vectors of the top and bottom of the reference plane respectively, and T top and T bottom represent the translation vectors of the top and bottom of the reference plane respectively;

[0110] Calculate the equidistant virtual planes according to the difference between the rotation vectors and translation vectors, as shown in the following formula:

[0111]

[0112] Among them, m is the number of generated virtual planes, rvec i and T iThe rotation vector and translation vector representing the obtained i-th virtual plane, and using Rodrigues' formula to convert the rotation vector rvec i into the rotation matrix R i .

[0113] Specifically, on the basis of obtaining the internal parameters of the camera and the projector by using Zhang Zhengyou calibration method, the PNP algorithm is further adopted to estimate the poses of two relatively parallel checkerboard calibration plates with respect to the projector and the camera, including the rotation vector and the translation vector, so as to generate a plurality of virtual planes and corresponding artificial phase diagrams and world coordinate data (three-dimensional data) for the calibration of a high-precision phase-three-dimensional coordinate mapping model. Generating virtual planes can avoid the influence of artificially placing calibration plates on the calibration accuracy. The PNP algorithm is an algorithm that uses a set of three-dimensional points in object space and their corresponding two-dimensional coordinate points in the image to solve their position correspondence relationship with the camera. Through the PNP algorithm, the pose of the calibration plate reference plane relative to the camera or the projector can be obtained. In order to generate the pose of the virtual plane, it is necessary to calculate the difference between the rotation vectors and translation vectors of the two reference planes.

[0114] In a specific embodiment, corresponding world coordinate data and absolute phase are generated on the reference plane and the virtual plane, specifically including:

[0115] Calculate the world coordinate data on the target plane according to the external parameters of the camera. The target plane includes the reference plane and the virtual plane, and the coordinate mapping relationship is as follows:

[0116]

[0117] Among them, A c represents the internal parameter matrix of the camera, R c represents the rotation matrix in the external parameters of the camera, T c is the translation vector in the external parameters of the camera. The formula from two-dimensional coordinates (u, v) to three-dimensional coordinates (X w , Y w , Z w ) is as follows:

[0118]

[0119] According to the plane assumption of the calibration plate, let Z w = 0, calculate the magnitude of the scale factor s c , and substitute the scale factor s c into the following formula to calculate [X w , Y w :

[0120]

[0121] Substitute the target plane coordinates (X w, Y w , Z w ) is converted to the coordinates in the reference coordinate system (X wr , Y wr , Z wr ), as shown in the following formula:

[0122]

[0123] Among them, the coordinate system of the generated bottommost virtual plane is used as the reference coordinate system, and its corresponding external parameters are

[0124] For any projector coordinates (u p , v p ), the corresponding unique camera coordinates (u p , v p );

[0125] In the projector space, its absolute phase Φ satisfies the following formula:

[0126] Φ = 2π × u p / t;

[0127] Among them, t represents the period of the fringe phase, and the fringe pattern changes along the sine direction;

[0128] According to the corresponding camera external parameters and projector external parameters generated for each virtual plane, the corresponding absolute phase is defined pixel by pixel.

[0129] Specifically, the internal parameter A C has been calibrated in advance using the Zhang Zhengyou calibration method, and [R c , T c has also been calculated. Only the scale factor s c is unknown and needs to be further solved. It should be noted that the world coordinate systems of the calculated three-dimensional coordinates are respectively on their corresponding calibration plate planes. At this time, Z W is equal to 0, and the three-dimensional coordinates need to be converted to the same reference coordinate system. Since the camera coordinate system is fixed, there is a one-to-one correspondence between the target plane coordinates (X w , Y w , Z w ) and the reference coordinate system coordinates (X wr , Y wr , Z wr ). In the actual use process, the coordinate system of the generated bottommost virtual plane is used as the reference coordinate system, and its corresponding external parameters are Therefore, in the solving process, first solve the three-dimensional coordinates (X w, Y w , Z w ), and convert it to the reference coordinate system. At this time, the coordinates are (X wr , Y wr , Z wr ). For each virtual plane, the corresponding extrinsic camera parameters and the extrinsic projector parameters can define the corresponding artificial absolute phase pixel by pixel. Using the pose of the checkerboard estimated by the PNP algorithm, generate virtual planes and the corresponding absolute phase and three-dimensional data, reducing human interference and improving the calibration accuracy of the system.

[0130] S4. Calibrate the P3DM model using the world coordinate data and absolute phase corresponding to the reference plane and the virtual plane, and use the LUT method to find the corresponding parameters in the P3DM model to achieve three-dimensional reconstruction.

[0131] In a specific embodiment, calibrate the P3DM model using the world coordinate data and absolute phase corresponding to the reference plane and the virtual plane, specifically including:

[0132] According to the derivation of the stereo vision model, represent the mapping relationship between the world coordinates and the absolute phase in the mathematical form of a polynomial as shown in the following formula:

[0133]

[0134] where N represents the order of the polynomial, Φ c is the absolute phase at a certain pixel point on the image plane, (X c , Y c , Z c ) is the world coordinate corresponding to this pixel point, a0 to a n , b0 to b n and c0 to c n are calibration parameters. Substitute the world coordinate data (X wi , Y wi , Z wi , Φ wi ) generated for each virtual plane into the following matrix:

[0135]

[0136] A = [a0 … a n T ;

[0137] X s = [1 / X w1 … 1 / X wm T ;

[0138] ​​Based on the above three equations, the following formula can be obtained:

[0139] X s = PA;

[0140] According to the least squares method, multiply both sides of the formula by the transpose of matrix P to obtain the following equation:

[0141] A = (P T P) -1 P T X s ;

[0142] The factors a0, a1, …, a n of matrix A are calibrated mapping coefficients. Similarly, b0, b1, …, b n and c0, c1, …, c n are calculated.

[0143] Specifically, according to the derivation of the stereo vision model, the mapping relationship between the world coordinates and the absolute phase can be expressed in the mathematical form of a polynomial. The non-linear characteristics of the polynomial can effectively reduce the influence of lens distortion. Define m as the number of virtual planes. To reduce the fitting error, as many virtual planes as possible should be generated to obtain higher calibration accuracy.

[0144] In a specific embodiment, the LUT method is used to find the corresponding parameters in the P3DM model to achieve three-dimensional reconstruction, which specifically includes:

[0145] Project a sinusoidal grating onto the object to be measured and collect images, calculate the corresponding absolute phase Φ obj , use the LUT to obtain the mapping coefficients of the corresponding pixels, and substitute the absolute phase into the corresponding mapping coefficients to obtain the corresponding world coordinate data.

[0146] Specifically, through pixel-by-pixel polynomial fitting, a mapping coefficient LUT is finally built, and the LUT method can be used to achieve efficient three-dimensional reconstruction. The process is as follows: First, project a sinusoidal grating onto the object to be measured and collect images, calculate the corresponding absolute phase Φ obj , use the LUT to find the mapping coefficients of the corresponding pixels, and substitute the absolute phase and the corresponding mapping coefficients into the following formula to obtain the corresponding X, Y, and Z coordinates:

[0147]

[0148] Use P3DM to establish the mapping relationship between the phase and the three-dimensional coordinates, and use the LUT algorithm for efficient three-dimensional reconstruction to achieve fast and high-precision three-dimensional reconstruction. Using the LUT-based P3DM technology to achieve three-dimensional reconstruction avoids the time-consuming matching point search, can further improve the measurement speed, and thus achieves the purpose of real-time measurement.

[0149] Reference Figure 4 , a ceramic standard block with a standard height difference of 0.2 mm was placed at different positions in space for measurement, and the average height differences obtained were 0.2067 mm, 0.2045 mm, 0.2020 mm, and 0.2023 mm respectively, verifying the high precision of the calibration method proposed by the present invention.

[0150] The above description is only a preferred embodiment of the present application and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solution formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present application.

Claims

1. A rapid calibration method for a structured light system for industrial sites, characterized in that, The steps include: Calibrate the internal parameters of the projector and the camera respectively. Fix the positions of the projector and the camera, and place the calibration board within the common field of view of the camera and the projector. Place the calibration board relatively parallel twice successively at two upper and lower reference poses to form two reference planes, and collect images of the calibration board corresponding to the two reference poses through the camera. Estimate the external parameters of the camera and the projector of the two reference planes using the PNP algorithm based on the images of the calibration board, generate multiple virtual planes, and calculate the corresponding external parameters of the camera and the projector generated by each virtual plane. Generate corresponding world coordinate data and absolute phase on the reference plane and the virtual plane. The internal parameters include focal length, principal point coordinates, and distortion parameters. The external parameters are the relative positions of the camera or the projector and the world coordinate system in a static scene, including a rotation matrix and a translation vector. The relationship between the internal parameters and the external parameters of the camera or the projector is as follows: Among them, the two-dimensional coordinates of the calibration board image collected by the camera are (u c , v c ), and the corresponding world coordinates are (X c , Y c , Z c ). The corresponding projector pixel coordinates are (u p , v p ), and the corresponding world coordinates are (X p , Y p , Z p ). The corresponding three-dimensional coordinates are (X w , Y w , Z w ). f c and f p respectively represent the focal lengths of the camera and the projector. represents the focal length on the horizontal axis of the camera, and f c v represents the focal length on the vertical axis of the camera. represents the focal length on the horizontal axis of the projector. represents the focal length on the vertical axis of the projector. (u0, v0) represents the image center coordinates. A c and A p are respectively the internal parameter matrices of the camera and the projector. [R c , T c and [R p , T p respectively represent the external parameter matrices of the camera and the projector; Calibrate the P3DM model using the world coordinate data and absolute phase corresponding to the reference plane and the virtual plane, and use the LUT method to find the corresponding parameters in the P3DM model to achieve three-dimensional reconstruction.

2. The rapid calibration method of the structured light system for industrial field according to claim 1, wherein The step of calibrating the internal parameters of the projector and the camera respectively specifically includes: Calibrate the internal parameters of the camera using the Zhang Zhengyou calibration method. Project sine gratings in two horizontal and vertical directions onto the calibration board through the projector, and use the camera to collect images of the calibration board. Among them, the gray values of the sine gratings in the two horizontal and vertical directions are respectively expressed as: where, λ u and λ v represent the fringe periods of the sine gratings in the horizontal and vertical directions respectively, represents the set phase shift amount, and represent the gray values of the sine gratings projected in the horizontal and vertical directions respectively; The light intensity distribution function on the image of the calibration board collected by the camera is: Among them, and represent the light intensities corresponding to the sine stripe projections in the horizontal and vertical directions collected by the camera onto the calibration board at (u c , v c ). α, a(u c , v c ) and β, b(u c , v c ) represent the encoded pattern and the collected background intensity and modulation intensity respectively; Calculate the wrapped phase through the least squares method, as shown in the following formula: Perform phase unwrapping on it using the multi-frequency heterodyne method to obtain the corresponding absolute phase Φ u (u c , v c ) and Φ v (u c , v c ); Calculate the corresponding projector pixel coordinates according to the absolute phase, as shown in the following formula: Regard the projector as an inverse camera and calibrate the internal parameters of the projector.

3. The rapid calibration method of the structured light system for industrial field according to claim 1, characterized in that The step of estimating the external parameters of the camera and the projector of the two reference planes using the PNP algorithm based on the images of the calibration board, generating multiple virtual planes, and calculating the corresponding external parameters of the camera and the projector generated by each virtual plane specifically includes: Estimate the rotation vectors and translation vectors of the two reference planes relative to the projector and the camera using the PNP algorithm. Calculate the difference between the rotation vectors and translation vectors of the two reference planes, as shown in the following formula: where the subscripts c and p represent the camera coordinate system and the projector coordinate system respectively, and the superscript * represents the difference, rvec top and rvec bottom represent the rotation vectors of the top and bottom of the reference plane respectively, T top and T bottom represent the translation vectors of the top and bottom of the reference plane respectively; Calculate equidistant virtual planes according to the difference between the rotation vectors and translation vectors, as shown in the following formula: where m is the number of generated virtual planes, rvec i and T i represent the rotation vector and translation vector of the obtained i-th virtual plane, and the rotation vector rvec i is converted to a rotation matrix R i .

4. The rapid calibration method of the structured light system for industrial field according to claim 1, characterized in that, The step of generating corresponding world coordinate data and absolute phase on the reference plane and the virtual plane specifically includes: Calculate the world coordinate data on the target plane according to the external parameters of the camera. The target plane includes the reference plane and the virtual plane, and the coordinate mapping relationship is as follows: Among them, A c represents the internal parameter matrix of the camera, R c represents the rotation matrix in the external parameters of the camera, T c is the translation vector in the external parameters of the camera. The formula from two-dimensional coordinates (u, v) to three-dimensional coordinates (X w , Y w , Z w ) is as follows: Based on the planar assumption of the calibration board, set Z w = 0, and calculate the size of the scale factor s c . Then substitute the scale factor s c into the following formula to calculate [X w , Y w : Convert the target plane coordinates (X w , Y w , Z w ) to the coordinates (X wr , Y wr , Z wr ) in the reference coordinate system as shown in the following formula: Among them, the coordinate system of the generated bottommost virtual plane is used as the reference coordinate system, and the corresponding external parameters are The unique camera coordinates (u p , v p ) corresponding to any projector coordinates (u p , v p ); In the projector space, its absolute phase Φ satisfies the following formula: Φ = 2π × u p / t; Among them, t represents the period of the fringe phase, and the fringe pattern changes along the sine direction. The corresponding extrinsic camera parameters generated according to each virtual plane and the extrinsic projector parameters Define the corresponding absolute phase pixel by pixel.

5. The rapid calibration method of the structured light system for industrial field according to claim 1, characterized in that The step of calibrating the P3DM model using the world coordinate data and absolute phase corresponding to the reference plane and the virtual plane specifically includes: According to the derivation of the stereo vision model, the mapping relationship between world coordinates and absolute phase is expressed in the mathematical form of a polynomial as shown in the following formula: where N represents the order of the polynomial, and Φ c is the absolute phase at a certain pixel point on the image plane. (X c , Y c , Z c ) is the world coordinate corresponding to this pixel point, a0 to a n , b0 to b n and c0 to c n are calibration parameters. Substitute the world coordinate data (X wi , Y wi , Z wi , Φ wi ) of each generated virtual plane into the following matrix: A = [a0…a n T ;​ X s = [1 / X w1 …1 / X wm T ;​ Based on the above three formulas, the following formula can be obtained: X s = PA; According to the least squares method, multiply both sides of the formula by the transpose of matrix P to obtain the following formula: A = (P T P) -1 P T X s ; The factors a0, a1, …, a of matrix A n are calibrated mapping coefficients. Similarly, b0, b1, …, b n and c0, c1, …, c n are calculated.

6. The rapid calibration method of the structured light system for industrial field according to claim 1, characterized in that The method of using the LUT method to find the corresponding parameters in the P3DM model to achieve three-dimensional reconstruction specifically includes: Project a sinusoidal grating onto the object to be measured and acquire an image, and calculate the corresponding absolute phase Φ obj , use the LUT to obtain the mapping coefficient of the corresponding pixel, substitute the absolute phase into the corresponding mapping coefficient, and obtain the corresponding world coordinate data.

Images